Machine Learning {.NET} Image Classification

One of the best scenes from Silicon Valley is Jian Yang demoing his “Hotdog, not hotdog” application. In this article, we will build our own “Hotdog, not hotdog” solution using ML.NET. After all, who would not want to determine if that dish is, or is not a hot dog? Just take a picture, upload it to the web or desktop application, and get results with almost 90% certainty in a second.

Although some may say this is not a very useful application, it is a fun way to explore another machine learning concept through ML.NET. I covered installing and getting started with ML.NET in Visual Studio in my previous article, so refer to it if you missed it.

Get the Data

For this example, we will use the dataset with the pictures of food organized in hot_dog and not_hot_dog folders from here. It is the subset of the full dataset that can be found here. The acknowledgments for the datasets go to the original authors. For any usage beyond scientific or educational, please contact them.

Build the Model in ML.NET Model Builder

The first step is to add ML.NET to our project. To do that, right-click the project > Add > Machine Learning.

This will open ML.NET Model Builder. Here, we will select the Image Classification option.

The next step is the training environment selection. We will use the local machine.

Now we need to add the data. Select a folder with the folders hot_dog and not_hot_dog.

After selecting the target folder, the data preview shows how many images there are in each category folder.

We can proceed and train the model.

Model training took about 10 minutes. In the image below, we can see that the best accuracy achieved was about 90%. It will be fine for our example. For the remaining 10%, we will need to use the old-fashioned tools – look at and taste the dish to know if it is a hot dog or not.

Now we can test and evaluate our model. Select an image and verify if it is recognized correctly and the percentage of certainty.

As we can see, the more complicated or unclear the image is, the less certain the result is. Still, for most images, it does a pretty good job at detection.

The next step in the Model builder will automatically add new projects to the solution.

Use the Model

To give a full tribute to the original Jian Yang’s app, I’ve put together a desktop version of it.

Here is the method that does the detection using our new ML model.

private bool IsHotDog(string path)
        {
            // Create single instance of sample data from first line of dataset for model input
            ModelInput sampleData = new ModelInput()
            {
                ImageSource = path
            };
            // Make a single prediction on the sample data and print results
            var predictionResult = ConsumeModel.Predict(sampleData);

            var isHotDog = predictionResult.Prediction.Equals("hot_dog");
            if (isHotDog)
                this.Text = string.Format("{1}% Hotdog", predictionResult.Prediction, (int)(predictionResult.Score[0] * 100));
            else
                this.Text = string.Format("{1}% Not hotdog", predictionResult.Prediction, (int)(predictionResult.Score[1] * 100));
            return isHotDog;
        }

Conclusion

I think it is fair to say that ML.NET proved to be simple and easy to use. It is hard to make the whole process more straightforward than it already is.

ML.NET seems to be a perfect tool for building simple projects like this one. Apart from the simple projects we have covered so far, we will try to solve some real-life scenarios, using machine learning knowledge and ML.NET in one of the next articles.

Thank you for sticking to the end and hope you will join us in the future to explore more exciting cases.