ML{.NET} Image Classification

One of the best scenes from Silicon Valley is Jian Yang demoing his “Hotdog, not hotdog” application. In this article, we will build our own “Hotdog, not hotdog” solution using ML.NET. After all, who would not want to determine if that dish is, or is not a hot dog? Just take a picture, upload it to the web or desktop application, and get results with almost 90% certainty in a second.

Although some may say this is not a very useful application, it is a fun way to explore another machine learning concept through ML.NET. I covered installing and getting started with ML.NET in Visual Studio in my previous article, so refer to it if you missed it.

ML{.NET} Image Classification

Get the Data

For this example, we will use the dataset with the pictures of food organized in hot_dog and not_hot_dog folders from here. It is the subset of the full dataset that can be found here. The acknowledgments for the datasets go to the original authors. For any usage beyond scientific or educational, please contact them.

Build the Model in ML.NET Model Builder

The first step is to add ML.NET to our project. To do that, right-click the project > Add > Machine Learning.

Adding ML .NET to our project

This will open ML.NET Model Builder. Here, we will select the Image Classification option.

Choosing Image Classification in ML.NET Model Builder

The next step is the training environment selection. We will use the local machine.

select training environment

Now we need to add the data. Select a folder with the folders hot_dog and not_hot_dog.

adding data to training environment

After selecting the target folder, the data preview shows how many images there are in each category folder.

data preview shows how many images there are in each category folder

We can proceed and train the model.

train the model
Model training

Model training took about 10 minutes. In the image below, we can see that the best accuracy achieved was about 90%. It will be fine for our example. For the remaining 10%, we will need to use the old-fashioned tools – look at and taste the dish to know if it is a hot dog or not.

the model is trained

Now we can test and evaluate our model. Select an image and verify if it is recognized correctly and the percentage of certainty.

testing and evaluating the model
results of training for the model

As we can see, the more complicated or unclear the image is, the less certain the result is. Still, for most images, it does a pretty good job at detection.

results of training for the model
results of training for the model
results of training for the model
results of training for the model

The next step in the Model builder will automatically add new projects to the solution.

Model builder will automatically add new projects to the solution
Model builder will automatically add new projects to the solution

Use the Model

To give a full tribute to the original Jian Yang’s app, I’ve put together a desktop version of it.

putting together a desktop version of the app

Here is the method that does the detection using our new ML model.

private bool IsHotDog(string path)
        {
            // Create single instance of sample data from first line of dataset for model input
            ModelInput sampleData = new ModelInput()
            {
                ImageSource = path
            };
            // Make a single prediction on the sample data and print results
            var predictionResult = ConsumeModel.Predict(sampleData);

            var isHotDog = predictionResult.Prediction.Equals("hot_dog");
            if (isHotDog)
                this.Text = string.Format("{1}% Hotdog", predictionResult.Prediction, (int)(predictionResult.Score[0] * 100));
            else
                this.Text = string.Format("{1}% Not hotdog", predictionResult.Prediction, (int)(predictionResult.Score[1] * 100));
            return isHotDog;
        }
method that does the detection using our new ML model

Conclusion

I think it is fair to say that ML.NET proved to be simple and easy to use. It is hard to make the whole process more straightforward than it already is.

ML.NET seems to be a perfect tool for building simple projects like this one. Apart from the simple projects we have covered so far, we will try to solve some real-life scenarios, using machine learning knowledge and ML.NET in one of the next articles.

Thank you for sticking to the end and hope you will join us in the future to explore more exciting cases.

Ivan Matec
Latest posts by Ivan Matec (see all)

Ivan Matec

Ivan Matec is experienced Software Engineer with a demonstrated history of working in the information technology and services industry. Skilled in Microsoft technologies (C#, VB.NET, Azure) and front-end technologies (jQuery, Angular). Strong engineering professional with a bacc. ing. comp. focused in Computer Programming from Polytechnic of Zagreb. Other professional interests include computer security and AI development.

Leave a Reply

Your email address will not be published. Required fields are marked *