Face Mask Detector with ML5.js
In this article, we will cover how to build a simple image classification using the ml5.js library to detect whether you are wearing a mask or not. You can also re-train the model to classify the new classes (for example, whether it is a cat or dog, holding a phone or not, etc) on your demand.
TLDR; here is the demo.
Terminology
We will be mainly using ml5.js in our application. So what is ml5.js?
ml5.js
ml5.js is the machine learning library in your web browser. It is built on top of tensorflow.js in which all the heavy-lifting or low-level tasks regarding machine learning are done in tensorflow.js. It is made for those who have no or little knowledge about machine learning.
Image Classification
In the domain of machine learning or neural network, we are often talking about two popular tasks that machine learning is very good at it. They are classification and regression tasks or problems. In this tutorial, we will be exploring the classification problem of machine learning. To be specific, image classification, given an input of image, we ask the machine to classify what is the class or category of this image.
MobileNet
To build it from scratch, we will have to train the machine by providing tons of images with the corresponding label. Design hidden layer, activation function, loss function, this, and that. Train it then tune it by changing the hyperparameters until the loss is low or accuracy is high.
Sounds like a give-up, isn’t it? Luckily for us, we can just use a pre-trained model called MobileNet.
MobileNet is a machine learning model trained to recognize the content of certain images.
Great, sound good. Well, there’s a catch. MobileNet could only recognize 1000 classes/categories. You can look it up here. It could recognize whether it is goldfish, stingray, cat, kite, all the way to toilet tissue. You know what? it couldn’t recognize a “person”
What does it mean?
It means that if you give the machine a picture of a person, machine doesn’t know what it is since the machine has never seen a person before.
FeatureExtractor
That’s why Feature Extractor comes to the rescue. The idea is to use the last layer of the neural network and map it to the new classes/categories (for example, a person wearing a mask or not). This way, we don’t have to care much about how we should train the model, adjust the hyperparameter, this, and that. This is known as Transfer Learning. And as you might guess, ml5 even simplifies it for us. You will find out soon how easy it is later when we touch the code. I recommend this video to find out know about feature extractor. He explain it funny clear.
Time to Code
First, let’s create index.html as following.
We insert the ml5 library to the head tag. At this time of writing, we are using version 0.6.1. In the body, we insert a video element. This video element is used for training new data as well as predicting. Going down, we have two buttons Add Class 1 and Add Class 2. These buttons are used to train new data to the model. Then we have the Train button to train or retrain the model. And finally, the Start Detecting button to start predicting whatever images show to the video camera.
Next, let’s write main.js piece by piece.
Start with declaring variables and grab all the DOM elements.
Next,
In this snippet, on line 6, we define the source of the video element as coming from the computer video camera. On line 11, we create FeatureExtractor object from the “MobileNet” model. once the model is loaded, modelLoaded function will be called. There you can do more stuff as you want such as to enable button. In this example, we simply just console log to the browser.
In the next line (on line 12), from the FeatureExtractor object, we create the Classification object in which the input source is the video element. This would mean that whatever we show to the camera will act as the input to the classification object.
Next,
To train the model to recognize this is a picture of you wearing a mask, for example, we need to give the machine a bunch of examples of you wearing the mask with a different pose, angle, or different mask color. With ml5, it just needs one line classifier.addImage('with_mask'). Since we already hook the video element in line 12 of the previous code, the classification will capture the current frame of video and put it in the category of with_mask category. In this case, to train a machine to recognize you wearing a mask, you can show your face to the camera wearing a mask then click on Add Class 2 button. Then, turn your face a little bit left and click on Add Class 2 button again. repeat it clicking the button with the same pose or different angle about 20 times.
After that, doing the same, but this time not wearing a mask and click on Add Class 1 button.
Next,
We call train function to train the model with the images we added earlier. Once the training start, we display the value of lossValue to the DOM. You can observe how the loss starts to decrease until a very small value (reaching 0). The lower the loss, the greater accuracy. The training process is complete once the lossValueis null .
Finally,
To start predicting or classifying, we call the classify function. It accepts the gotResults callback function. In the returned function, it returns results object contains two elements. It is sorted by the confidence property. So we output the first element with label and confidence value to the DOM. Then we call classify function again on line 7 to keep detecting.
Bonus
Once you are feeling good about your trained model, call featureExtractor.save() to save your model and use featureExtractor.load('model.json') to load model.json if you save it at the project root.
All the source code can be found here — https://github.com/yong-asial/ml5-feature-extractor
Conclusion
In this tutorial, we’ve seen how to use ml5.js library to classify the image by re-train the MobileNet model using Transfer Learning method. ml5.js made it easy for us. So shout out to them for making our life easier.
Happy Coding!
