Finger Detection Movement Project

Object detection is a form of machine learning that utilizes computer vision and deep learning techniques. It can be used to identify targeted objects in images and videos with precise accuracy. For detecting hand gestures, the method most commonly used is computer vision. When using computer vision, there are some factors to consider that could impact the results of the experiment. Due to this, a variety of different techniques were developed in order to mitigate the significance those factors have on the data. This paper will present one technique to use when creating a machine to detect hand gestures

Hello, my name is Justin Matthews, and for the Summer Team Impact Program I worked on this project titled Finger Detection Movement.

For some background knowledge, Machine learning is “a field of study that gives computers the ability to learn without being explicitly programmed”. The origin of machine learning begins with a psychologist named Frank Rosenblatt from Cornell University. He based the design of the machine on the human nervous system. The machine was named “perceptron” and its purpose was to recognize the letters of the alphabet. A task that falls under machine learning is called object detection. Object detection is a form of machine learning that utilizes computer vision and deep learning techniques. It can be used to identify targeted objects in images and videos with precise accuracy. For my project, I created a machine to detect hand gestures during a real time video streamed from a webcam.

Materials that I used for this project include: Visual Studio Code, Anaconda Prompt, Tensorflow, Python, and OpenCV. Visual Studio Code was used to edit my code. Anaconda Prompt was used to run my code. Tensorflow in combination with Keras, was used to save and load the data of the model that was created and trained. Python was used to code the project. Lastly, OpenCV was used to access my laptop’s webcam and display real time video coming from it.

The next portion of this video will be a demonstration on how my program works. What you are seeing now is Anaconda Prompt. This is the normal set up I do before starting to code, test, etc. What I am doing is going into the proper directory and checking to make sure I am on the right path. You can see that I have a folder named dataset and when I enter and display the contents, notice how it is empty. That is because we have to populate that folder which is what my “builDataset” script does. I have to type python *script name* *name I want the image folder to be* *# of images*. Unfortunately I am unable to record the window that appears but as you can see by the next line, 300 images are saved. Once I open my dataset folder you can see that a new folder title Zero appears and in it are all the photos that were taken. I repeat this process for the rest of the numbers (1-5). Once all of that is done it is now time to build and train the model. I simply run the command that you see there and a bunch of text that you can ignore appears. You can see that a loading bar will appear and next to it loss and accuracy. Accuracy is pretty self explanatory and that just means how accurate your model is at guessing the finger position and loss is how much data was lost during that training period. In total there are 10 total training periods each taking roughly 40 seconds each. But with the power of movie magic we can cut all of that. Once the training is complete a graph of the accuracy and lost will be displayed (but my recording software couldn’t record that). Now we can test to see if the model is accurate. First I’ll be testing if it can detect the number three which is does successfully. Let’s try it again but with a different photo. Again success. Now let’s see if it can detect zero. Success once again

These are the graphs that display after the training is complete. On the left is the accuracy graph and on the right is the loss graph. As the accuracy of the model increases, the data loss during training decreases. This means that accuracy and loss have an inverse relationship with one another.

This next screenshot is the window that appears after running the buildDataset script. The way it works is simple as you hold up your hand so that it is within the parameter of the rectangle. You then press the ‘a’ key to start taking screenshots. To pause the screenshot process you just hit the ‘a’ key again. To exit the window early, just press the ‘q’ button. Once the specified number of screenshots are taken, the window will automatically close.

Thank you for your time and thanks to Mason and the OSCAR office for this opportunity. If you want to know more about my project you can visit the following websites: https://detect.onmason.com/