Description
In this project I’m going to make a Gesture/Finger Brightness controller of a LED with the help of media-pipe library . For this will require software and some hardware components and here I’m using C and python because Arduino code is written in C and the sending code that is which is communicating with Arduino is written in Python so let’s move further and make this project, below are the list of component and software which you will required in this project.
Hardware Components & Software
Components
[1] Arduino
[2] LED
[3] Jumper Wires
[4] USB connector
Software
Arduino Code & its working
const int redPin = 3;
void setup()
{
Serial.begin(9600);
pinMode(redPin, OUTPUT);
}
void loop()
{
while (Serial.available() > 0){
int red = Serial.parseInt();
if (Serial.read() == '\n')
{
red = constrain(red, 0, 255);
analogWrite(redPin, red);
Serial.print(red);}
}
}
Above code is for Arduino , this code is receiving the data from laptop/PC through serial communication for this I’m using Serial communication that is UART. Let’s understand how code is working?
int red = Serial.parseInt(); if (Serial.read() == '\n')
This above line of code is responsible for receiving the data serially which is sending through Python code . which I have written above. And as you saw we used ‘\n’ for to send the data and here this code of line will receive data when this character receive. After this data is collecting in this variable ‘red’ .
Below code of block is here to set the data according to the received value and this value is in the range of 0-255 as Arduino is a 8 bit microcontroller. Here variable ‘red’ collecting values from PC and setting the value and then this value sending through serial communication.
red = constrain(red, 0, 255);
analogWrite(redPin, red);
Serial.print(red);}
Now copy this Arduino code and paste it in your Arduino IDE but before that connect circuit in this mannner as given in the below circuit-diagram
Now upload the code in Arduino after selection of Arduino board and port.
Python Code & its working
Let’s understand how it’s working. Here I’m using ‘cvzone’ .This is a Computer vision package that makes its easy to run Image processing and AI functions. At the core it uses OpenCV and Mediapipe libraries, and this library has many inbuilt function in it for this project I am using Hand Tracking function from that. This function has many features in it. For this function we will use this command to import this as given below.
Here this hand detector has two parameters , first one is ‘detectionCon’ which vary from 0 to 1 and it indicates how much you want to make it precise when you select 1 then it will detect hand if your hand is clearly visible in the camera or I can say this will give you high efficiency so select according to your requirement. Here I have selected as 0.8 because it was giving me a perfect output at the time of making so I have selected this value.
HandDetector(detectionCon=0.8, maxHands=1)
And another parameter is ‘maxhands’ this has 1 or 2 values, that is if you write 1 here you can able to detect the one hand only and if you write here 2 then you can detect two hand from the camera.
Python Code :-
import serial
import cvzone
import math
import cv2
from cvzone.HandTrackingModule import HandDetector
serialcomm = serial.Serial('COM5', 9600)
serialcomm.timeout = 1
cap=cv2.VideoCapture(0)
detector = HandDetector(detectionCon=0.8, maxHands=1)
l=[]
while True:
success,img =cap.read()
img=cv2.resize(img,(500, 500))
img=detector.findHands(img)
l,box = detector.findPosition(img,draw=False)
if l:
#f=detector.fingersUp()
x1=l[4][0]
y1=l[4][1]
x2=l[8][0]
y2=l[8][1]
cv2.circle(img,(x1,y1),7,(0,255,255),1)
cv2.circle(img,(x2,y2),7,(0,255,255),1)
cv2.line(img,(x1,y1),(x2,y2),(0,255,0),2)
d=int(math.sqrt(math.pow(x2 - x1, 2)+math.pow(y2 - y1, 2) * 1.0))
d=int((d/110)*255)
e='\n'
if 0<d<256:
cv2.putText(img,str(d),(20,30),cv2.FONT_HERSHEY_COMPLEX,.7,(255,255,255),1)
serialcomm.write(str(d).encode())
serialcomm.write(e.encode())
cv2.imshow('Image',img)
if cv2.waitKey(20) & 0xFF == 27:
break
cv2.destroyAllWindows()
Below are the coordinates of fingers which we are using to control the LED and here we have used index and thumb and these two coordinates are of tip of index and thumb.
x1=l[4][0] y1=l[4][1] x2=l[8][0] y2=l[8][1]
And these are the circles and line which we have drawn on the hand to show the control bar in between index finger and thumb.
cv2.circle(img,(x1,y1),7,(0,255,255),1) cv2.circle(img,(x2,y2),7,(0,255,255),1) cv2.line(img,(x1,y1),(x2,y2),(0,255,0),2)
Below lines of code are responsible for the calculation of distance between index and thumb and after that ,this distance converted to a range of 0-255 then this data sending to the Arduino with the help of Serial library, as you can see in the below code.
d=int(math.sqrt(math.pow(x2 - x1, 2)+math.pow(y2 - y1, 2) * 1.0))
d=int((d/110)*255)
e='\n'
if 0<d<256:
cv2.putText(img,str(d),(20,30),cv2.FONT_HERSHEY_COMPLEX,.7,(255,255,255),1)
serialcomm.write(str(d).encode())
serialcomm.write(e.encode())
So in this way this code is working now its time to run this code , here I am using Jupiter to run this code but you can use cmd also.
Output Video
Now we have completed with this project. Below, with output vide0 YouTube video also given , if you have any doubt or not understand well go and check out YouTube video.
YouTube Video
Thankyou so much to read the article till now there is YouTube video on this project so please go through that video to understand the whole procedure in a better manner there is link for video go and check it out.
