Braccio Robot Manipulator

Braccio Robot Manipulator Control

Preview

Introduction

This project demonstrates how to control a Braccio robot manipulator using hand tracking with OpenCV and PySerial for communication with an Arduino. The system uses computer vision to detect hand gestures, which are translated into motor commands for the robotic arm.

The first hand acts as a motor selector (motors 1-5), while the second hand handles motors 5-6 and controls the direction of movement for the selected motor. So for example, I would use my index finger on my right hand to choose a motor, and on the other hand, I would use the index and middle finger to subtract 10 degrees (each second) from the selected motor.

Components Needed

Component	Price	Description	Link
Braccio Robot Arm	~100	TinkerKit Braccio Robot Arm	Arduino Store
Arduino Board	~20	Compatible with Braccio shield	Arduino Uno
Webcam	~10	USB camera for hand tracking
PySerial Library	Free	Python serial communication	pip install
OpenCV	Free	Computer vision library	pip install
Braccio Library	Free	Braccio control library	click

Hand Tracking Setup Example

Using OpenCV’s hand tracking capabilities, we detect hand positions and gestures in real-time video feed.

import cv2
import mediapipe as mp

mp_hands = mp.solutions.hands
hands = mp_hands.Hands()

cap = cv2.VideoCapture(0)
while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    
    results = hands.process(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
    ...

Arduino Communication Example

The Python script sends motor index and direction data via serial communication to the Arduino controlling the Braccio arm.

import serial

ser = serial.Serial('COM3', 9600)

def send_command(motor_index, direction):
    data = f"{motor_index},{direction}\n"
    ser.write(data.encode())
    ...

Conclusion

Not the most effective and intuitive way of controlling the robot, but it was a fun experience.