Project Video
Robotics
[Hand Gesture Control System for TurtleBot (MediaPipe-Based Vision Control)]
A vision-based robotic control system that enables a TurtleBot to be navigated using real-time hand gestures detected through a camera and MediaPipe, translating gestures into ROS movement commands.
Overview
This project implements a hand gesture recognition system to control a TurtleBot using real-time camera input. The system leverages computer vision and MediaPipe hand tracking to detect specific hand gestures, which are then mapped to robot movement commands.
Instead of using traditional controllers or keyboards, the robot responds naturally to human hand movements, creating an intuitive and contactless interaction method for navigation and control.
How It Works
The TurtleBot continuously captures a live video feed using its onboard camera or an external webcam. Each frame is processed using MediaPipe Hands, which detects and tracks key landmarks of the hand in real time.
Once hand landmarks are extracted, the system analyzes finger positions and gestures such as:
- Open palm → Move forward
- Closed fist → Stop
- peace sign → Turn left
- three middle fingers right → Turn right
- index finger -> Forward
These detected gestures are converted into command signals.
The commands are then published through ROS topics to control the TurtleBot’s motors, enabling real-time movement based on hand gestures.
A smoothing and filtering mechanism is applied to reduce jitter and prevent false gesture detection.
Features
- Real-time hand tracking using camera input
- MediaPipe-based landmark detection
- Gesture-to-command mapping system
- ROS-based robot control integration
- Contactless human–robot interaction
- Low-latency gesture recognition
- Smooth motion filtering for stability
- Intuitive control interface
Technologies Used
- Python
- ROS / ROS2
- TurtleBot3 Burger
- OpenCV
- MediaPipe
- NumPy
- Linux (Ubuntu)
- Camera Vision System
- Robot Operating System Communication (ROS Topics & Nodes)
Applications
- Human–robot interaction systems
- Assistive robotics for accessibility
- Smart environments and automation
- Gesture-based control interfaces
- Educational robotics demonstrations
- Touchless control systems (hygiene-focused environments)
- Research in vision-based robotics
Learning Outcomes
This project improved skills in:
- Real-time computer vision processing
