AirGesture
Real-time computer vision app using OpenCV and MediaPipe that translates hand gestures into presentation controls for touchless classroom interaction.
Overview
A real-time gesture recognition system built using MediaPipe and OpenCV that enables users to control presentations through hand gestures, reducing dependence on traditional input devices.
Enabled fully touchless presentation control using just a webcam.
The Problem
Teachers and presenters frequently need to interact with keyboards or clickers while presenting, interrupting the natural flow of communication and forcing them to stay close to their device.
Why I Built It
To provide an intuitive touchless interaction system that enables smoother classroom teaching and presentations, letting the presenter move freely while maintaining full control.
Objectives
- Recognize hand gestures reliably in real-time from a standard webcam
- Map gestures to presentation control actions
- Maintain low latency to keep interaction feeling natural
- Work without specialized hardware — just a webcam
My Role
Built the gesture recognition pipeline, implemented real-time computer vision processing, designed the gesture-to-command mapping system, and optimized the pipeline for minimal latency.
Architecture
MediaPipe Hands detects 21 hand landmarks per frame in real time. A landmark analysis layer classifies gesture patterns. The control dispatcher maps recognized gestures to system keyboard events, enabling control of any presentation software.
Tech Stack
Computer Vision
System Control
Engineering Challenges
False Positive Gestures
Natural hand movement during speech triggered unintended slide changes. Solved by requiring gesture hold duration before triggering actions.
Lighting Sensitivity
MediaPipe landmark detection degrades in poor lighting. Added preprocessing to normalize webcam input.
Latency
Processing every frame for landmark detection created perceptible lag. Optimized by skipping frames and caching landmark results.
Development Process
- 01Researched MediaPipe Hands landmark structure and gesture classification approaches
- 02Built basic landmark visualization and gesture logging
- 03Defined gesture vocabulary: swipe right (next), swipe left (previous), open palm (pause), pinch (pointer toggle)
- 04Implemented gesture classification from landmark positions
- 05Added gesture hold duration to reduce false positives
- 06Integrated PyAutoGUI for system keyboard event dispatch
- 07Tested across different lighting conditions and webcam qualities
Key Features
Next Slide
Swipe right with an open hand — moves to the next slide in any presentation.
Previous Slide
Swipe left with an open hand — moves to the previous slide.
Pause / Blank Screen
Open palm held still — triggers presentation pause or blank screen.
Laser Pointer
Index finger point — activates pointer mode, tracking finger position on screen.
Implementation Details
Each video frame is processed by MediaPipe to extract 21 3D hand landmarks. A rule-based classifier analyzes relative landmark positions and velocity vectors to determine gesture type. Confirmed gestures trigger PyAutoGUI keyboard events that work with PowerPoint, Keynote, Google Slides, and any other presentation software.
Future Improvements
- Add ML-based gesture classifier to handle more complex gestures
- Support two-hand gestures for zoom and annotation
- Build a calibration mode for different hand sizes and webcam positions
- Package as a standalone executable for easy installation
Lessons Learned
Real-time computer vision is as much about filtering bad data as detecting good data
User experience in gesture interfaces requires careful threshold tuning — too sensitive is worse than not sensitive enough
MediaPipe is remarkably powerful for the zero-configuration it requires