Gesture Control

Standard pointer input assumes able hands on a desk. I wanted to prototype how far webcam-only control could go as an alternative input channel — useful for presentations, accessibility scenarios and touchless interfaces.

Hand landmarks are smoothed and mapped to screen space with adaptive gain, pinch detection triggers clicks, and a dead-zone near screen edges prevents accidental system interactions.

• Smooth absolute cursor mapping with adjustable gain
• Pinch-to-click and held-pinch drag support
• Smoothing tuned against both jitter and latency
• Visual debug overlay showing tracking confidence

• Python
• OpenCV
• MediaPipe
• OS Automation

• Direct coordinate mapping made small movements swing wildly across the screen.

  Added a nonlinear gain curve — fine control in the center of the frame, faster travel toward edges.

• Click registration fired accidentally during fast motion.

  Required pinch stability across consecutive frames with velocity gating before committing a click.

• Absolute mapping with gain curves

  Relative mode drifts without a physical surface; absolute keeps spatial intuition.

• Debug overlay from day one

  Vision failures are invisible without seeing confidence and raw landmarks.

• A functional prototype demonstrating reliable cursor control and clicking in good lighting.
• A reusable tracking stack ready for future vision-based interfaces.

• →Input systems are judged on their worst frames, not their best demos.
• →Every vision-based controller needs an escape hatch — dead zones save sanity.