NullPointerException — Hand Gesture System

A touchless human-computer interaction system that replaces your mouse entirely using a webcam and hand tracking. Built with MediaPipe and OpenCV.

---

Features

• Cursor control — move cursor with index finger
• Left click — pinch index + thumb
• Right click — pinch middle + thumb
• Double click — pinch pinky + thumb
• Drag and drop — pinch and hold index + thumb
• Scroll — peace sign + move hand up/down
• Zoom — left fist + spread/pinch right hand
• Screenshot — right fist
• Alt+Tab — left three fingers + swipe right hand
• Volume control — left index only + move right hand up/down
• Brightness control — left index + middle + move right hand up/down
• Power aware — pauses automatically on battery
• System tray — pause/resume/quit from taskbar
• OSD overlay — shows current mode on screen

---

Requirements

• Python 3.10
• Webcam (720p minimum)
• Windows 10/11
• Laptop must be plugged in to run (by design)

---

Installation

1. Clone the repository

git clone https://github.com/KV225511/NullPointerException.git
cd NullPointerException/hand_gesture_system

2. Create virtual environment

python -m venv venv
venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt

4. Download MediaPipe hand landmarker model

Download hand_landmarker.task from: https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

Place it in the models/ folder.

5. Run the system

python main.py

6. Setup auto-start on login (optional)

Run as Administrator:

scripts\setup_startup.bat

To remove auto-start:

scripts\remove_startup.bat

---

Gesture Reference

Right hand gestures

Gesture	Action
Index finger pointing	Move cursor
Index + thumb pinch	Left click
Middle + thumb pinch	Right click
Pinky + thumb pinch	Double click
Index + thumb hold	Drag and drop
Peace sign + move up	Scroll up
Peace sign + move down	Scroll down
Fist (first time)	Screenshot
Fist (second time)	Close screenshot

Left hand modes

Gesture	Mode
No gesture	Cursor mode (default)
Index only	Volume control
Index + middle	Brightness control
Index + middle + ring	Alt+Tab mode
Fist	Zoom mode

While in modes (right hand)

Mode	Gesture	Action
Volume	Move up	Volume up
Volume	Move down	Volume down
Brightness	Move up	Brightness up
Brightness	Move down	Brightness down
Zoom	Spread fingers	Zoom in
Zoom	Pinch fingers	Zoom out
Alt+Tab	Swipe right	Next app
Alt+Tab	Swipe left	Previous app

---

Project Structure

hand_gesture_system/

core/

hand_tracker.py # MediaPipe wrapper

gesture_detector.py # gesture classification

cursor_controller.py # mouse movement and mapping

smoother.py # coordinate smoothing

power_manager.py # AC power detection

addons/

scroll_controller.py # scroll action

zoom_controller.py # zoom in/out

screenshot_controller.py # screenshot

alt_tab_controller.py # app switcher

volume_controller.py # volume control

brightness_controller.py # brightness control

ui/

osd_overlay.py # on screen display

tray_icon.py # system tray

config/

settings.yaml # tunable settings

models/

hand_landmarker.task # MediaPipe model (download separately)

scripts/

startup_win.bat # launch script

startup_win.vbs # hidden launcher

setup_startup.bat # register auto-start

remove_startup.bat # remove auto-start

main.py

requirements.txt

README.md

---

Troubleshooting

Webcam not detected

• Make sure no other app is using the webcam
• Try changing camera.index to 1 in config/settings.yaml

Brightness control not working

• Only works on laptop internal displays
• External monitors may not be supported

System not starting on login

• Make sure setup_startup.bat was run as Administrator
• Check Task Scheduler for NullPointerException task

Gestures not detected reliably

• Ensure good lighting
• Keep hand within the active zone
• Adjust thresholds in config/settings.yaml

---

Built With

• MediaPipe — hand landmark detection
• OpenCV — computer vision
• PyAutoGUI — mouse and keyboard control
• pycaw — Windows audio control
• pystray — system tray
• psutil — power management

---