This project is a real-time cognitive analytics system designed for online classroom environments, where a teacher can monitor student engagement through visual cues captured via a webcam.
The system processes live video input and applies computer vision techniques using MediaPipe facial landmark detection to extract meaningful behavioral signals.
It consists of multiple independent modules that analyze different aspects of a student’s state, including:
- Drowsiness
- Attention
- Stress
- Confusion
Each module operates on the same video frame but computes distinct metrics based on facial features such as:
- Eye Aspect Ratio (EAR)
- Mouth Aspect Ratio (MAR)
- Head pose
- Eyebrow movement
- Blink rate
The drowsiness module detects signs of fatigue using a combination of:
- Eye closure patterns
- Yawning frequency
It outputs a binary score indicating whether the student is sleepy.
The attention module tracks:
- Head orientation
- Movement stability
- Frequency of looking away from the screen
These signals are converted into an attention score representing the student’s focus level.
The stress module evaluates facial tension through:
- Eyebrow compression
- Lip tightening
- Blink frequency
These indicators are combined to produce a stress score.
The confusion module analyzes subtle behavioral cues such as:
- Eyebrow asymmetry
- Inward brow movement
- Slight squinting
- Head tilt
- Repeated side glances
These signals are used to estimate a confusion score.
All modules are integrated into a unified dashboard that displays six synchronized video panels:
- Raw camera feed
- Facial landmarks
- Drowsiness detector output
- Attention detector output
- Stress detector output
- Confusion detector output
Alongside the visual interface, the system includes a separate analytics window built using:
- PyQt
- PyQtGraph
This analytics window plots real-time graphs of:
- Drowsiness
- Attention
- Stress
- Confusion
The graphs provide a continuous view of the student’s cognitive state, enabling instructors to observe trends and identify moments of disengagement or difficulty.
The system follows a modular architecture where each detector:
- Independently processes video frames
- Returns annotated visuals
- Produces numerical scores
These scores are aggregated and streamed into the analytics layer without interrupting the video processing pipeline.
The system is optimized for real-time performance and designed to be extensible, making it suitable for:
- Scaling to multiple students
- Integrating additional behavioral metrics
This project demonstrates how computer vision can enhance digital learning environments by providing actionable insights into student engagement and well-being.
