Skip to content

5. Output Format

Jump to bottom
Maneesh edited this page May 20, 2024 · 2 revisions

PyAFAR outputs a CSV (or JSON) file with the following columns (or fields):

  • Frame: Frame number in the input video
  • Eye Aspect Ratio: Ratio of width to height of the eye, this is the mean aspect ratio between the left eye and the right eye
  • Mouth Aspect Ratio: Ratio of width to height of the mouth

Tracking

The output CSV file contains predictions for frames where at least one individual in the video is detected. Hence the number of unique frames in the CSV could be less than the total number of frames in the input video.

Head Pose

Pitch, Yaw, Roll: Head orientation along Pitch, Yaw and Roll (in degrees or radians?)

Facial Landmarks

For a given landmark i the landmarks in the output CSV are represented using x_i, y_i and z_i. PyAFAR predicts 468 3D (x, y, z) landmarks.

Action Unit (AU) Predictions

PyAFAR predicts Action Units using separate prediction models for adults and infants. Note that not all AU occurrences are predicted for both adults and infants. Predictions available are indicated below in (). An interactive refresher to AUs can be found here.

The AU detector module of PyAFAR can predict the occurrence and intensity of the following action units.

Occurrence

AU 1 (adult, infant): Inner Brow Raise
AU 2 (adult, infant): Outer Brow Raise
AU 3 (infant only): Inner corner Brow Tightener
AU 4 (adult, infant): Brow Lowerer
AU 6 (adult, infant): Cheek Raise
AU 7 (adult only): Lids Tight
AU 9 (infant only): Nose wrinkle
AU 10 (adult only): Upper Lip Raiser
AU 12 (adult, infant): Lip Corner Puller
AU 14 (adult only): Dimpler
AU 15 (adult only): Lip Corner Depressor
AU 17 (adult only): Chin Raiser
AU 20 (infant only): Lip Stretch
AU 23 (adult only): Lip Tightener
AU 24 (adult only): Lip Presser
AU 28 (infant only): Lip Suck

where Occ_au_i column in the CSV is the likelihood of AU i expressed by Person_ID in the frame.

Intensity

AU 6 (adult only)
AU 10 (adult only)
AU 12 (adult only)
AU 14 (adult only)
AU 17 (adult only)

where Int_au_i column in the CSV is the intensity of AU i expressed by Person_ID in the frame. Intensity predictions lie in [0, 5] range.

Visualization

PyAFAR can predict various face based affect related features as demonstrated below

demo_gif

Clone this wiki locally