In this article, we will explore how to determine the tilt of a face using OpenCV, a widely-used computer vision library in Python. By leveraging OpenCV’s capabilities, we can detect faces in real-time from a webcam feed or a video file and calculate the angle at which the face is tilted.

Requirements

To get started, you will need to install OpenCV for Python. Additionally, we will utilize two pre-trained XML classifiers for face and eye detection, respectively. These classifiers can be downloaded from the OpenCV library:

• Classifier for face detection: haarcascade_frontalface_default.xml
• Classifier for eye detection: haarcascade_eye.xml

Algorithm

1. Face Detection: First, we detect faces in the webcam feed or video using the Haar cascade classifier for faces. We draw a green bounding box around each detected face.
2. Eye Detection: Next, we detect eyes within the detected face using a similar Haar cascade classifier trained on eyes. We draw a red bounding box around each detected eye.
3. Identify Eye Centers: For each detected eye, we identify and store the center of the bounding box. We assume that the center of the bounding box is approximately the center of the eye.
4. Calculate Tilt Angle: To compute the face tilt angle, we assume that the line joining the centers of the two eyes is perpendicular to the face. Given the coordinates of the two eye centers (x1,y1)(x_1, y_1)(x1​,y1​) and (x2,y2)(x_2, y_2)(x2​,y2​), the angle θ\thetaθ that the line joining these points makes with the x-axis can be calculated using the following geometric expression:

1. Interpret the Angle: The computed angle indicates the tilt of the face:
   • A positive angle indicates a right tilt.
   • A negative angle indicates a left tilt.
2. Classify Tilt: We define a margin of error of 10 degrees. If the face tilts more than 10 degrees to either side, the program classifies the tilt as either right or left.

Example Code

Here is a Python code to detect face tilt using OpenCV:

import cv2
import numpy as np
import math

def detect_face_and_eyes(frame, face_cascade, eye_cascade):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)

    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        
        eyes = eye_cascade.detectMultiScale(roi_gray)
        eye_centers = []

        for (ex, ey, ew, eh) in eyes:
            cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 0, 255), 2)
            eye_center = (ex + ew // 2, ey + eh // 2)
            eye_centers.append(eye_center)
            cv2.circle(roi_color, eye_center, 5, (255, 0, 0), -1)
        
        if len(eye_centers) >= 2:
            dx = eye_centers[1][0] - eye_centers[0][0]
            dy = eye_centers[1][1] - eye_centers[0][1]
            angle = math.degrees(math.atan2(dy, dx))
            return angle

    return None

def main():
    face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
    eye_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_eye.xml')
    
    cap = cv2.VideoCapture(0)

    while True:
        ret, frame = cap.read()
        if not ret:
            break
        
        angle = detect_face_and_eyes(frame, face_cascade, eye_cascade)
        
        if angle is not None:
            if abs(angle) > 10:
                tilt_direction = "right" if angle > 0 else "left"
                cv2.putText(frame, f"Face Tilt: {tilt_direction} ({int(angle)} degrees)", 
                            (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
            else:
                cv2.putText(frame, "Face Tilt: Straight", 
                            (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2, cv2.LINE_AA)
        
        cv2.imshow('Face Tilt Detection', frame)
        
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    cap.release()
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

Detailed Explanation

The provided code demonstrates how to create a simple program to detect face tilt using OpenCV. Here’s a step-by-step breakdown of the process:

1. Initialize the Haar Cascades: We initialize the face and eye Haar cascades using OpenCV’s pre-trained XML files.
2. Capture Video Stream: We start capturing the video stream from the webcam.
3. Detect Faces and Eyes: For each frame, we convert it to grayscale and detect faces. Within each detected face, we then detect the eyes.
4. Calculate Eye Centers: For each detected eye, we compute the center of the bounding box, which we assume to be the center of the eye.
5. Compute Tilt Angle: Using the coordinates of the eye centers, we calculate the angle of tilt. This angle is displayed on the video feed along with the direction of the tilt (left or right).
6. Display Results: The results, including the detected faces, eyes, and tilt information, are displayed on the video feed.

By following this approach, you can create a robust system for detecting face tilt using OpenCV in Python. This method can be extended and integrated into various applications, such as user interaction systems, security systems, and more.

Explanation of the above code:

Importing Libraries

• cv2: OpenCV library for computer vision tasks.
• numpy: Library for numerical operations.
• math: Library for mathematical functions.

detect_face_and_eyes Function

1. Convert Frame to Grayscale
   • gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
   • Converts the input frame from color (BGR) to grayscale.
2. Detect Faces
   • faces = face_cascade.detectMultiScale(gray, 1.3, 5)
   • Uses the face cascade classifier to detect faces in the grayscale frame.
3. Draw Bounding Box for Faces
   • cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
   • Draws a green rectangle around each detected face.
4. Region of Interest (ROI) for Eyes
   • roi_gray = gray[y:y+h, x:x+w]
   • roi_color = frame[y:y+h, x:x+w]
   • Defines the region of interest within the face for eye detection.
5. Detect Eyes
   • eyes = eye_cascade.detectMultiScale(roi_gray)
   • Uses the eye cascade classifier to detect eyes within the face ROI.
6. Draw Bounding Box for Eyes and Store Eye Centers
   • For each detected eye:
     • cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 0, 255), 2)
     • Draws a red rectangle around each detected eye.
     • eye_center = (ex + ew // 2, ey + eh // 2)
     • eye_centers.append(eye_center)
     • Calculates and stores the center of each eye, and draws a blue circle at the center.
7. Calculate Tilt Angle
   • If at least two eyes are detected:
     • dx = eye_centers[1][0] - eye_centers[0][0]
     • dy = eye_centers[1][1] - eye_centers[0][1]
     • angle = math.degrees(math.atan2(dy, dx))
     • Calculates the tilt angle of the face based on the coordinates of the eye centers.
8. Return Angle
   • return angle
   • Returns the calculated tilt angle.
   • If no eyes are detected, returns None.

main Function

1. Load Haar Cascades
   • face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
   • eye_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_eye.xml')
   • Loads the pre-trained Haar cascades for face and eye detection.
2. Capture Video from Webcam
   • cap = cv2.VideoCapture(0)
   • Captures video feed from the default webcam.
3. Process Video Frames
   • ret, frame = cap.read()
   • Reads frames from the webcam in a loop.
   • If a frame is read successfully, proceeds with processing.
4. Detect Face and Eyes in Frame
   • angle = detect_face_and_eyes(frame, face_cascade, eye_cascade)
   • Calls the detect_face_and_eyes function to detect face tilt in the current frame.
5. Display Face Tilt Information
   • If an angle is detected:
     • If the absolute angle is greater than 10 degrees, determines the tilt direction (left or right).
     • Displays the tilt direction and angle on the frame.
     • If the angle is within 10 degrees, displays “Face Tilt: Straight”.
6. Show Frame with Annotations
   • cv2.imshow('Face Tilt Detection', frame)
   • Displays the processed frame with annotations in a window.
7. Exit on ‘q’ Key Press
   • if cv2.waitKey(1) & 0xFF == ord('q'):
   • Exits the loop and closes the window if the ‘q’ key is pressed.
8. Release Resources
   • cap.release()
   • Releases the webcam.
   • cv2.destroyAllWindows()
   • Closes all OpenCV windows.

Execution

• if __name__ == "__main__":
• main()
• Executes the main function if the script is run directly.