Ever wondered how movie makers create stunning visual effects with actors appearing in magical places or futuristic cities? One of the secrets behind these effects is the green screen technique. In this process, actors perform in front of a bright green backdrop, which is later replaced with any background they want using computer software. This technique isn’t just for Hollywood—it’s something you can try at home with a bit of coding magic!

Codemagnet is here again to help you guys learn how to replace a green screen background using Python and a powerful library called OpenCV. OpenCV (Open Source Computer Vision Library) is a collection of programming functions mainly aimed at real-time computer vision. With it, you can manipulate images and videos, detect objects, and even recognize faces.

Here’s what you’ll learn in this tutorial:

1. Understanding the Green Screen Effect: We’ll start with a quick overview of how the green screen effect works and why green is commonly used.
2. Setting Up Your Environment: Learn how to install Python and OpenCV on your computer, so you’re ready to start coding.
3. Reading and Displaying Images: We’ll cover how to load images into your Python program and display them using OpenCV.
4. Detecting the Green Screen: Discover how to identify the green background in an image and prepare it for replacement.
5. Replacing the Background: Finally, we’ll write the code to replace the green screen with a new background image of your choice.

By the end of this tutorial, you’ll have a basic understanding of image processing with OpenCV and a cool new skill to impress your friends and family. Whether you’re a beginner in programming or looking to expand your knowledge in computer vision, this tutorial will provide a fun and practical way to learn.

Ready to make your own movie magic? Let’s dive in and learn how to replace a green screen using Python and OpenCV!

Steps to Follow:

1. Import all necessary libraries
2. Load the images or videos
3. Resize the images and the videos to the same size
4. Load the upper and lower BGR values of the green color
5. Apply the mask and then use bitwise_and
6. Subtract bitwise_and from the original green screen image
7. Check for matrix value 0 after subtraction and replace it by the second image
8. You get the desired results.

Code:

import cv2
import numpy as np

video = cv2.VideoCapture("green.mp4")
image = cv2.imread("bg.jpg")

while True:
    ret, frame = video.read()
    if not ret:
        break

    frame = cv2.resize(frame, (640, 480))
    image = cv2.resize(image, (640, 480))

    # Adjusted green color range
    lower_green = np.array([35, 40, 40])
    upper_green = np.array([85, 255, 255])

    # Convert the frame to HSV color space for better color detection
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, lower_green, upper_green)

    # Improve the mask using morphological operations
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8))
    mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, np.ones((3, 3), np.uint8))

    # Create the inverse mask
    mask_inv = cv2.bitwise_not(mask)

    # Use the mask to extract the foreground (original frame) and the background (replacement image)
    fg = cv2.bitwise_and(frame, frame, mask=mask_inv)
    bg = cv2.bitwise_and(image, image, mask=mask)

    # Combine the foreground and background
    final = cv2.add(fg, bg)

    # Concatenate the original frame and the final output side by side
    combined = np.hstack((frame, final))

    cv2.imshow("Original and Replaced Video", combined)

    if cv2.waitKey(25) == 27:
        break

video.release()
cv2.destroyAllWindows()

Output:

Explanation of the above code:

import cv2
import numpy as np

• import cv2: This line imports the OpenCV library, which is a powerful tool for image and video processing.
• import numpy as np: This line imports the NumPy library, which is used for numerical operations on arrays.

video = cv2.VideoCapture("green.mp4")
image = cv2.imread("bg.jpg")

• video = cv2.VideoCapture("green.mp4"): This line opens the video file named “green.mp4” for reading. cv2.VideoCapture is a class for video capturing from video files, image sequences, or cameras.
• image = cv2.imread("bg.jpg"): This line reads the image file “bg.jpg” into an array. cv2.imread is used to load an image from a file.

while True:

This starts an infinite loop, which will process each frame of the video one by one until it is explicitly broken.

    ret, frame = video.read()
    if not ret:
        break

• ret, frame = video.read(): This line reads the next frame from the video. ret is a boolean indicating whether the frame was read successfully, and frame is the image array of the current frame.
• if not ret: If the frame was not read successfully (ret is False), the loop is broken using break.

    frame = cv2.resize(frame, (640, 480))
    image = cv2.resize(image, (640, 480))

• frame = cv2.resize(frame, (640, 480)): This resizes the current video frame to 640×480 pixels.
• image = cv2.resize(image, (640, 480)): This resizes the background image to 640×480 pixels.

    # Adjusted green color range
    lower_green = np.array([35, 40, 40])
    upper_green = np.array([85, 255, 255])

These lines define the lower and upper bounds of the green color range in the HSV color space. Any color within this range will be considered green.

    # Convert the frame to HSV color space for better color detection
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, lower_green, upper_green)

• hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV): This converts the current frame from BGR color space (which OpenCV uses by default) to HSV color space. HSV (Hue, Saturation, Value) is often more suitable for color detection.
• mask = cv2.inRange(hsv, lower_green, upper_green): This creates a binary mask where the pixels within the green color range are set to 255 (white), and all other pixels are set to 0 (black).

    # Improve the mask using morphological operations
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8))
    mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, np.ones((3, 3), np.uint8))

• mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8)): This applies a morphological open operation (erosion followed by dilation) to remove small noise in the mask. np.ones((3, 3), np.uint8) creates a 3×3 kernel of ones for the operation.
• mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, np.ones((3, 3), np.uint8)):This applies a dilation operation to the mask to further refine it by enlarging the white regions.

    # Create the inverse mask
    mask_inv = cv2.bitwise_not(mask)

mask_inv = cv2.bitwise_not(mask): This creates the inverse of the mask, where the white and black regions are swapped.

Let’s summarize the process:

1. Basic Masking:
   • Convert the image to the LAB color space.
   • Threshold the A-channel to isolate the green background.
   • Create a binary mask by applying the threshold to the original image.
   • Use the mask to extract the foreground (object) and the background separately.
   • The result will have a dark background where the green screen was.
2. Removing Green Shade Along the Border:
   • Convert the masked image back to the LAB color space.
   • Normalize the A-channel intensity to use the entire range between 0 and 255.
   • Apply an inverse binary threshold to select the areas with green borders.
   • Set the intensity value in the A-channel of the selected pixels to 127.
   • This step helps remove the green shade along the object’s border.
3. Overlaying the Background:
   • To complete the process, replace the black screen (previously the green screen) with the desired background image.
   • Use the cv2.bitwise_and() function to overlay the background image onto the frame, effectively replacing the green screen¹²³.

Remember that the quality of the final result depends on the accuracy of the masking and the choice of background image. Happy coding! 🌟