<h1>Fine-Tuning and Transfer Learning Explained</h1><p>Build Better Models Faster Using Knowledge from Existing Networks</p><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/CNN_bounding_box_detect/cnn_architecture.drawio.png?raw=true"><p>Training deep learning models from scratch requires large amounts of data, computing power, and time. Transfer learning and fine-tuning offer a more efficient approach by allowing you to build on the knowledge captured by existing models trained on large datasets.</p><p>This article breaks down what transfer learning and fine-tuning are, when to use them, and how to apply them effectively.</p><h2>What is Transfer Learning?</h2><p>Transfer learning is the practice of taking a model that has already been trained on a large and general dataset, and adapting it to a different but related task. The idea is that the model has already learned useful features that don’t need to be relearned from scratch.</p><p>For example, a model trained on millions of general images can recognize edges, textures, and shapes that are also useful for identifying specific categories like medical scans, food items, or vehicle types.</p><h2>When is Transfer Learning Useful?</h2><p>Transfer learning is most helpful when:</p><ul><li><p>You have a small dataset or limited access to labeled data.</p></li><li><p>Your task is related to a well-established domain with existing models.</p></li><li><p>You want to reduce the cost and time of training.</p></li></ul><p>It is commonly used in image <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/N31Fp0bfBxv92NdKlpp3?type=publishedchapter">classification</a>, <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/EdidCGkFJIJLkz9JpQhK?type=publishedchapter">object detection</a>, text classification, sentiment analysis, and other tasks where large pre-trained models exist.</p><h2>What is Fine-Tuning?</h2><p>Fine-tuning is the process of adjusting a pre-trained model so that it performs better on a specific task. This is usually done after applying transfer learning.</p><p>There are two main approaches:</p><h3>1. Feature Extraction</h3><p>With <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/OCiRGYXlA16BEuQUjSfj?type=publishedchapter">feature extraction</a> you keep the entire pre-trained model unchanged and only train a new classification layer on top of it. The model acts as a fixed feature extractor.</p><h3>2. Fine-Tuning</h3><p>You unfreeze some or all layers of the pre-trained model and continue training it on your dataset. This allows the model to adapt its internal representations to the new task more precisely.</p><p>Fine-tuning tends to give better results, especially if the new task is significantly different from the original one.</p><h2>How to Approach Fine-Tuning</h2><p>A typical workflow looks like this:</p><ol><li><p><strong>Start with a pre-trained model</strong> trained on a large dataset.</p></li><li><p><strong>Replace the final output layer</strong> to match the number of classes in your new task.</p></li><li><p><strong>Freeze the pre-trained layers</strong> and train only the new output layer.</p></li><li><p><strong>Optionally unfreeze selected layers</strong> and fine-tune the model on the new data, typically using a smaller learning rate.</p></li></ol><p>This process allows you to gradually adapt the model while preserving the valuable features it learned during its initial training.</p><h2>Tips for Effective Transfer Learning</h2><ul><li><p>Use <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/blog/n4BQpYEx2krZ74URQcFe">data augmentation</a> when working with small datasets to avoid overfitting.</p></li><li><p>Always start with training just the new layers before unfreezing the rest of the model.</p></li><li><p>Fine-tune with a lower learning rate to avoid destroying the pre-trained weights.</p></li><li><p>Monitor validation performance closely, especially when unfreezing more layers.</p></li></ul><p>Transfer learning and fine-tuning are essential tools for building deep learning models efficiently. They let you leverage powerful models trained on massive datasets and adapt them to solve your own problems, even when your data is limited.</p><p>For a detailed walk-through of how you can fine-tune a YOLO11 model for object detection, check out this article: <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/ikvBmuF2FKWN04v3hnii?type=publishedchapter"><strong>Fine-tuning a YOLO11 Object Detection Model for Kidney Stones Detection</strong></a></p>

Build Better Models Faster Using Knowledge from Existing Networks

Fine-Tuning and Transfer Learning Explained

<h1><strong>An Overview of Image Data Augmentation in Deep Learning</strong></h1><p>Solving the Problem of Limited Data with Image Augmentation</p><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Augmented_Images_Grid.png?raw=true"><p style="text-align: center">Various Image Augmentation Techniques on a Kangaroo Image from <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://www.kaggle.com/datasets/hugozanini1/kangaroodataset">kangaroo-dataset on Kaggle</a></p><p>In machine learning, particularly in computer vision, having a large and diverse dataset is crucial for training accurate models. However, obtaining such datasets can be expensive, time-consuming, or impractical, especially when labeled data is scarce.</p><p>In these cases, data augmentation becomes a valuable tool. By applying transformations to existing images, augmentation artificially increases the size and diversity of the dataset, helping the model generalize better. It introduces variations that are common in real-world scenarios, such as changes in orientation, lighting, and scale.</p><p>Data augmentation is particularly important when working with limited data. It helps prevent overfitting, a common issue where a model memorizes training data instead of learning generalizable patterns. With augmentation, the model is exposed to diverse examples, making it more robust and adaptable to unseen data.</p><h2><strong>What is Image Augmentation?</strong></h2><p>Image augmentation is a technique used to expand a dataset without needing to collect new images. This is done by applying random transformations to existing images, simulating real-world variations. These variations might include:</p><ul><li><p>Changes in orientation (e.g., flipping, rotating)</p></li><li><p>Adjustments in brightness and contrast</p></li><li><p>Scaling and cropping to simulate different object sizes</p></li></ul><p>Below is a summary of some augmentation techniques we applied and how they transform the images:</p><table style="min-width: 50px"><colgroup><col style="min-width: 25px"><col style="min-width: 25px"></colgroup><tbody><tr><th colspan="1" rowspan="1"><p><strong>Augmentation Technique</strong></p></th><th colspan="1" rowspan="1"><p><strong>Transformation Applied</strong></p></th></tr><tr><td colspan="1" rowspan="1"><p><strong>Grayscale (Black &amp; White)</strong></p></td><td colspan="1" rowspan="1"><p>Converts the image to grayscale, simulating a lack of color information.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Horizontal &amp; Vertical Flip</strong></p></td><td colspan="1" rowspan="1"><p>Flips the image horizontally or vertically to simulate different orientations of the same object.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Random Rotation</strong></p></td><td colspan="1" rowspan="1"><p>Rotates the image by a random angle to simulate objects appearing at various angles.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Random Translation</strong></p></td><td colspan="1" rowspan="1"><p>Shifts the image along the x and y axes, simulating slight movements or translations of objects.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Random Shearing</strong></p></td><td colspan="1" rowspan="1"><p>Applies a shear transformation, tilting the image at random angles to simulate distortion or changes in perspective.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Random Brightness Adjustment</strong></p></td><td colspan="1" rowspan="1"><p>Adjusts the brightness of the image, simulating different lighting conditions.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Random Contrast Adjustment</strong></p></td><td colspan="1" rowspan="1"><p>Alters the contrast of the image, creating variations in image clarity or intensity.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>RGB Shift</strong></p></td><td colspan="1" rowspan="1"><p>Shifts the color channels (Red, Green, Blue) to simulate color variations and sensor differences.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Channel Shuffle</strong></p></td><td colspan="1" rowspan="1"><p>Shuffles the RGB channels to create new color combinations, enhancing model robustness.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Noise &amp; Gaussian Blur</strong></p></td><td colspan="1" rowspan="1"><p>Adds random noise and applies a blur effect, simulating noisy environments or motion blur.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>Cutout (Coarse Dropout)</strong></p></td><td colspan="1" rowspan="1"><p>Randomly masks out portions of the image, mimicking occlusions or parts of objects being missing.</p></td></tr><tr><td colspan="1" rowspan="1"><p><strong>CLAHE (Histogram Equalization)</strong></p></td><td colspan="1" rowspan="1"><p>Improves the image contrast using adaptive histogram equalization to enhance features.</p></td></tr></tbody></table><h2>Disadvantages of Image Augmentation</h2><p>However, over-relying on data augmentation can have its drawbacks. Excessive augmentation may introduce unrealistic variations, leading to models that struggle with real-world data.</p><p>Moreover, if not carefully applied, augmentation can amplify biases in the original dataset, reinforcing and perpetuating existing disparities.</p><ul><li><p><strong>Unrealistic data variations</strong>: Over-augmentation can lead to data that doesn’t reflect real-world conditions, impairing model performance.</p></li><li><p><strong>Amplification of bias</strong>: If the original dataset contains biases, augmentation can exacerbate them, reinforcing skewed predictions.</p></li><li><p><strong>Excessive noise</strong>: Too much noise in the augmented data can confuse the model, hindering its ability to learn key patterns.</p></li><li><p><strong>Increased training time</strong>: Larger augmented datasets result in longer training times and higher computational costs.</p></li></ul><h2>Python Visualization</h2><blockquote><p>Access the code in this guide on this <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://github.com/Brianhulela/ebook_object_detection/tree/master/chapters/data_augmentation">GitHub Repository</a>.</p></blockquote><h3>Setting Up the Python Environment</h3><p>Before diving into the code, ensure you have the necessary libraries installed:</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">%pip install albumentations opencv-python pillow matplotlib Pillow</code></pre></div><p>Next, let's import the required libraries. <code>Albumentations</code> is an open-source library for image augmentation, and we'll use <code>PIL</code> for image handling, <code>NumPy</code> for array manipulation, and <code>Matplotlib</code> for visualization.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import albumentations as A
from albumentations import Compose, Rotate, ShiftScaleRotate, Affine, RandomBrightnessContrast, RGBShift, ChannelShuffle

plt.style.use('dark_background')</code></pre></div><h3>Visualizing the Original Image</h3><p>First, we need to load the original image that we'll be augmenting. Let's define a simple function to display the image:</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">def show(image, title="Augmented Image"):
    plt.imshow(image)
    plt.title(title)
    plt.axis("off")
    plt.show()

# Load image using PIL and convert to RGB
image = np.array(Image.open("sample_image.jpg").convert("RGB"))

# Display original
show(image, "Original Image")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Original%20Image.png?raw=true"><h3>Grayscale (Black &amp; White) Conversion</h3><p>One of the simplest augmentations is converting an image to grayscale, which is especially useful for tasks where color information is not critical. This can help the model focus more on shapes and structures rather than colors.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Define the transformation to convert to grayscale (black &amp; white)
gray_augment = A.Compose([
    A.ToGray(p=1.0)  # Convert the image to grayscale with 100% probability
])

# Apply the grayscale transformation
gray_image = gray_augment(image=image)["image"]

# Show the black and white image
show(gray_image, "Black &amp; White (Grayscale)")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Black%20&amp;%20White%20(Grayscale).png?raw=true"><h3>Horizontal and Vertical Flips</h3><p>Flipping an image horizontally or vertically can simulate different orientations, making it an essential augmentation technique in many scenarios. This is particularly useful for images of objects that may appear in different orientations.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">transform = A.Compose([
    A.HorizontalFlip(p=1.0),
    A.VerticalFlip(p=1.0)
])

augmented = transform(image=image)["image"]
show(augmented, "Flipped Horizontally and Vertically")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Flipped%20Horizontally%20and%20Vertically.png?raw=true"><h3>Random Rotation</h3><p>Random rotation is a popular augmentation that allows the model to learn to recognize objects regardless of their orientation. You can apply a random rotation within a specific range of angles, like this:</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply a random rotation
rotation_augment = Compose([
    Rotate(limit=(-45, 45), p=1.0)  # Rotate by a random angle between -45 to 45 degrees
])

rotated_image = rotation_augment(image=image)["image"]
show(rotated_image, "Random Rotation")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20Rotation.png?raw=true"><h3>Random Translation (Shifting)</h3><p>Translation refers to shifting an image along the x or y axis. This technique helps simulate scenarios where objects might be partially cut off or located at different positions within the image.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply random translation (shifting)
shift_augment = Compose([
    ShiftScaleRotate(shift_limit=0.3, scale_limit=0, rotate_limit=0, p=1.0)  # Translation only
])

shifted_image = shift_augment(image=image)["image"]
show(shifted_image, "Random Translation")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20Translation.png?raw=true"><h3>Random Shearing</h3><p>Shearing is a transformation that distorts the image along one axis. This augmentation simulates the effect of viewing an object from an angle or from a perspective that is different from the original view.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply random shearing
shear_augment = Compose([
    Affine(shear=10, p=1.0)  # Apply shear with a factor of 10 degrees
])

sheared_image = shear_augment(image=image)["image"]
show(sheared_image, "Random Shearing")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20Shearing.png?raw=true"><h3>Brightness and Contrast Adjustments</h3><p>In real-world conditions, the lighting may vary, and images may appear either too bright or too dark. Random brightness and contrast adjustments help the model learn to handle such variations.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply random brightness adjustment
brightness_augment = Compose([
    RandomBrightnessContrast(brightness_limit=0.3, p=1.0)  # Random brightness adjustment
])

brightness_image = brightness_augment(image=image)["image"]
show(brightness_image, "Random Brightness Adjustment")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20Brightness%20Adjustment.png?raw=true"><p>Similarly, for contrast adjustment:</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply random contrast adjustment
contrast_augment = Compose([
    RandomBrightnessContrast(contrast_limit=0.3, p=1.0)  # Random contrast adjustment
])

contrast_image = contrast_augment(image=image)["image"]
show(contrast_image, "Random Contrast Adjustment")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20Contrast%20Adjustment.png?raw=true"><h3>RGB Shift</h3><p>Shifting the RGB channels helps the model learn to deal with color variations. By applying random shifts to the Red, Green, and Blue channels, you make the model invariant to slight color shifts.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply RGB shift
rgb_shift_augment = Compose([
    RGBShift(r_shift_limit=20, g_shift_limit=20, b_shift_limit=20, p=1.0)  # Shift each channel by a random amount
])

rgb_shifted_image = rgb_shift_augment(image=image)["image"]
show(rgb_shifted_image, "Random RGB Shift")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Random%20RGB%20Shift.png?raw=true"><h3>Channel Shuffle</h3><p>Shuffling the color channels (Red, Green, and Blue) is another augmentation technique that can help the model generalize better. By randomizing the order of the color channels, you simulate the effect of different sensor outputs or variations in color representations.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python"># Apply Channel Shuffle
channel_shuffle_augment = Compose([
    ChannelShuffle(p=1.0)  # Shuffle the RGB channels
])

channel_shuffled_image = channel_shuffle_augment(image=image)["image"]
show(channel_shuffled_image, "Channel Shuffle")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Channel%20Shuffle.png?raw=true"><h3>Noise and Gaussian Blur</h3><p>Adding noise and applying a Gaussian blur are useful for simulating real-world noise and blurring effects that can occur in images. These transformations help the model focus on relevant features while ignoring noise.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">transform = A.Compose([
    A.GaussNoise(var_limit=(10.0, 50.0), p=1.0),
    A.GaussianBlur(blur_limit=(3, 7), p=1.0)
])

augmented = transform(image=image)["image"]
show(augmented, "Noise &amp; Gaussian Blur")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Noise%20&amp;%20Gaussian%20Blur.png?raw=true"><h3>Cutout (Coarse Dropout)</h3><p>Cutout is a form of data augmentation where rectangular sections of an image are randomly dropped out. This helps the model focus on the remaining parts of the image and improves robustness.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">transform = A.Compose([
    A.CoarseDropout(max_holes=8, max_height=32, max_width=32,
                    fill_value=0, p=1.0)
])

augmented = transform(image=image)["image"]
show(augmented, "Cutout (Coarse Dropout)")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/Cutout%20(Coarse%20Dropout).png?raw=true"><h3>CLAHE (Histogram Equalization)</h3><p>Contrast Limited Adaptive Histogram Equalization (CLAHE) is a technique used to enhance the contrast of an image. It divides the image into small tiles and applies histogram equalization to each tile.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">transform = A.Compose([
    A.CLAHE(clip_limit=4.0, tile_grid_size=(8, 8), p=1.0)
])

augmented = transform(image=image)["image"]
show(augmented, "CLAHE (Histogram Equalization)")</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/data_augmentation/CLAHE%20(Histogram%20Equalization).png?raw=true"><h2>Conclusion</h2><p>These are just a few of the many image augmentation techniques that can be used to diversify training datasets and improve model generalization. When using these augmentations, remember to apply them thoughtfully to ensure they make sense for your specific task.</p>

Solving the Problem of Limited Data with Image Augmentation

An Overview of Image Data Augmentation in Deep Learning

<h1>Non-Maximum Suppression for Object Detection</h1><p>A Practical Python Guide to Reducing Overlapping Bounding Boxes</p><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/non_maximal_suppression/befor_after_nms.drawio.png?raw=true"><p style="text-align: center">Before And After Non Maximum Suppression</p><p>In <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/EdidCGkFJIJLkz9JpQhK?type=publishedchapter">object detection</a> tasks, models often return multiple bounding boxes around a single object. These overlapping predictions can lead to ambiguity in the final output.</p><p>Non-Maximum Suppression (NMS) is a critical post-processing step used to filter these predictions, ensuring that only the most relevant bounding boxes are retained.</p><p>In this comprehensive guide, we will examine the problem NMS is designed to solve, explore the algorithm in detail, and walk through a step-by-step implementation using Python.</p><p>By visualizing bounding boxes before and after applying NMS, you will gain a practical understanding of its impact and how it improves the quality of object detection outputs.</p><h2><strong>Why Do We Need Non-Maximum Suppression?</strong></h2><p>Object detection models such as <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://docs.ultralytics.com/tasks/detect/">YOLO</a>, <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://developers.arcgis.com/python/latest/guide/how-ssd-works/">SSD</a>, and <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://arxiv.org/abs/1506.01497">Faster R-CNN</a> work by predicting multiple bounding boxes with associated confidence scores. When a model detects an object, it often produces several overlapping boxes, each with a slightly different position and score.</p><p>This redundancy is expected, it provides robustness, but we need a way to eliminate unnecessary predictions.</p><p>NMS ensures that only the highest-quality predictions remain, removing overlapping boxes that likely refer to the same object. This is essential for clarity, performance, and downstream applications such as object tracking, classification, or visualization.</p><h2><strong>The Problem: Multiple Overlapping Predictions</strong></h2><p>To better understand the issue, consider a detector that identifies a single object but produces multiple bounding boxes for it:</p><ul><li><p>Each box has a confidence score indicating the model’s certainty.</p></li><li><p>The boxes often overlap significantly, especially when the object is well-centered in the image.</p></li><li><p>Without post-processing, all of these predictions may be kept, cluttering the result.</p></li></ul><p>This is where Non-Maximum Suppression becomes essential.</p><h2><strong>What is Non-Maximum Suppression (NMS)?</strong></h2><p>Non-Maximum Suppression is a filtering algorithm that reduces redundant bounding boxes in object detection. Its purpose is to retain only the most confident detections while suppressing others that have a significant overlap with them.</p><p><strong>The NMS Algorithm (Step-by-Step):</strong></p><ol><li><p>Sort all predicted bounding boxes by their confidence scores in descending order.</p></li><li><p>Select the bounding box with the highest confidence score.</p></li><li><p>Compute the Intersection over Union (IoU) between this box and all other boxes.</p></li><li><p>Remove boxes that have an IoU greater than a predefined threshold (e.g., 0.5).</p></li><li><p>Repeat the process with the remaining boxes.</p></li></ol><p>By following this approach, the algorithm prioritizes the most accurate detections while suppressing redundant or overlapping ones.</p><h2><strong>Implementing NMS from Scratch in Python</strong></h2><p>Now that we’ve covered the theory behind Non-Maximum Suppression, let’s move into implementation. In this section, we will write Python code to implement NMS, visualize bounding boxes before and after applying it, and develop an intuitive understanding of how it works.</p><h3>1. Import Required Libraries</h3><p>We will start by importing the necessary libraries for image processing and visualization. We will also apply a dark theme to the plots to enhance visual clarity.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">import cv2
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches
plt.style.use('dark_background')</code></pre></div><h3>2. Load and Display the Sample Image</h3><p>Next, we will load an image and display it using Matplotlib. This image will serve as our base, where we will draw multiple overlapping bounding boxes.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">image = cv2.imread('sample_image.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
height, width, _ = image.shape

plt.figure(figsize=(10, 8))
plt.imshow(image)
plt.title('Sample Image', color='white')  # white title for dark background
plt.axis('off')
plt.show()</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/non_maximal_suppression/sample_image.jpg?raw=true"><p style="text-align: center">Image of an Apple from <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://www.kaggle.com/datasets/mbkinaci/fruit-images-for-object-detection">Fruit Images for Object Detection</a></p><h3>3. Define Sample Bounding Boxes (with Confidence Scores)</h3><p>We now define multiple bounding boxes around the object in the image. These boxes are represented by five values: <code>[x_min, y_min, x_max, y_max, confidence_score]</code>. These bounding boxes simulate predictions from an object detection model.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">boxes = np.array([
    [227, 62, 723, 500, 0.95],  # Ground truth/highest score
    [220, 60, 715, 495, 0.9],
    [230, 65, 725, 510, 0.85],
    [240, 70, 730, 515, 0.75],
    [300, 100, 600, 480, 0.6],  # Smaller box
    [100, 50, 600, 400, 0.5]    # Slightly shifted box
])</code></pre></div><h3>4. Visualize Bounding Boxes Before Applying NMS</h3><p>We will use the following function to draw all the predicted boxes on the image. This allows us to observe how much the boxes overlap before applying any filtering.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">def draw_boxes_on_image(img, boxes, title=''):
    img_copy = img.copy()
    fig, ax = plt.subplots(1, figsize=(10, 8))
    ax.imshow(img_copy)
    ax.set_title(title)
    
    for box in boxes:
        x1, y1, x2, y2, score = box
        rect = patches.Rectangle(
            (x1, y1), x2 - x1, y2 - y1,
            linewidth=2, edgecolor=(0, 0, 1), facecolor='none'
        )
        ax.add_patch(rect)
        ax.text(x1, y1 - 10, f'{score:.2f}', color='white',
                bbox=dict(facecolor='blue', edgecolor='none', pad=1))
    
    ax.axis('off')
    plt.savefig(f'{title}.png', bbox_inches='tight')
    plt.show()</code></pre></div><p>Now, we visualize the bounding boxes before applying Non-Maximum Suppression:</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">draw_boxes_on_image(image, boxes, title='Predicted Boxes (Before NMS)')</code></pre></div><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/non_maximal_suppression/Predicted%20Boxes%20(Before%20NMS).png?raw=true"><p style="text-align: center">Predicted Bounding Boxes Before NMS</p><h3>5. Define IoU (Intersection over Union)</h3><p>The Intersection over Union (IoU) metric is fundamental in NMS. It measures the overlap between two bounding boxes. If the overlap exceeds a certain threshold, one of the boxes will be suppressed.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">def iou(box1, box2):
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])

    inter_area = max(0, x2 - x1) * max(0, y2 - y1)
    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])
    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])

    union_area = box1_area + box2_area - inter_area
    return inter_area / union_area if union_area &gt; 0 else 0</code></pre></div><h3>6. Implementing the NMS Algorithm</h3><p>Now, we will implement the Non-Maximum Suppression algorithm. This function takes the list of boxes and applies the suppression based on the IoU threshold.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">def non_max_suppression(boxes, iou_threshold=0.3):
    if len(boxes) == 0:
        return []

    boxes = boxes[boxes[:, 4].argsort()[::-1]]  # Sort boxes by confidence
    selected_boxes = []

    while len(boxes) &gt; 0:
        chosen_box = boxes[0]
        selected_boxes.append(chosen_box)

        remaining_boxes = []
        for box in boxes[1:]:
            if iou(chosen_box, box) &lt; iou_threshold:
                remaining_boxes.append(box)
        boxes = np.array(remaining_boxes)

    return np.array(selected_boxes)</code></pre></div><p>This algorithm works by sorting the boxes based on their confidence score. The highest-scoring box is selected, and any boxes that have a high overlap (IoU) with it are removed. This process repeats until no boxes remain.</p><h3>7. Apply NMS and Visualize the Final Result</h3><p>Finally, we apply the NMS function and visualize the bounding boxes that remain after suppression.</p><div class="code-block-wrapper"><div class="code-language">Python</div><pre data-language="python"><code class="language-python">nms_boxes = non_max_suppression(boxes, iou_threshold=0.3)
draw_boxes_on_image(image, nms_boxes, title='Boxes After NMS')</code></pre></div><p>The resulting image will show only the most relevant bounding boxes, filtering out any redundant boxes that have high overlap with others. This is exactly how modern object detectors ensure that the predictions are clean and non-redundant.</p><img src="https://github.com/Brianhulela/ebook_object_detection/blob/master/chapters/non_maximal_suppression/Predicted%20Boxes%20(After%20NMS).png?raw=true"><p style="text-align: center">Predicted Bounding Boxes After NMS</p><h2><strong>Understanding the Outcome</strong></h2><p>After applying NMS, we observe that:</p><ul><li><p>Only the bounding box with the highest confidence score remains for each cluster of overlapping boxes.</p></li><li><p>The result is a cleaner and more reliable set of detections.</p></li></ul><p>This filtering process is essential in real-world scenarios where object detectors may return hundreds of overlapping predictions.</p><blockquote><p>Learn more about <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://hulela.co.za/learn/rU5hAJuIJkfAanGTspUT/ikvBmuF2FKWN04v3hnii?type=publishedchapter">Object Detection with YOLO.</a></p></blockquote><h2><strong>Conclusion</strong></h2><p>Non-Maximum Suppression is a fundamental concept in object detection. Although the idea is simple, its role is crucial for reducing noise in predictions and improving model accuracy.</p><p>By implementing it manually and visualizing the outcome, we gain a deeper appreciation of how object detectors work under the hood.</p><p>Understanding NMS equips you with the tools to customize detection outputs, debug model predictions, and experiment with alternative post-processing techniques.</p><blockquote><p>All the code for this guide can be accessed on this <a target="_blank" rel="noopener noreferrer nofollow" class="link" href="https://github.com/Brianhulela/ebook_object_detection/tree/master/chapters/non_maximal_suppression">GitHub Repository</a>.</p></blockquote>

Read More