Lazy loaded imageDIP III Edge Detection

I. Motivation

Edge detection is a common image processing technique used to identify boundaries in an image, which can be used for preserving the main structure of the image and extracting features for high-level tasks like object recognition, activity detection, segmentation, etc. Edge detection can be done locally or globally. In a local sense, the edges are considered changes of intensity in an image; in a global sense, the edges should be the boundaries between objects and the background. After using this technique to highlight and extract useful information such as curves, corners, and structures from an image, the following application can be conducted based on them. For example, fingerprint recognition is basically conducted based on the contour of the fingerprint, where edge detection can be used as the first step to extract the structures and curves of fingers. Obviously, a better edge detection algorithm can help with the accuracy of this application, which is important for security, criminal investigation, and even our daily lives.

II. Algorithm

(a) Sobel Edge Detector

The overall steps of Sobel edge detection are:
  1. (Optional) converting the RGB images to grayscale if it’s in RGB;
  1. padding the image with a padding size of 1;
  1. calculating x-gradient and y-gradient maps of the image with Sobel kernels by convolution operation;
  1. (Optional) normalizing the x- and y-gradient maps for visualization;
  1. calculating magnitude map with un-normalized x- and y-gradient maps, then normalizing the calculated magnitude map;
  1. choosing a percentage as a threshold to get the final binarized edge image, where black pixels represent the edge and white pixels are the background.
Sobel kernels. There are two Sobel kernels for calculating the first derivatives: row-gradient kernel and column-gradient kernel. They can be visualized as follows.
Figure 1. (left) row-gradient kernel
Figure 1. (left) row-gradient kernel
(right) column-gradient kernel
(right) column-gradient kernel
Then use these two kernels as convolution matrices to calculate gradient maps.
Padding with "reflect" mode. The reason for choosing the "reflect" mode for padding is simple: make the gradients on edges equal to 0. If we choose "symmetric" mode or any other constants such as 0, we may get a high gradient value at the edge.
Normalization. Under this context, unlike the standard normalization that normalizes data to a range of (0,1), we normalize the range of pixel values to 0 – 255 before storing it as an image for visualization. It can be formulated as follows.
Where the and are the original and normalized values at the pixel location of , respectively. The and are the y-axis (height) and x-axis (width) locations in the image. The and are the maximum and minimum pixel values in the original image, respectively.
Conversion from RGB to gray. This is formulated as follows.
Magnitude map. To get the magnitude map of image, the following formula can be used:
Where the represent the magnitude of the gradient at ; the and are the gradient in y-axis and x-axis, respectively.
Thresholding. After calculating the magnitude map and normalizing it to 0 - 255, we can choose a percentage to decide how many pixels are passed, and the left is considered as edges. For example, if we choose 90%, 90% of pixels will be filtered, and only 10% will be left as edge. To achieve this, we can calculate the cumulative distribution function (CDF) for the gradient magnitude map, then find the y value that is equal to 90% and its corresponding x value. The x value will be the final value threshold for determining the edges. If the pixel value is higher than the threshold, then it’s edge and set it to 0; otherwise, it’s background and set it to 255.
<ins/>

(b) Canny Edge Detector

The overall steps for canny detection are:
  1. (optional) converting RGB image to gray-level image if needed;
  1. (optional) denoising image with a Gaussian filter, where the kernel size is adjustable;
  1. finding the gradient maps in the x-axis and y-axis by using the Sobel detector and calculating the corresponding magnitude map and orientation map;
  1. applying the Non-maximum Suppression (NMS) to eliminate some of the pixels in the gradient magnitude map; (5) applying the hysteresis (double) thresholding to get the final edge map.
Derivative of Gaussian kernel. The steps of 3 are to compute the gradient maps, which are done by using the derivative of Gaussian. A good approximation of the derivative of Gaussian is the Sobel kernels, which is also the operation that the OpenCV C++ used for gradient calculation.
NMS. The idea behind this NMS is to remove some unwanted pixels that are near the edge but are not part of the edge. To do this, an orientation map is needed. For each pixel, we find its near pixels by using the orientation map, where these pixels are in the line normal to the edge. Then, we check if the current pixel has the largest gradient magnitude. If yes, then we keep it as the edges; if no, we remove it. After all this process, we will end up with a thinner magnitude map.
Figure 2. Source: Forsyth, D.A. and Ponce, J., 2002. Computer vision: a modern approach. prentice hall professional technical reference.
Figure 2. Source: Forsyth, D.A. and Ponce, J., 2002. Computer vision: a modern approach. prentice hall professional technical reference.
Hysteresis Thresholding. This is also called double thresholding because we use two threshold parameters to get the final edge map instead of one, which can be denoted by maxVal and minVal. For any pixel values that are above maxVal, then we consider it as the edge; for pixel values that are below minVal, we consider it as non-edge. For the pixel values that are within the range of minVal and maxVal, they are edge candidates. We only keep the candidate pixels that are directly connected to the edge pixels (the ones that are above maxVal) to be the edge. This behavior can help reduce the discontinuity between edges that are acquired by the single hard thresholding like in the Sobel edge detector and also remove unwanted noise.
We will use OpenCV’s implementation, and there is an additional argument in the Canny function of OpenCV C++: L2gradient. This is a flag of whether to use the L2 norm to calculate the gradient. We set this to be true for consistency with the Sobel magnitude calculation, as mentioned in the previous section.
Note: the higher threshold maxVal could be even higher than 255 because the thresholding is not applied on the original image but the raw magnitude map, which may have value that are higher than 255. Unlike Sobel detector where the magnitude map is normalized to 0 - 255 and then apply thresholding.
<ins/>

III. Experiment

(a) Sobel Edge Detector

Example Code (C++)
  • Sobel gradient kernels
    • Sobel magnitude and thresholding
      Inputs
      notion image
      notion image
      Results
      Figure 3. The left and middle are the normalized x-gradient and y-gradient maps for the tiger image; the right is the normalized gradient magnitude map.
      Figure 3. The left and middle are the normalized x-gradient and y-gradient maps for the tiger image; the right is the normalized gradient magnitude map.
      Figure 4. The left and middle are the normalized x-gradient and y-gradient maps for the pig image; the right is corresponding normalized gradient magnitude map.
      Figure 4. The left and middle are the normalized x-gradient and y-gradient maps for the pig image; the right is corresponding normalized gradient magnitude map.
      Figure 5. From left to right are the final edge map with thresholding percentage of 0.87, 0.90, 0.92, 0.95, and 0.97, respectively.
      Figure 5. From left to right are the final edge map with thresholding percentage of 0.87, 0.90, 0.92, 0.95, and 0.97, respectively.
      Figure 6. The potential best edge map with the percentage of thresholding chosen to be 0.95.
      Figure 6. The potential best edge map with the percentage of thresholding chosen to be 0.95.
      The results with thresholding percentages of 0.87, 0.90, 0.92, 0.95, and 0.97 are shown in Figure 5. As the percentage becomes higher, the detected edges become fewer. Simultaneously, the background noises also become fewer. Therefore, after a trade-off between clearer edges of main objects and less background noise, the result is probably better when the thresholding percentage is chosen to be 0.95, as shown in Figure 6, where the main contour of subjects is still there but the noise is much less.
      To reduce noise, we can apply a Gaussian filter or Laplacian of Gaussian (LoG, utilizing the second-order derivatives) to the original images and then apply the Sobel edge detector.
      <ins/>

      (b) Canny Edge Detector

      Example Code (C++)
      Inputs
      notion image
      notion image
      Results
      Figure 7. Three results from using three pairs of high and low thresholding in Canny detection. The left column using (50, 200) as threshold, where 50 is the low thresholding value and 200 is the high thresholding value; the middle column using (50, 300); the right column using (150, 300).
      Figure 7. Three results from using three pairs of high and low thresholding in Canny detection. The left column using (50, 200) as threshold, where 50 is the low thresholding value and 200 is the high thresholding value; the middle column using (50, 300); the right column using (150, 300).
      For the left column in Figure 7, because of relatively lower thresholds of both minVal and maxVal, there are many background pixels classified as the edge. In the middle column, the minVal is kept the same but the maxVal is 100 higher. In such a case, the bar of being directly considered as edge is much higher, so the resulting image has much fewer unwanted edges appearing in the background. As for the right column, the minVal is 100 higher, and the maxVal is kept the same as the middle column. Here, the pixels that are directly considered as the edge is not changed. But because of the higher minVal, which reduces the range of edge candidates, fewer pixels candidates have the opportunity to be considered as the edge. So we can clearly see that the background noise is much less compared to the result in the middle column.
      One of the advantages of the Canny detector is that it removes many outliers and background noise in the final edge map compared to the Sobel edge detector. With a proper parameter choice, the resulting edge map can be more pleasing. However, the complexity of the Canny detector is higher than Sobel's because of
      • the additional Gaussian denoising step,
      • the NMS operation, and
      • the hysteresis thresholding.
      The NMS will add one more iteration through the whole magnitude map to reduce some unwanted pixels; the hysteresis thresholding also adds one more iteration because we need to first determine if it can be directly considered as the edge (if above maxVal) for each pixel, then we determine if it’s edge candidate (if above minVal) and if it’s connected to the edge (the pixels that have values above maxVal).

      IV. Source Code

      All inputs and results are in .raw file form and we provide an online tool to visualize it: raw2image.
      Colab link. Warning: the installation of OpenCV on Colab requires more than one hour. If you’d like to run the code, installing locally with for example Docker is recommended.
      Prev
      DIP II Image Denoising
      Next
      DIP IV Digital Half-toning
      Loading...
      Article List
      LLM Learning Roadmap
      ✨ Awesome-Anything
      Lazy loaded image Awesome-AI-Tutorials
      Lazy loaded image Awesome-AI-Researchs
      Lazy loaded image Awesome-GitHub-Repos
      Lazy loaded image Awesome-AI-Examples
      🖼️ Digital Image Processing
      Lazy loaded image DIP I Image Demosaicing and Histogram Manipulation
      Lazy loaded image DIP II Image Denoising
      Lazy loaded image DIP III Edge Detection
      Lazy loaded image DIP IV Digital Half-toning
      Lazy loaded image DIP V Geometric Image Modification
      🍃 LLM Components
      Lazy loaded image 1.1 Tokenization
      Lazy loaded image 1.2 Embedding
      Lazy loaded image 1.3 Attention
      Lazy loaded image 1.4 FFN, Residual Addition, and LN
      Lazy loaded image 1.5 Positional Encoding
      Lazy loaded image 1.6 Architecture & Decoding Policy
      🌱 LLM Pre-training
      Lazy loaded image 2.1 Pre-training Definition
      Lazy loaded image 2.2 Pre-training Data
      Lazy loaded image 2.3 Pre-training Workflow
      Lazy loaded image 2.4 Continued Pre-training (CPT)
      Lazy loaded image 2.5 Pre-training Evaluation
      ☘️ LLM Post-Training
      Lazy loaded image 3.1 Supervised Fine-Tuning (SFT)
      Lazy loaded image 3.2 Reinforcement Learning
      Lazy loaded image 3.3 RLHF: Reinforcement Learning from Human Feedback
      Lazy loaded image 3.4 Optimizing PPO-Based Algorithms
      Lazy loaded image 3.5 Direct Preference Optimization (DPO)
      🍀 LLM Popular Models
      Lazy loaded image 4.1 BERT & Variants
      Lazy loaded image 4.2 PaLM Series
      Lazy loaded image 4.3 GPT Series
      Lazy loaded image 4.4 LLaMA Series
      Lazy loaded image 4.5 GLM Series
      Lazy loaded image 4.6 Qwen Series
      Lazy loaded image 4.7 DeepSeek Series
      Lazy loaded image 4.8 MOE Series
      Lazy loaded image 4.9 Other
      🪴 LLM Applications
      Lazy loaded image 5.1 Prompt Engineering
      Lazy loaded image 5.2 LLM-based Agents
      Lazy loaded image 5.3 Retrieval-Augmented Generation (RAG)
      Lazy loaded image 5.4 Enterprise-level RAG Practice (I)
      Lazy loaded image 5.5 Enterprise-level RAG Practice (II)
      🌿 LLM Optimization
      Lazy loaded image 6.1 GPU Usage Analysis
      Lazy loaded image 6.2 Latency Analysis for Inference
      Lazy loaded image 6.3 FlashAttention
      Lazy loaded image 6.4 PagedAttention
      Lazy loaded image 6.5 Packing for Efficient Training
      Lazy loaded image 6.6 PEFT
      🌾 LLM Compression
      Lazy loaded image 7.1 Quantization
      Lazy loaded image 7.2 Pruning
      Lazy loaded image 7.3 Knowledge Distillation
      🌵 LLM Hands-on Practice
      Lazy loaded image 8.1 nanoGPT: Training a GPT from Scratch
      Lazy loaded image 8.2 Instruction Fine-Tuning Qwen3-0.6B: A Minimal Working Example
      Lazy loaded image 8.3 OpenRLHF: SFT Qwen3-0.6B and Code Analysis
      Lazy loaded image 8.4 OpenRLHF: Reward Model Training and Code Analysis
      Lazy loaded image 8.5 OpenRLHF: PPO Tutorial and Code Analysis
      🌴 LLM Must-read Papers
      Lazy loaded image 9.1 Technical Reports Explained
      Lazy loaded image 9.2 MedLLM
      Lazy loaded image 9.3 o1 Techinical Paper
      Lazy loaded image 9.4 Prompting
      Lazy loaded image 9.5 LLM-as-Judge
      🌳 LLM Q&A
      Lazy loaded image 10.1 Why GRPO training loss starts at 0
      Lazy loaded image 10.2 VLM-R1 Reproduction Results
      Lazy loaded image 10.3 KL(P||Q) or KL(Q||P)?
      Lazy loaded image 10.4 Left Padding or Right Padding?
      Lazy loaded image 10.5 What is QK-Norm?
      Lazy loaded image 10.6 Why LLM training rarely adopts Dropout?
      Lazy loaded image 10.7 Dropout and DropPath
      🐝 VLM Image Encoders
      Lazy loaded image 11.0 Introduction
      Lazy loaded image 11.1 Vision Transformer
      Lazy loaded image 11.2 CLIP
      Lazy loaded image 11.3 BLIP
      Lazy loaded image 11.4 EVA
      Lazy loaded image 11.6 BLIP-2
      Lazy loaded image 11.7 SigLIP
      📝 MISC.
      🤔 Large Language Models Pass the Turing Test.
      Lazy loaded image OpenAI ChatGPT Progression
      ⏳ Claude Sonnet 4 Test with Sandtris
      🔍 What DeepSeek-R1's Surge Teaches Us About SEO
      💰 CivitAI’s Payment Issue
      🎓 arXivSub, an Academia Assistant