Week 3 - Corner Detection

Welcome to my week 3 blog post! This week, I focused on finding ways to get the corners found in the images by tweaking the parameters for Hough Line Transform, as well as trying different methods.

Get hands on with my program!

Just as last week, I deployed the program I wrote as a web app. You may find it here: https://senior-project.jessexu.me/?file_selection=Week+3

You can now select a different sample image to experiment with in the variable control panel (but I will use the default one throughout this blog post). You can also tweak the parameters for Hough Line Transform there.

Solving problem in contour extraction

Last week, I had a first glance at Hough Line Transform. However, the results I got from it were not impressive. The detected lines were short and piled up, which are no what we wanted. We want to get long and clean lines from the contours, so that we can calculate the corners of the shapes by taking the intersection of the lines.

Previous Hough Line Transform Results

I soon found the issue in my code that caused a line to be detected multiple times. I was using cv2.findContours() then cv2.drawContours() to find and draw the contours. However, I drew the contours with a line thickness of 3 to make it look more obvious, which makes Hough Line Transform think that there are multiple short lines there. After changing the line thickness to 1, many repeated lines were eliminated.

# cv2.drawContours() usage:
# drawContours: (image, contours, contourIdx, color, thickness=..., ...) 

# My previous code: 
img = cv2.drawContours(blank, contours, -1, (255, 255, 255), 3)

# Improved code:
img = cv2.drawContours(blank, contours, -1, (255, 255, 255), 1) # Changed thickness

However, when I was trying to solve this problem, I stumbled upon an algorithm called Canny edge detection. It is conveniently available in OpenCV as the method cv2.Canny(), it produces almost identical results with the combination of cv2.findContours() and cv2.drawContours() if not better. So, I decided to use Canny in the end.

Canny and findContours comparison

Tweaking Hough Line Transform

After solving the problem in contour extraction, we got better results with Hough Line Transform. However, the results are still far from usable for corner detection. The following picture is about the best I got from tweaking the various parameters for the Hough Line Transform function.

Hough Line Transform after contour extraction

The lines are not piled up anymore, but there are still many short lines that did not merge with the longer lines. It’s a tricky situation: if I turn up the minimum line length parameter, then some line segments don’t get detected and leaving it blank; if I turn up the longest gap parameter, the short gaps in between the shapes gets connected and recognized as a line.

I’ve also looked up the meaning of the parameters and the mechanism behind Hough Line Transform, but I still cannot find the best parameters that could give satisfying results. I believe it’s due to the fact that the lines are not exactly straight and that they have rounded corners.

# My understanding of the parameters
cv2.HoughLinesP(
  image, # Input image
    rho, # Step size for line searching (pixel)
    theta, # Angle step size for line searching (radians) (I set it to pi/360)
    threshold, # Threshold for accumulation of pixels to form a line
    lines, # Limit the number of lines to be detected
    minLineLength, # Minimum length of line, lines with length shorter than this will be discarded
    maxLineGap, # Maximum distance between lines to merge the two lines
)

Trying other corner detection algorithms

Since I can’t get satisfying results from tweaking Hough Line Transform, I turn to find other corner detection algorithms. I found two that are commonly used: Harris corner detection and Shi-Tomasi corner detection.

Harris Corner Detection

The code is quiet simple, the algorithm is also implemented as a method in OpenCV. See code below.

corners = cv2.cornerHarris(img, blockSize=5, ksize=5, k=0.04) # Find the corners with Harris
harris_img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR) # Convert the grayscale image to colored image so we can draw the corners found
harris_img[corners > 0.1*corners.max()] = [0, 255, 0] # Draw the corners as green points

Sadly, it does not produce satisfying results either. Some corners have multiple points marked as corner, and some corners have no points marked.

Harris Corner Detection

Although there are parameters for the cv2.cornerHarris() that I can tweak, the underlying mechanism of the algorithm is too hard for me to understand since it requires some knowledge in matrix.

Shi-Tomasi Corner Detection

So, I decide to try another corner detection algorithm, and I found Shi-Tomasi corner detection. The code is also quiet simple:

corners_s = cv2.goodFeaturesToTrack(img, maxCorners=100, qualityLevel=0.01, minDistance=0) # Find corners
corners_s = np.int0(corners_s) # Convert the format of the results
shitomasi_img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR) # Convert the grayscale image to colored image so we can draw the corners found
for i in corners_s:
    x, y = i.ravel()
    cv2.circle(shitomasi_img, (x, y), 5, (0, 255, 0), -1) # Draw the points

Unfortunately, the results aren’t good either. The mechanism behind it is also quiet complicated that I couldn’t understand.

Shi-Tomasi Corner Detection

Summary

This week, I tried multiple ways to detect corners of the shapes, but with little success. Next week, I will continue working on finding better solutions to this topic.