Chapter 5: Line, Edge and Contours Detection

a. Laplace algorithm

I’m not going to explain how works the Laplace algorithm, but the only thing you need to know is that this algorithm is the first step for line and edge detection. This is also used by more complex algorithm included into OpenCV.

The following example show how to apply the Laplace algorithm on both a gray picture and a colour picture. As we can see lines and edges start appearing on resulting images.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

import cv2.cv as cv

im=cv.LoadImage('img/building.png', cv.CV_LOAD_IMAGE_COLOR)

# Laplace on a gray scale picture
gray = cv.CreateImage(cv.GetSize(im), 8, 1)
cv.CvtColor(im, gray, cv.CV_BGR2GRAY)

aperture=3

dst = cv.CreateImage(cv.GetSize(gray), cv.IPL_DEPTH_32F, 1)
cv.Laplace(gray, dst,aperture)

cv.Convert(dst,gray)

thresholded = cv.CloneImage(im)
cv.Threshold(im, thresholded, 50, 255, cv.CV_THRESH_BINARY_INV)

cv.ShowImage('Laplaced grayscale',gray)
#------------------------------------

# Laplace on color
planes = [cv.CreateImage(cv.GetSize(im), 8, 1) for i in range(3)]
laplace = cv.CreateImage(cv.GetSize(im), cv.IPL_DEPTH_16S, 1)
colorlaplace = cv.CreateImage(cv.GetSize(im), 8, 3)

cv.Split(im, planes[0], planes[1], planes[2], None) #Split channels to apply laplace on each
for plane in planes:
    cv.Laplace(plane, laplace, 3)
    cv.ConvertScaleAbs(laplace, plane, 1, 0)

cv.Merge(planes[0], planes[1], planes[2], None, colorlaplace)

cv.ShowImage('Laplace Color', colorlaplace)
#-------------------------------------

cv.WaitKey(0)

Additional informations:

  • The aperture size used is the default value. This is the kernel size, it is better to keep this value unless you know what you are doing.
  • To apply Laplace on a colour image we should split all the channels and then merge it back.
  • As we can see on the results the difference between the gray image and the colour image is not important. So it is more interesting to work on grayscale picture than colour images.

Original picture:

Laplace gray image:

Laplace colour image:

b. Sobel

Sobel is a well-known algorithm used for contour detection. For more information about this algorithm check the wikpedia page.  The small example below use the Sobel algorithm to put in evidence the contours of the build which works quite well.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import cv2.cv as cv

im=cv.LoadImage('img/building.png', cv.CV_LOAD_IMAGE_GRAYSCALE)

sobx = cv.CreateImage(cv.GetSize(im), cv.IPL_DEPTH_16S, 1)
cv.Sobel(im, sobx, 1, 0, 3) #Sobel with x-order=1

soby = cv.CreateImage(cv.GetSize(im), cv.IPL_DEPTH_16S, 1)
cv.Sobel(im, soby, 0, 1, 3) #Sobel withy-oder=1

cv.Abs(sobx, sobx)
cv.Abs(soby, soby)

result = cv.CloneImage(im)
cv.Add(sobx, soby, result) #Add the two results together.

cv.Threshold(result, result, 100, 255, cv.CV_THRESH_BINARY_INV)

cv.ShowImage('Image', im)
cv.ShowImage('Result', result)

cv.WaitKey(0)

c. Yet another algorithm

The last basic function that will be presented for basic edge/contours detection is the cv.MorphologyEx. The documentation available here explain how works every arguments but the one we are interested in here is CV_MOP_GRADIENT that do dilate and substract the result to an erode. Apply this filtersin this specific order has to effect to release all the contours and edges on a picture. This is based on the fact that comparison of an image and an eroded will mostly differ at edges location (where the intensity of neighboor vary more). As argument we can also provide a structuring element like a cross a diamond on which apply filters and also the number of iterations. More iterations can give more precise contours but can also erase some.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import cv2.cv as cv

image=cv.LoadImage('img/build.png', cv.CV_LOAD_IMAGE_GRAYSCALE)

#Get edges
morphed = cv.CloneImage(image)
cv.MorphologyEx(image, morphed, None, None, cv.CV_MOP_GRADIENT) # Apply a dilate - Erode

cv.Threshold(morphed, morphed, 30, 255, cv.CV_THRESH_BINARY_INV)

cv.ShowImage("Image", image)
cv.ShowImage("Morphed", morphed)

cv.WaitKey(0)

d. Line detection with Canny

Canny is an algorithm made for edge detection. This is the base algorithm for any line edge or contour detection for his accuracy and his ease to use. The example presented below will show how to detect lines into an image with the canny algorithm. Note that the canny algoirthm use the sobel algorithm in the background. To detect lines on the image we will use the cv.HoughLines2 that do the job of find lines from a “cannied” image. The example show the result using the standard HoughLines and the probabilistic way. Take a look at the documentation for more details.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

import cv2.cv as cv
import math

im=cv.LoadImage('img/road.png', cv.CV_LOAD_IMAGE_GRAYSCALE)

pi = math.pi #Pi value

dst = cv.CreateImage(cv.GetSize(im), 8, 1)

cv.Canny(im, dst, 200, 200)
cv.Threshold(dst, dst, 100, 255, cv.CV_THRESH_BINARY)

#---- Standard ----
color_dst_standard = cv.CreateImage(cv.GetSize(im), 8, 3)
cv.CvtColor(im, color_dst_standard, cv.CV_GRAY2BGR)#Create output image in RGB to put red lines

lines = cv.HoughLines2(dst, cv.CreateMemStorage(0), cv.CV_HOUGH_STANDARD, 1, pi / 180, 100, 0, 0)
for (rho, theta) in lines[:100]:
    a = math.cos(theta) #Calculate orientation in order to print them
    b = math.sin(theta)
    x0 = a * rho
    y0 = b * rho
    pt1 = (cv.Round(x0 + 1000*(-b)), cv.Round(y0 + 1000*(a)))
    pt2 = (cv.Round(x0 - 1000*(-b)), cv.Round(y0 - 1000*(a)))
    cv.Line(color_dst_standard, pt1, pt2, cv.CV_RGB(255, 0, 0), 2, 4) #Draw the line

#---- Probabilistic ----
color_dst_proba = cv.CreateImage(cv.GetSize(im), 8, 3)
cv.CvtColor(im, color_dst_proba, cv.CV_GRAY2BGR) # idem

rho=1
theta=pi/180
thresh = 50
minLength= 120 # Values can be changed approximately to fit your image edges
maxGap= 20

lines = cv.HoughLines2(dst, cv.CreateMemStorage(0), cv.CV_HOUGH_PROBABILISTIC, rho, theta, thresh, minLength, maxGap)
for line in lines:
    cv.Line(color_dst_proba, line[0], line[1], cv.CV_RGB(255, 0, 0), 2, 8)

cv.ShowImage('Image',im)
cv.ShowImage("Cannied", dst)
cv.ShowImage("Hough Standard", color_dst_standard)
cv.ShowImage("Hough Probabilistic", color_dst_proba)
cv.WaitKey(0)

Original image:

Canny image:

Standard HoughLines:

Probablistic HoughLines:

e. Contours detection

To do contours detection OpenCV provide a function called FindContours which intent to find contours in the image. Of course to some treatment should be applied to the picture in order to get a good contours detection. In the example below we first use the MorphologyEx function with CV_MOP_OPEN method and CV_MOP_CLOSE to release contours. Then we apply the FindContours function to find contours and print them on the colour image even though we work on a grayscale version of the image.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import cv2.cv as cv

orig = cv.LoadImage('img/build.png', cv.CV_LOAD_IMAGE_COLOR)
im = cv.CreateImage(cv.GetSize(orig), 8, 1)
cv.CvtColor(orig, im, cv.CV_BGR2GRAY)
#Keep the original in colour to draw contours in the end

cv.Threshold(im, im, 128, 255, cv.CV_THRESH_BINARY)
cv.ShowImage("Threshold 1", im)

element = cv.CreateStructuringElementEx(5*2+1, 5*2+1, 5, 5, cv.CV_SHAPE_RECT)

cv.MorphologyEx(im, im, None, element, cv.CV_MOP_OPEN) #Open and close to make appear contours
cv.MorphologyEx(im, im, None, element, cv.CV_MOP_CLOSE)
cv.Threshold(im, im, 128, 255, cv.CV_THRESH_BINARY_INV)
cv.ShowImage("After MorphologyEx", im)
# --------------------------------

vals = cv.CloneImage(im) #Make a clone because FindContours can modify the image
contours=cv.FindContours(vals, cv.CreateMemStorage(0), cv.CV_RETR_LIST, cv.CV_CHAIN_APPROX_SIMPLE, (0,0))

_red = (0, 0, 255); #Red for external contours
_green = (0, 255, 0);# Gren internal contours
levels=2 #1 contours drawn, 2 internal contours as well, 3 ...
cv.DrawContours (orig, contours, _red, _green, levels, 2, cv.CV_FILLED) #Draw contours on the colour image

cv.ShowImage("Image", orig)
cv.WaitKey(0)

Additional informations:

  • The image is first converted in gray because we used to work on grayscale picture
  • A simple threshold is applied to roughly get contours.
  • The OPEN and CLOSE trick is applied to get more smooth contours
  • The FindContours function is called and a list of contours is retrieved
  • Contours a finaly drawn on the colour as if we were working on it.

Original image:

After the first threshold:

After MorphologyEx filters and the second Threshold:

Contours drawn:

f. Edges detection

The last feature we are interested in is the edge detection that will be put in action with the Harris algorithm. For more technicals details about this algorithm check the wikipedia page at this address. In this example we will apply the cv.CornerHarris to the image which return roughly the corners. Then various filters are applied to reduce raw corners to keep only on pixel for each corner from which we will retrieve the coordinates and then draw them on the original image.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

import cv2.cv as cv

im = cv.LoadImage("img/build.png", cv.CV_LOAD_IMAGE_GRAYSCALE)

dst_32f = cv.CreateImage(cv.GetSize(im), cv.IPL_DEPTH_32F, 1)

neighbourhood = 3
aperture = 3
k = 0.01
maxStrength = 0.0
threshold = 0.01
nonMaxSize = 3

cv.CornerHarris(im, dst_32f, neighbourhood, aperture, k)

minv, maxv, minl, maxl = cv.MinMaxLoc(dst_32f)

dilated = cv.CloneImage(dst_32f)
cv.Dilate(dst_32f, dilated) # By this way we are sure that pixel with local max value will not be changed, and all the others will

localMax = cv.CreateMat(dst_32f.height, dst_32f.width, cv.CV_8U)
cv.Cmp(dst_32f, dilated, localMax, cv.CV_CMP_EQ) #compare allow to keep only non modified pixel which are local maximum values which are corners.

threshold = 0.01 * maxv
cv.Threshold(dst_32f, dst_32f, threshold, 255, cv.CV_THRESH_BINARY)

cornerMap = cv.CreateMat(dst_32f.height, dst_32f.width, cv.CV_8U)
cv.Convert(dst_32f, cornerMap) #Convert to make the and
cv.And(cornerMap, localMax, cornerMap) #Delete all modified pixels

radius = 3
thickness = 2

l = []
for x in range(cornerMap.height): #Create the list of point take all pixel that are not 0 (so not black)
    for y in range(cornerMap.width):
        if cornerMap[x,y]:
            l.append((y,x))

for center in l:
    cv.Circle(im, center, radius, (255,255,255), thickness)


cv.ShowImage("Image", im)
cv.ShowImage("CornerHarris Result", dst_32f)
cv.ShowImage("Unique Points after Dilatation/CMP/And", cornerMap)

cv.WaitKey(0)

After Harris Corner Detection:

Single points Corners:

Corners drawn:

<<Histogram And Backprojection | Home | Object Detection>>