The goals of this project was to make a pipeline that finds lane lines on the road and to reflect on my work in a written report.

My pipeline consisted of 7 steps:

1. Masking of yellow and white colors. I did experiment with HSV, HSL and RGB colorspaces, however RGB masking seemed sufficient enough for further processing. Below is the function used to apply color masks, complete with thresholds defining the color ranges.

def apply_color_masks(image):
    """Applies white and yellow colors masks"""

    mask_white = cv2.inRange(image, np.array([90, 90, 200]), np.array([255, 255, 255]))
    mask_yellow = cv2.inRange(image, np.array([215, 170, 0]), np.array([255, 255, 180]))
    mask_combined = cv2.bitwise_or(mask_white, mask_yellow)
    return cv2.bitwise_and(image, image, mask = mask_combined)

Original	White&yellow masking

2. Convertion of a masked image to grayscale colorspace as it is required in Canny Edge Detection step.

White&yellow masking	Grayscaled

3. Gaussian smoothing to prevent false detection caused by noise. This filter is built into Canny Edge Detection algorithm, however separating it as another step makes the filter more flexible.

Grayscaled	Gaussian smoothing

4. Applying Canny Edge Detection.

Gaussian smoothing	Canny edge detection

5. Masking of a front lane ahead of the car, so only this area would be processed in further steps.

Canny edge detection	Front lane masking

6. Applying Hough transform to easily aggregate and draw lane lines.

Front lane masking	Hough transform

7. Final step of overlaying the lines onto original image.

Hough transform	Overlaying

In order to draw a single line on the left and right lanes, I modified the draw_lines() function by incorporating basic line equations. For every line the slope, y-interception and length are being calculated. Slope of the line is used to make a distinction between left or right lane line. Y-axis goes from top-to-bottom of the image so negative slope results in left lane line.

if line_slope < 0:
    if not l_lines.size:
        l_lines = np.array([[line_slope, line_y_intercept, line_length]])
    l_lines = np.concatenate((l_lines, [[line_slope, line_y_intercept, line_length]]), axis = 0)
else:
    if not r_lines.size:
        r_lines = np.array([[line_slope, line_y_intercept, line_length]])
    r_lines = np.concatenate((r_lines, [[line_slope, line_y_intercept, line_length]]), axis = 0)

Next step is to calculated the weighted average of all slopes and all y_intercepts, with weights being line lengths.

wavg_l_slope = np.average(l_lines[:,0], weights=l_lines[:,2])
wavg_l_y_intercept = np.average(l_lines[:,1], weights=l_lines[:,2])

Once we have those parameters averaged we can calculate x cooridinates of the points creating the line. We assume y_0 is the bottom of the image (full height) and y_1 is around y_1 = 0.65 * y_0 (at y_1 < 0.6 * y_0 lane lines cross each other).

l_x1 = np.int_((y1 - wavg_l_y_intercept) / wavg_l_slope)
l_x2 = np.int_((y2 - wavg_l_y_intercept) / wavg_l_slope)
cv2.line(img, (l_x1, np.int_(y1)), (l_x2, np.int_(y2)), color, thickness)

Once we have line coordinates all we have to do is to draw the lines onte the original image.

Below is the process of Lane Lines Detection for all images from test_images directory.

Original	White&yellow masking	Grayscaled	Gaussian smoothing	Canny edge detection	Front lane masking	Hough transform	Overlaying

One potential shortcoming would be what would happen when the lines are occasionally not detected at all or are heavily shadowed. Another shortcoming could be detecting the lines from the wrong road surface markings. Another shortcoming could be detecting left line but not detecting the other line.

A possible improvement would be to narrow down a range of possible line slopes. Another potential improvement could be to keep memory of e.g. last 5 time frames. Then if the line is not detected it could be derived from previous time frame. In case we can detect only one of the line, we can assume the slope of other lane line since lane lines always follow the same direction.