Sergei's incoherrent ramblings

Here I will describe how I use OpenCV for capturing RTSP streams, with purpose of motion detection.

For basic OpenCV I use these two libraries:

import cv2
import numpy as np

cv2 is OpenCV library (second version), and numpy is python numeric lybrary (useful for manipulating matrices among other things).

To initiate capture one simply does following:

cap = cv2.VideoCapture('rtsp://192.168.1.69:554/Streaming/Channels/2')

In this example I use second stream (of lower resolution) for motion detection.

From there you can get heigh and width of the frame (this will be useful later):

width = cap.get(3)
height = cap.get(4)

I use BackgroundSubtractorMOG for motion detection (somewhat cheating ;)):

bg = cv2.BackgroundSubtractorMOG(100,3,0.6,30)

The magic is in parameters, I used following:
100 – history
3 – number of Gaussian mixtures
0.6 – background ratio
30 – noise strength
The numbers above are not necessarily “correct” but I came to them with error and trial (and “guestimation”).
Here is document in detail describing this algorithm: http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf

The actual capture loop looks something like this:

while(True):
    ...
    ret, frame = cap.read()
    motion = bg.apply(frame, learningRate=0.005)
    kernel = np.ones((3, 3), np.uint8)
    motion = cv2.morphologyEx(motion, cv2.MORPH_CLOSE, kernel, iterations=1)
    motion = cv2.morphologyEx(motion, cv2.MORPH_OPEN, kernel, iterations=1)
    motion = cv2.dilate(motion,kernel,iterations = 1)
    contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    ...

ret, frame = cap.read()

captures a single frame

motion = bg.apply(frame, learningRate=0.005)

extracts a black and white image with the background removed (learnignRate value has been chosed by error and trial).

Next four lines simply manipulate extracted image in such that it does following:
MORPH_CLOSE: removes small holes (up to 3×3 pixel, defined by kernel) within the object (“white”) in the extracted motion matrix.
MORPH_OPEN: removes small dots within the “background” (black) in the extracted motion matrix.
dilate: is making sure there all adjacent islands are joined together, so when we extract contours we get small amount of contours as result.

The “3×3 pixel” block comes from here:

kernel = np.ones((3, 3), np.uint8

The last step from processing frame is extracting the contours:

contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)

The extracted contours can then be iterated and hull drawn around them:

for cnt in contours:
    hull = cv2.convexHull(cnt)

The hull points then can be checked against the mask if motion is inside of the area of interest:

Lets define the mask as the whole frame (I am pretty sure there is a better way;)):

mask_points = [
( 0 , 0 ),
( 1 , 0 ),
( 1 , 1 ),
( 0 , 1 ),
]

mask_array=[]
for point in mask_points:
    mask_array.append([[int(point[0] * width ), int(point[1] * height )]])
mask = np.array(mask_array, np.int32)

This looks cumbersome, but what I am achieving here is converting mask_points list of human readable relative coordinate tuples (eg: centre will be at (0.5,0.5)). Mask can be defined as a polygon with relative positioning of each corner to the frame (independent from pixel size).

We check if hull point is inside our mask

for point in hull:
    distance = cv2.pointPolygonTest(mask,tuple(point[0]),1)
    if distance > 0:
        it_is_inside()

and vice-versa (in case if mask is smaller than the frame):

for point in mask:
    distance = cv2.pointPolygonTest(hull,tuple(point[0]),1)
    if distance > 0:
        it_is_inside()

The above will tell if the motion contour extracted is within of area interest.
In addition to the checking if the motion happens withing of area of interest the severity/size of motion can be calculated by calculating the area of the hull via the following:

area += cv2.contourArea(hull)

which then can be compared with the total area:

surface = cv2.contourArea(mask)

The ratio can be converted to a percentage value and thus be used to trigger the recording if the value is above certain threshold:

relative = area * 100.0 / surface

I use ffmpeg for actual recording (it is way more efficient than dumping frames from HD OpenCV capture). I simply launch an ffmpeg subprocess when motion is detected and send a SIGTERM when motion is over:

p=subprocess.Popen(record_cmd,shell=False) # motion start
....
p.terminate() # motion stop

Note: the ffmpeg will cleanly close the recording if it sent the SIGTERM (opposed to SIGKILL).

For debug and entertainment purposes the image could be displayed via following:

To draw contours and hulls:

for cnt in contours:
    hull = cv2.convexHull(cnt)
    cv2.drawContours(f, [cnt], 0, (0,255,0),1)
    cv2.drawContours(f,[hull],0,(0,0,255),1)

Note: The colour is defined by this tuple: (0,255,0)

Then do display the whole thing insert this inside of the while(True):

cv2.imshow('motion',frame)
k = cv2.waitKey(30) & 0xff
if k == 27:
    break

The above is basic idea behind my motion detection scripts. I have omitted a lot of glue logic and arithmetic due to my script is not ready for public display ;).