Here I will describe how I use OpenCV for capturing RTSP streams, with purpose of motion detection.
For basic OpenCV I use these two libraries:
import cv2
import numpy as np
|
cv2 is OpenCV library (second version), and numpy is python numeric lybrary (useful for manipulating matrices among other things).
To initiate capture one simply does following:
cap = cv2.VideoCapture('rtsp://192.168.1.69:554/Streaming/Channels/2')
|
In this example I use second stream (of lower resolution) for motion detection.
From there you can get heigh and width of the frame (this will be useful later):
width = cap.get(3)
height = cap.get(4)
|
I use BackgroundSubtractorMOG for motion detection (somewhat cheating ;)):
bg = cv2.BackgroundSubtractorMOG(100,3,0.6,30)
|
The magic is in parameters, I used following:
100 – history
3 – number of Gaussian mixtures
0.6 – background ratio
30 – noise strength
The numbers above are not necessarily “correct” but I came to them with error and trial (and “guestimation”).
Here is document in detail describing this algorithm: http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf
The actual capture loop looks something like this:
while(True):
...
ret, frame = cap.read()
motion = bg.apply(frame, learningRate=0.005)
kernel = np.ones((3, 3), np.uint8)
motion = cv2.morphologyEx(motion, cv2.MORPH_CLOSE, kernel, iterations=1)
motion = cv2.morphologyEx(motion, cv2.MORPH_OPEN, kernel, iterations=1)
motion = cv2.dilate(motion,kernel,iterations = 1)
contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
...
|
captures a single frame
motion = bg.apply(frame, learningRate=0.005)
|
extracts a black and white image with the background removed (learnignRate value has been chosed by error and trial).
Next four lines simply manipulate extracted image in such that it does following:
MORPH_CLOSE: removes small holes (up to 3×3 pixel, defined by kernel) within the object (“white”) in the extracted motion matrix.
MORPH_OPEN: removes small dots within the “background” (black) in the extracted motion matrix.
dilate: is making sure there all adjacent islands are joined together, so when we extract contours we get small amount of contours as result.
The “3×3 pixel” block comes from here:
kernel = np.ones((3, 3), np.uint8
|
The last step from processing frame is extracting the contours:
contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
|
The extracted contours can then be iterated and hull drawn around them:
for cnt in contours:
hull = cv2.convexHull(cnt)
|
The hull points then can be checked against the mask if motion is inside of the area of interest:
Lets define the mask as the whole frame (I am pretty sure there is a better way;)):
mask_points = [
( 0 , 0 ),
( 1 , 0 ),
( 1 , 1 ),
( 0 , 1 ),
]
mask_array=[]
for point in mask_points:
mask_array.append([[int(point[0] * width ), int(point[1] * height )]])
mask = np.array(mask_array, np.int32)
|
This looks cumbersome, but what I am achieving here is converting mask_points list of human readable relative coordinate tuples (eg: centre will be at (0.5,0.5)). Mask can be defined as a polygon with relative positioning of each corner to the frame (independent from pixel size).
We check if hull point is inside our mask
for point in hull:
distance = cv2.pointPolygonTest(mask,tuple(point[0]),1)
if distance > 0:
it_is_inside()
and vice-versa (in case if mask is smaller than the frame):
[py]for point in mask:
distance = cv2.pointPolygonTest(hull,tuple(point[0]),1)
if distance > 0:
it_is_inside()
|
The above will tell if the motion contour extracted is within of area interest.
In addition to the checking if the motion happens withing of area of interest the severity/size of motion can be calculated by calculating the area of the hull via the following:
area += cv2.contourArea(hull)
|
which then can be compared with the total area:
surface = cv2.contourArea(mask)
|
The ratio can be converted to a percentage value and thus be used to trigger the recording if the value is above certain threshold:
relative = area * 100.0 / surface
|
I use ffmpeg for actual recording (it is way more efficient than dumping frames from HD OpenCV capture). I simply launch an ffmpeg subprocess when motion is detected and send a SIGTERM when motion is over:
p=subprocess.Popen(record_cmd,shell=False)
....
p.terminate()
|
Note: the ffmpeg will cleanly close the recording if it sent the SIGTERM (opposed to SIGKILL).
For debug and entertainment purposes the image could be displayed via following:
To draw contours and hulls:
for cnt in contours:
hull = cv2.convexHull(cnt)
cv2.drawContours(f, [cnt], 0, (0,255,0),1)
cv2.drawContours(f,[hull],0,(0,0,255),1)
|
Note: The colour is defined by this tuple: (0,255,0)
Then do display the whole thing insert this inside of the while(True):
cv2.imshow('motion',frame)
k = cv2.waitKey(30) & 0xff
if k == 27:
break
|
The above is basic idea behind my motion detection scripts. I have omitted a lot of glue logic and arithmetic due to my script is not ready for public display ;).
Leave a Reply