Here I will describe how I use OpenCV for capturing RTSP streams, with purpose of motion detection.
For basic OpenCV I use these two libraries:
import cv2 import numpy as np
cv2 is OpenCV library (second version), and numpy is python numeric lybrary (useful for manipulating matrices among other things).
To initiate capture one simply does following:
cap = cv2.VideoCapture('rtsp://192.168.1.69:554/Streaming/Channels/2')
In this example I use second stream (of lower resolution) for motion detection.
From there you can get heigh and width of the frame (this will be useful later):
width = cap.get(3) height = cap.get(4)
I use BackgroundSubtractorMOG for motion detection (somewhat cheating ;)):
bg = cv2.BackgroundSubtractorMOG(100,3,0.6,30)
The magic is in parameters, I used following:
100 – history
3 – number of Gaussian mixtures
0.6 – background ratio
30 – noise strength
The numbers above are not necessarily “correct” but I came to them with error and trial (and “guestimation”).
Here is document in detail describing this algorithm: http://personal.ee.surrey.ac.uk/Personal/R.Bowden/publications/avbs01/avbs01.pdf
The actual capture loop looks something like this:
while(True): ... ret, frame = cap.read() motion = bg.apply(frame, learningRate=0.005) kernel = np.ones((3, 3), np.uint8) motion = cv2.morphologyEx(motion, cv2.MORPH_CLOSE, kernel, iterations=1) motion = cv2.morphologyEx(motion, cv2.MORPH_OPEN, kernel, iterations=1) motion = cv2.dilate(motion,kernel,iterations = 1) contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) ...
ret, frame = cap.read()
captures a single frame
motion = bg.apply(frame, learningRate=0.005)
extracts a black and white image with the background removed (learnignRate value has been chosed by error and trial).
Next four lines simply manipulate extracted image in such that it does following:
MORPH_CLOSE: removes small holes (up to 3×3 pixel, defined by kernel) within the object (“white”) in the extracted motion matrix.
MORPH_OPEN: removes small dots within the “background” (black) in the extracted motion matrix.
dilate: is making sure there all adjacent islands are joined together, so when we extract contours we get small amount of contours as result.
The “3×3 pixel” block comes from here:
kernel = np.ones((3, 3), np.uint8
The last step from processing frame is extracting the contours:
contours, hierarchy = cv2.findContours(motion, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
The extracted contours can then be iterated and hull drawn around them:
for cnt in contours: hull = cv2.convexHull(cnt)
The hull points then can be checked against the mask if motion is inside of the area of interest:
Lets define the mask as the whole frame (I am pretty sure there is a better way;)):
mask_points = [ ( 0 , 0 ), ( 1 , 0 ), ( 1 , 1 ), ( 0 , 1 ), ] mask_array= for point in mask_points: mask_array.append([[int(point * width ), int(point * height )]]) mask = np.array(mask_array, np.int32)
This looks cumbersome, but what I am achieving here is converting mask_points list of human readable relative coordinate tuples (eg: centre will be at (0.5,0.5)). Mask can be defined as a polygon with relative positioning of each corner to the frame (independent from pixel size).
We check if hull point is inside our mask
for point in hull: distance = cv2.pointPolygonTest(mask,tuple(point),1) if distance > 0: it_is_inside() and vice-versa (in case if mask is smaller than the frame):
Note: The colour is defined by this tuple: (0,255,0)
Then do display the whole thing insert this inside of the while(True):cv2.imshow('motion',frame) k = cv2.waitKey(30) & 0xff if k == 27: break
The above is basic idea behind my motion detection scripts. I have omitted a lot of glue logic and arithmetic due to my script is not ready for public display ;).