TRACK THE REGION OF INTEREST

Authors: Abhishek Kumar Annamraju, Akashdeep Singh, Adhesh Shrivastava

Hello Friends,

Lets go through interesting stuff that computer vision can provide. Confused by the title????

With the application I am going to introduce, you can track down a region from a live streaming video. Suppose you take a live stream from your web-cam, and in that window you draw a rectangle using your mouse,then in the next coming frames the application will track down that region unless and untill that part of the region is in the frame. The main crux of the application is “Good features to track” and “Optical flow”

Seems interesting!!!!!!

Download the code from here:
https://github.com/abhi-kumar/OPENCV_MISC/blob/master/tracker.cpp

Now lets understand the code:

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <string.h>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>

These libraries are the standard C and C++ libs.

#include <cv.h>
#include <highgui.h>

These libs are the ones from opencv C

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/tracking.hpp"

These libs are the ones from opencv C++,also caleed as opencv2

using namespace cv;
using namespace std;

Including the standard namespaces.

IplImage* frame, * img1;
CvPoint point;
int drag = 0;
int x_point,width_point,y_point,height_point;

Initialising the parameters to capture a video from webcam and to set the mouse.

int key = 0;
CvRect rect;
Rect region_of_interest;
int test;
Mat src,src_gray,image,src_gray_prev,src1,src_gray1,copy,copy1,frames,copy2;
int maxCorners = 23;
RNG rng(12345);
vector<Point2f> corners,corners_prev,corners_temp;
double qualityLevel = 0.01;
double minDistance = 10;
int blockSize = 3;
bool useHarrisDetector = false;
double k = 0.04;
vector<uchar> status;
vector<float> err;
float x_cord[100];
float y_cord[100];

There are the parameters we will use in “Good features to track” and “Optical flow”.

void mouseHandler(int event, int x, int y, int flags, void* param)
{
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        point = cvPoint(x, y);
        drag = 1;
    }
    
    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        img1 = cvCloneImage(frame);
        cvRectangle(img1,point,cvPoint(x, y),CV_RGB(255, 0, 0),1,8,0);
        cvShowImage("result", img1);
    }
    
    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        rect = cvRect(point.x,point.y,x-point.x,y-point.y);
		x_point = point.x;
		y_point = point.y;
		width_point = x-point.x;
		height_point = y-point.y;
        cvShowImage("result", frame);
        drag = 0;
    }

    
    if (event == CV_EVENT_RBUTTONUP)
    {
        drag = 0;
    }
}

This is the code to draw and select a region of interest from the video window.

int main(int argc, char *argv[])
{
    capture = cvCaptureFromCAM( 0 ); 
    if ( !capture ) {
        printf("Cannot open initialize webcam!\n" );
        exit(0);
    }
    cvNamedWindow( "result", CV_WINDOW_AUTOSIZE );
	int small,big; //declares integer
    
	int x = 1;

The above snippet captures the first frame.

while( key != 'q' )
    {
        frame = cvQueryFrame( capture );

These line make sure that video is available till you press the key “q”

if (rect.width>0)

To check if the rectangle has been chosen or not

if(corners.size() == 0 || x==0)
			{
				Mat frames(frame);
				src = frames.clone();
				cvtColor( src, src_gray, CV_BGR2GRAY );
				cv::Mat mask1 = cv::Mat::zeros(src.size(), CV_8UC1);  
				cv::Mat roi(mask1, cv::Rect(x_point,y_point,width_point,height_point));
				roi = cv::Scalar(255, 255, 255);
				copy1 = src.clone();		
				goodFeaturesToTrack( src_gray,
    		           corners,
    		           maxCorners,
    		           qualityLevel,
    		           minDistance,
    		           mask1,
    		           blockSize,
    		           useHarrisDetector,
    	           k );

				int rad = 3;
  				for( int i = 0; i < corners.size(); i++ )
  				   { circle( copy1, corners[i], rad, Scalar(rng.uniform(0,255), rng.uniform(0,255),
  			            rng.uniform(0,255)), -1, 8, 0 );
			
					}
				IplImage test1 = copy1;
			  	IplImage* test2 = &test1;
				x = 1;

			    cvShowImage("result", test2);
			}

If the rectangle has just been drawn in the previous frame,the above code finds good features and saves it.

else
				{
					src_gray_prev = src_gray.clone();
					corners_prev = corners;
					Mat framess(frame);
					src = framess.clone();
					cvtColor( src, src_gray, CV_BGR2GRAY ); 
					cv::Mat mask = cv::Mat::zeros(src.size(), CV_8UC1);  
					cv::Mat roi(mask, cv::Rect(x_point,y_point,width_point,height_point));
					roi = cv::Scalar(255, 255, 255);	
					Mat copy;
  					copy = src.clone();
					goodFeaturesToTrack( src_gray,
    		           corners,
    		           maxCorners,
    		           qualityLevel,
    		           minDistance,
    		           mask,
    		           blockSize,
    		           useHarrisDetector,
    		           k );
	
					calcOpticalFlowPyrLK(src_gray_prev, src_gray, corners_prev, corners, status, err);
			
  					int r = 3;
  					for( int i = 0; i < corners.size(); i++ )
    				 { circle( copy, corners[i], r, Scalar(rng.uniform(0,255), rng.uniform(0,255),
    			          rng.uniform(0,255)), -1, 8, 0 );
					 x_cord[i] = corners[i].x;
					 y_cord[i] = corners[i].y;
					 
					}			
					IplImage test3 = copy;
					IplImage* test4 = &test3;
					cvShowImage("result", test4);			
					
				}

Now once the features have been saved,it is tracked.

vSetMouseCallback("result", mouseHandler, NULL);
        key = cvWaitKey(10);
        if( (char) key== 'r' )
	{ 
		rect = cvRect(0,0,0,0); cvResetImageROI(frame);
		x = 0;
	}
        cvShowImage("result", frame);

Calling the mouse handler function and setting up the key to reset the region of interest.

This is major explanation gor the code.

Compilation and running:(for ubuntu users)
1)Save the and name it tracker.cpp
2)Open a terminal and traverse to that folder where you saved the code and type:
a)chmod +x tracker.cpp
b)g++ -ggdb `pkg-config –cflags opencv` -o `basename mouse2.cpp .cpp` mouse2.cpp `pkg-config –libs opencv`
c)./tracker

Now the video will open,select the box and play with the tracker.If you want to reset the tracker to draw a new window press the key “r”.

I hope you like the application. Will be back with a more revised and more robust application in a few days.

Thank you 🙂

Advertisements

4 thoughts on “TRACK THE REGION OF INTEREST”

  1. This is awesome. There is one point I would be happy to have. When I the region of interest isn’t in the picture anmore, it gets resetted. Is there any posibilty to prevent it form doing so?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s