WORKING WITH OPENCV IN WINDOWS

Hello Friends,

Till today I was working with OpenCV on ubuntu platform. But we should also have knowledge about using the OpenCV libraries with Visual studio in Windows. Here I will be explaining how to integrate OpenCV with Windows 32-bit and 64-bit versions.

Step-I:

Download OpenCV from here: http://opencv.org/downloads.html

Double click on the downloaded exe file and when it is is being extracted select the folder name as “opencv” and extract it in your C drive.

Step-II:

Now we need to add the path of the extracted libraries to Environment variables.
Go to “Control Panel” >> “System and Security” >> “System” and click on “advanced system properties” and a window will appear(the one inside the red rectangle):
Screenshot (18)

Now click on the “environment variable”, in the “system variables” select “Path” and edit it by adding these lines to it:
64-Bit users
C:\opencv\build\x64\vc10\bin;C:\opencv\build\common\tbb\intel64\vc10;

32-Bit users
C:\opencv\build\x86\vc10\bin;

Assuming that you are using Visual Studio 2010
You are basically adding path to bin in the environment variables, so make sure the path is correct and make appropriate changes if necessary.

Click on “OK” in every window that has been opened to make the changes.

Step-III:
Open Visual studio 2010 and create a new visual c++ win32 console application project. Name it something and create it.

Now in that select the “View” menu and click on “Property Manager”.

Step-IV:
Only for 64-Bit version users.

Select the “Project” Menu and click on “Properties”.
Screenshot (19)

Click on the “Configuration Manager”
Select the “Active Solution Configuration” as “Release”
Select the “Active Solution Platform” and click on “”
Screenshot (20)

And in that Select the new platform as “X64”, click on “ok” and close the configuration manager.

Step-V:
Now in the properties window,
Select the “Configuration Properties” >> “C/C++” >> “General” and in that edit the “Additional Include libraries” by adding these two lines to it:
Screenshot (21)

C:\opencv\build\include\opencv;C:\opencv\build\include
Here we are adding path to include folders,make sure the path is correct as per your computer.

Select the “Configuration Properties” >> “C/C++” >> “Preprocessor” and in that select the “Preprocessor definition”,edit it by adding this to it :
_CRT_SECURE_NO_WARNINGS

Step-VI:
Now in the properties window,
Select the “Configuration Properties” >> “Linker” >> “General” and select and edit the “Additional Library Directories”
Screenshot (22)

64-Bit version users add this line to it:
C:\opencv\build\x64\vc10\lib;

32-Bit version users add this line to it:
C:\opencv\build\x86\vc10\lib;

Make sure the path to lib is correct as per your settings.

Select the “Configuration Properties” >> “Linker” >> “Input” and click on “Additional dependencies” and edit it:
Screenshot (23)

opencv_core246.lib
opencv_imgproc246.lib
opencv_highgui246.lib
opencv_ml246.lib
opencv_video246.lib
opencv_features2d246.lib
opencv_calib3d246.lib
opencv_objdetect246.lib
opencv_contrib246.lib
opencv_legacy246.lib
opencv_flann246.lib

Note: In …..246.lib,246 is the version of opencv,for me its OpenCV-2.4.6 ,So make appropriate changes according to the version you have downloaded.

Click on “apply” and “ok”

Now in the code DELETE everything and copy the test code from here: https://github.com/abhi-kumar/OPENCV_MISC/blob/master/tracker.cpp
Note:on the top of the code add this line: #include “stdafx.h”

Keep the mode as release and run it:
Screenshot (24)

Get the details of the code from : https://abhishek4273.wordpress.com/2014/07/05/track-the-region-of-interest/

So, now you have integrated OpenCV with Windows Visual Studio

Thanks 🙂

Advertisements

PLAYING WITH STEREO IMAGES AND DEPTH MAP

Hello friends,

Computer stereo vision is the extraction of 3D information from digital images, such as obtained by a CCD camera. By comparing information about a scene from two vantage points, 3D information can be extracted by examination of the relative positions of objects in the two panels. This is similar to the biological process Stereopsis.

Stereo means solid and opsis means sight. Left and right images are shifted by some amount,this shift is disparity. This best disparity is used to obtain distances of object from camera,via depth maps.

Before finding the disparity, a matching is done between the two images, so that the similarities are found out, then the left out differences are nothing but disparities.

Well I present to you the code to find depth map using Stereo_BM through OpenCV.

Download the code here: https://github.com/abhi-kumar/OPENCV_MISC/blob/master/stereoBM.cpp

Compilation and usage:
open and terminal,move to the downloaded file’s location and type
a)chmod +x stereoBM.cpp
b)g++ -ggdb `pkg-config –cflags opencv` -o `basename stereoBM.cpp .cpp` stereoBM.cpp `pkg-config –libs opencv`
c)./stereoBM
You can adjust the parameters in the image window.
You will get three windows like this :
Screenshot from 2014-07-09 21:45:43

Thank you 🙂

TRACK THE REGION OF INTEREST

Authors: Abhishek Kumar Annamraju, Akashdeep Singh, Adhesh Shrivastava

Hello Friends,

Lets go through interesting stuff that computer vision can provide. Confused by the title????

With the application I am going to introduce, you can track down a region from a live streaming video. Suppose you take a live stream from your web-cam, and in that window you draw a rectangle using your mouse,then in the next coming frames the application will track down that region unless and untill that part of the region is in the frame. The main crux of the application is “Good features to track” and “Optical flow”

Seems interesting!!!!!!

Download the code from here:
https://github.com/abhi-kumar/OPENCV_MISC/blob/master/tracker.cpp

Now lets understand the code:

#include <stdlib.h>
#include <stdio.h>
#include <math.h>
#include <string.h>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>

These libraries are the standard C and C++ libs.

#include <cv.h>
#include <highgui.h>

These libs are the ones from opencv C

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/tracking.hpp"

These libs are the ones from opencv C++,also caleed as opencv2

using namespace cv;
using namespace std;

Including the standard namespaces.

IplImage* frame, * img1;
CvPoint point;
int drag = 0;
int x_point,width_point,y_point,height_point;

Initialising the parameters to capture a video from webcam and to set the mouse.

int key = 0;
CvRect rect;
Rect region_of_interest;
int test;
Mat src,src_gray,image,src_gray_prev,src1,src_gray1,copy,copy1,frames,copy2;
int maxCorners = 23;
RNG rng(12345);
vector<Point2f> corners,corners_prev,corners_temp;
double qualityLevel = 0.01;
double minDistance = 10;
int blockSize = 3;
bool useHarrisDetector = false;
double k = 0.04;
vector<uchar> status;
vector<float> err;
float x_cord[100];
float y_cord[100];

There are the parameters we will use in “Good features to track” and “Optical flow”.

void mouseHandler(int event, int x, int y, int flags, void* param)
{
    if (event == CV_EVENT_LBUTTONDOWN && !drag)
    {
        point = cvPoint(x, y);
        drag = 1;
    }
    
    if (event == CV_EVENT_MOUSEMOVE && drag)
    {
        img1 = cvCloneImage(frame);
        cvRectangle(img1,point,cvPoint(x, y),CV_RGB(255, 0, 0),1,8,0);
        cvShowImage("result", img1);
    }
    
    if (event == CV_EVENT_LBUTTONUP && drag)
    {
        rect = cvRect(point.x,point.y,x-point.x,y-point.y);
		x_point = point.x;
		y_point = point.y;
		width_point = x-point.x;
		height_point = y-point.y;
        cvShowImage("result", frame);
        drag = 0;
    }

    
    if (event == CV_EVENT_RBUTTONUP)
    {
        drag = 0;
    }
}

This is the code to draw and select a region of interest from the video window.

int main(int argc, char *argv[])
{
    capture = cvCaptureFromCAM( 0 ); 
    if ( !capture ) {
        printf("Cannot open initialize webcam!\n" );
        exit(0);
    }
    cvNamedWindow( "result", CV_WINDOW_AUTOSIZE );
	int small,big; //declares integer
    
	int x = 1;

The above snippet captures the first frame.

while( key != 'q' )
    {
        frame = cvQueryFrame( capture );

These line make sure that video is available till you press the key “q”

if (rect.width>0)

To check if the rectangle has been chosen or not

if(corners.size() == 0 || x==0)
			{
				Mat frames(frame);
				src = frames.clone();
				cvtColor( src, src_gray, CV_BGR2GRAY );
				cv::Mat mask1 = cv::Mat::zeros(src.size(), CV_8UC1);  
				cv::Mat roi(mask1, cv::Rect(x_point,y_point,width_point,height_point));
				roi = cv::Scalar(255, 255, 255);
				copy1 = src.clone();		
				goodFeaturesToTrack( src_gray,
    		           corners,
    		           maxCorners,
    		           qualityLevel,
    		           minDistance,
    		           mask1,
    		           blockSize,
    		           useHarrisDetector,
    	           k );

				int rad = 3;
  				for( int i = 0; i < corners.size(); i++ )
  				   { circle( copy1, corners[i], rad, Scalar(rng.uniform(0,255), rng.uniform(0,255),
  			            rng.uniform(0,255)), -1, 8, 0 );
			
					}
				IplImage test1 = copy1;
			  	IplImage* test2 = &test1;
				x = 1;

			    cvShowImage("result", test2);
			}

If the rectangle has just been drawn in the previous frame,the above code finds good features and saves it.

else
				{
					src_gray_prev = src_gray.clone();
					corners_prev = corners;
					Mat framess(frame);
					src = framess.clone();
					cvtColor( src, src_gray, CV_BGR2GRAY ); 
					cv::Mat mask = cv::Mat::zeros(src.size(), CV_8UC1);  
					cv::Mat roi(mask, cv::Rect(x_point,y_point,width_point,height_point));
					roi = cv::Scalar(255, 255, 255);	
					Mat copy;
  					copy = src.clone();
					goodFeaturesToTrack( src_gray,
    		           corners,
    		           maxCorners,
    		           qualityLevel,
    		           minDistance,
    		           mask,
    		           blockSize,
    		           useHarrisDetector,
    		           k );
	
					calcOpticalFlowPyrLK(src_gray_prev, src_gray, corners_prev, corners, status, err);
			
  					int r = 3;
  					for( int i = 0; i < corners.size(); i++ )
    				 { circle( copy, corners[i], r, Scalar(rng.uniform(0,255), rng.uniform(0,255),
    			          rng.uniform(0,255)), -1, 8, 0 );
					 x_cord[i] = corners[i].x;
					 y_cord[i] = corners[i].y;
					 
					}			
					IplImage test3 = copy;
					IplImage* test4 = &test3;
					cvShowImage("result", test4);			
					
				}

Now once the features have been saved,it is tracked.

vSetMouseCallback("result", mouseHandler, NULL);
        key = cvWaitKey(10);
        if( (char) key== 'r' )
	{ 
		rect = cvRect(0,0,0,0); cvResetImageROI(frame);
		x = 0;
	}
        cvShowImage("result", frame);

Calling the mouse handler function and setting up the key to reset the region of interest.

This is major explanation gor the code.

Compilation and running:(for ubuntu users)
1)Save the and name it tracker.cpp
2)Open a terminal and traverse to that folder where you saved the code and type:
a)chmod +x tracker.cpp
b)g++ -ggdb `pkg-config –cflags opencv` -o `basename mouse2.cpp .cpp` mouse2.cpp `pkg-config –libs opencv`
c)./tracker

Now the video will open,select the box and play with the tracker.If you want to reset the tracker to draw a new window press the key “r”.

I hope you like the application. Will be back with a more revised and more robust application in a few days.

Thank you 🙂

OPENCV AND PCL(FILTERING A PCD FILE)

Hello friends,

After doing a lot of research in point cloud library,I came up with the successful integration of computer vision with point cloud library.

I hope you have opencv and pcl installed by now.If not see:
1)https://abhishek4273.wordpress.com/2014/02/05/install-opencv/
2)https://abhishek4273.wordpress.com/2014/05/26/install-point-cloud-library/

This post is the result of inspiration from http://ramsrigoutham.com/2012/06/28/integrating-pcl-and-opencv-passthrough-filter-example/

Since I was not successful in compiling and running the code from the blog mentioned above in ubuntu 12.04 I came up with another solution.

The code below filters the point cloud from a PCD file and visualizes it.

The code will work properly on every PCD file after you make sure the filter_limit parameters are set in a proper way.

Download the code from here : https://github.com/abhi-kumar/OPENCV_MISC/blob/master/opencv_pcl_filter.cpp

Here goes the explaination:

#include <pcl/point_cloud.h>
#include <pcl/io/pcd_io.h>
#include <pcl/io/ply_io.h>
#include <pcl/point_types.h>
#include <pcl/filters/passthrough.h>
#include <pcl/visualization/pcl_visualizer.h>
#include <vtkSmartPointer.h>

These are the required PCL libraries

int a = 22;
int b = 12;
int c=  10;

These are pre-set values for the trackbar

pcl::visualization::PCLVisualizer viewer ("Get the view here");

This is the window to display/visualize stuff

pcl::PointCloud<pcl::PointXYZ>::Ptr cloud(new pcl::PointCloud<pcl::PointXYZ>);
pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_filtered(new pcl::PointCloud<pcl::PointXYZ>);

Here we have declared two point clouds of type POINTXYZ,you can change the type according to your PCD file.

pcl::io::loadPCDFile (argv[1], *cloud);
pcl::copyPointCloud( *cloud,*cloud_filtered);

Now we have loaded the file into the cloud and then copied into the other cloud.

cv::namedWindow( "picture");
cvCreateTrackbar("X_limit", "picture", &a, 30, NULL);
cvCreateTrackbar("Y_limit", "picture", &b, 30, NULL);
cvCreateTrackbar("Z_limit", "picture", &c, 30, NULL);

This will look familiar to OpenCV users.For those who dont know this,it is a way of creating trackbars to control any parameter,here,filter limits.

pcl::PassThrough<pcl::PointXYZ> pass;

Creating an object of class PassThrough filter

while (!viewer.wasStopped ())
             {

		pcl::copyPointCloud(*cloud_filtered, *cloud);

        i = 0.1*((float)a);
        j = 0.1*((float)b);
        k = 0.1*((float)c);

//        cout << "i = " << i << " j = " << j << " k = " << k << endl;

        pass.setInputCloud (cloud);
	pass.setFilterFieldName ("y");
        pass.setFilterLimits (-j, j);
        pass.filter (*cloud);

        pass.setInputCloud (cloud);
        pass.setFilterFieldName ("x");
        pass.setFilterLimits (-i, i);
        pass.filter (*cloud);

	pass.setInputCloud (cloud);
        pass.setFilterFieldName ("z");
        pass.setFilterLimits (-k,k);
        pass.filter (*cloud);

	viewer.addPointCloud (cloud, "scene_cloud");
	viewer.spinOnce ();
	viewer.removePointCloud("scene_cloud");
	waitKey(10);
	}

Iterating though the cloud,setting filter parameters and linking it to the trackbars.

Now for the compiling part.
1)Download this file and save it as pcl.sh in the same folder where you have saved opencv_pcl_filter.cpp : https://github.com/abhi-kumar/OPENCV_MISC/blob/master/pcl.sh

2)Download this sample PCD file and save as test.pcd in the same folder : https://github.com/abhi-kumar/OPENCV_MISC/blob/master/test.pcd

3) open a terminal and type
a)chmod +x pcl.sh
b)./pcl.sh opencv_pcl_filter.cpp
c)./opencv_pcl_filter test.pcd

And you will see something like this:
Screenshot from 2014-07-01 17:29:43

And thats it,we have successfully created a filter in PCL.

Thanks 🙂

SERIAL PORT AND VISUAL STUDIO

Hello everyone,

Getting a little off-track from our image processing,in my internship I had to integrate Serial Port with Visual Studio(C++).In that process I got stuck in retreiving data from the serial port as the event handler errors were coming up.After a very intensive research I tried to compile all my knowledge to implement it and “KA-BOOM”….. it worked 🙂

So I would like to share with you the entire process.

First open a windows c++ form application.

Then,in the form the very first thing you do is add a serial port from tool-box,in its properties set the BaudRate,Parity,StopBits,Number of bits to be transferred according to the hardware.

Now,select “button” from tool-box and drop it down on the form,set Text = Transfer and Name = btnTransfer.
Select another button from tool-box and drop it down on the form.set Text = Open and Name = btnOpen.
Select another button from tool-box and drop it down on the form.set Text = Close and Name = btnClose.

Select a comboBox from tool-box and drop it down on the form.Change its DropDown Style poperty from DropDown to DropDownList.

Select a label and place it in front of the comboBox and name it “Port”.

Select “TextBox” from tool-box and drop it down on the form,set Name = txtTransfer.

Double click anywhere on the form to go to the form1.h file.

Firstly in the namespace add:
using namespace System::IO::Ports;

Then after the following part of the code:
public ref class Form1 : public System::Windows::Forms::Form
{
public:
Form1(void)
{
InitializeComponent();

Add This: findPorts();

Just before the “#pragma region” add:
String^ Data;
String^ data1;

Now inside the function named void “InitializeComponent(void)” add:
this->serialPort1->DataReceived += gcnew System::IO::Ports::SerialDataReceivedEventHandler(this, &Form1::serialPort1_DataReceived_1);

After the “#pragma endregion” add:

private: void findPorts(void)
{
// get port names
array<Object^>^ objectArray = SerialPort::GetPortNames();
// add string array to combobox
this->comboBox1->Items->AddRange( objectArray );
}

Come to the form(GUI) and select the serial port,in the properties window,select the bar which is like lightning flashing and click on the DataReceieved and write its value as “serialPort1_DataReceived_1”.

Now in the code add this:

private: System::Void SetTextCallback(System::Object^ sender, System::EventArgs^ e)
{
this->txtTransfer->AppendText(data1);
Data = this->txtTransfer->Text;
}

private: System::Void serialPort1_DataReceived_1(System::Object^ sender, System::IO::Ports::SerialDataReceivedEventArgs^ e) {
data1 = serialPort1->ReadExisting();
this->Invoke(gcnew EventHandler(this, &Form1::SetTextCallback));
}

Now go to the form and double click on “Open” button and paste the code inside the function created:

if(this->comboBox1->Text==String::Empty)
this->txtTransfer->Text=”Please Select Port Settings”;
else {
try{
// make sure port isn’t open
if(!this->serialPort1->IsOpen){
this->serialPort1->PortName=this->comboBox1->Text;

//open serial port
this->serialPort1->Open();
}
else
this->txtTransfer->Text=”Port isn’t openned”;
}
catch(UnauthorizedAccessException^){
this->txtTransfer->Text=”UnauthorizedAccess”;
}
}

Now go to the form and double click on “Transfer” button and paste the code inside the function created:

this->serialPort1->Write(“THE INVOKING STRING GOES HERE”);

Now go to the form and double click on “Close” button and paste the code inside the function created:

this->serialPort1->Close();

Compile and run it.
And as soon as the string reaches the port you will receive the data immediately in the textBox we created.

Thank you 🙂

OBJECT DETECTION USING GENETIC PROGRAMMING (PRE-TEST-1)

Hello Friends,

This is the partial test implementation of http://blindperception.wordpress.com/2014/05/18/object-detection-in-multi-modal-images-using-genetic-programming/

Please visit the above link to understand what is given below.

Today I would like to share a part with you the current project am doing.In this post I will just be showing how a series of operators can be able to detect a region/object of interest from a given image.

Later I will be introducing genetic programming into it,but for now lets just see the basic(core) part of the project. We will stick to testing on satellite images.

My result:-

A T72 tank image from a satellite:
test2

Its ground truth detection:
gt2

My detection:
test

The series of operators used were :-
(MED (SQRT (MULC (MULC (SUBC (MULC(SQRT (SUBC (SQRT PFIM2)))))))))

Download the testing code from:-
https://github.com/abhi-kumar/OBJECT-DETECTION-USING-GENETIC-PROFRAMMING/blob/master/test1.cpp

The entire implementation will be up in a few days.

Thank you 🙂

Install Point Cloud Library(Bugs Resolved)

Hello friends,

In our last post we had a brief description of of what the Point Cloud Library is,so now lets install it and get started with PCL.

Here I will only be telling the installation procedure for Ubuntu(Linux),

SO here it goes: –

Open the terminal and type

a) sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl

b) sudo apt-get update

c) sudo apt-get install libpcl-all

And its done 🙂

Now open a new terminal and type :
sudo gedit /usr/lib/pkgconfig/flann.pc

and in that file remove “-lflann_cpp-gd”. Save and close the file.

Now lets test it.

Method 1:-

a) Make a folder,and in the folder download the code and place it in a file named pcl_write.cpp

b) Download code from here :- http://pointclouds.org/documentation/tutorials/writing_pcd.php#writing-pcd

c) Make a new folder named build inside the current folder.

d) Make a new file named CMakeLists.txt and copy the following contents in it :

Contents:-

cmake_minimum_required(VERSION 2.6 FATAL_ERROR)
project(MY_GRAND_PROJECT)
find_package(PCL 1.3 REQUIRED COMPONENTS common io)
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})
add_executable(pcd_write_test pcd_write.cpp)
target_link_libraries(pcd_write_test ${PCL_COMMON_LIBRARIES} ${PCL_IO_LIBRARIES})

e) Enter the folder build by typing “cd build” in the terminal.

f) now in the terminal type these two commands:
cmake ..
sudo make

And a executable file will be generated.

Method 2 ( One of my seniors Jay Rambhia helped me get this,buts its slightly modified):-

Make a file named pcl.sh in the folder where ur code file is there and copy these contents in it : https://github.com/abhi-kumar/OPENCV_MISC/blob/master/pcl.sh

Open a terminal and navigate to the folder where the code and pcl.sh is present and type:
a) chmod +X pcl.sh
b) ./pcl.sh YOUR_CODE_NAME.CPP

Second method is simple isn’t it??

So its done 🙂

Thank you 🙂

Robotics And Computer Vision