Category Archives: OPENCV

A Stop at Genetic Programming Basics

AUTHORS : Abhishek Kumar Annamraju, Adhesh Shrivastava, Akashdeep Singh

Hello Everyone,

Over the years Computer vision lovers have have developed a genuine interest in the field of genetic programming.But why has this field been chosen????

Lets have a close look at what actually is this genetic programming???

In our real world,only the fortunate or the fitter ones survive.Obviously we cannot use the fortune element in computer vision,so fitness was taken into account.In real world,population never remains steady and the children do not replace the parents.On the other hand,in computer world of genetic programming,any population is fixed and the child replaces the least fit parent or any other individual in the population.

gaimg1

The diagram above shows the very basic genetic programming cycle.Each and every individual from the population is selected for fitness evaluation and then selected for mating as per their fitness levels.Now,their child(the descriptor) enters the population and the procedure continues till you get the fittest descriptor according to the user’s need.

For example there are primitive functions present int the population like adding pixels of two images,finding means,medians etc.Now if I set my goal as finding the chain of these primitives which selects my region of interest.For this I have a training image and a ground truth to evaluate the fitness.I put everything in the cycle and start evaluation and finally select the best possible one.This is what the algorithm does for us.

Most popularly used genetic programming is the multiple tree based algorithm since it is accurate and covers most of the population.Others are linear genetic programming and graphical genetic programming.

The linear genetic programming if further divided into three parts,stack based,register based and machine code.
Just to brief you up in stack based GP,each program instruction takes its primitive descriptor from a stack,performs the task and puts it back into the stack.On the other hand the register based and machine code one are similar,here,instead of stack permanent storage registers are used.I will not go much deep into it.

Lets stick to tree based one.To move further you should have a basic knowledge of fer terminologies:-

1)Fitness measure: Now with genetic programming a huge amount of composite feature operators will be generated. So, to give them a rank this fitness measure is introduced. Let G be the foreground in the ground truth image and G’ be the foreground in the composite feature image formed from the primitive feature image after being passed over by the composite feature operator. Let n(X) be the number of pixels within a region X. Therefore, the fitness measure of that composite operator will be n(G∩G’) / n(G U G’).

2) Selection: It is the process of selecting a composite operator from current population of composite operators generated. The number of individuals are randomly selected and the one with highest fitness value is copied to new population.

3) Code bloat problem: This is the basic problem in generation using genetic algorithm. In this the sizes of composite parameters by crossover may become very large. This slows down the process and consumes a lot of memory.

4) Crossover: It is a process similar to gene crossovers. First we select two composite operators on the basis of good fitness measures and term them as parents. Then one inter-node in each parent is selected randomly and the two subtrees rooted at that node are exchanged to generate two new composite operators called offspring. Here many times code bloat problem occurs, so, as a remedy they have come up with limiting the operator size with a soft size limit. If size of offspring > limit, then genetic programming keeps it, but evaluates it further. If the evaluation results in best fitness then it is kept otherwise discarded.

5) Mutation: This is done to avoid premature convergence which means that the resultant population halted too early with less generations or small generations. It is done in any of three ways:
a)Randomly select a node of a composite operator, and replace it(along with the selected node) with a subtree at this node by another operator(tree).
b)Randomly select a node of a composite operator, replace the primitive operator stored in the node with any other primitive operator with same input capacity.
c)Select any two subtrees from two operators and swap them.

6) Crossover probability/rate: It defines ratio of how many couple composite operators will be taken with respect to the number of couple composite operators present during a devised loop. A couple composite operator is any two operators taken at random.

7) Mutation probability/rate: It sets a maximum limit rate to which a how many mutation method can be applied to single composite operator. If probability is 5% and number of nodes in it 100,then only 5 times mutation will be done.

I think this much is enough for the introduction.I will get more deeper into the genetic programming in my next posts.

Thank you 🙂

Also see :- http://blindperception.wordpress.com/

Advertisements

OBJECT DETECTION USING MULTIPLE TRAINCASCADED XML FILES

AUTHORS:Abhishek Kumar Annamraju,Akashdeep Singh,Adhesh Shrivastava

Hello Friends

In our previous post we learnt how to generate a cascade.xml file using opencv_traincascade.We usually tend to use it with facedetect.cpp present in OpenCV folder.

Now what if we want to use multiple xml files over the same image and display a combined result????

Well,after a little struggle we came up with an optimum code that does the above mentioned task in a very fast way.The code has only been implemented to images,will try to develop it further for videos too.

Download the code from here:https://github.com/abhi-kumar/OPENCV_MISC/blob/master/objectDetectionMultipleCascadeInput.cpp

The code is well commented,so you will understand everything once you carefully go through it.

Some of the results of the code:

I used it to detect cars,database was chosen as UIUC car database.

The cascade files used can be downloaded from here:https://github.com/abhi-kumar/CAR-DETECTION.git

These are sum of the results:
*******************************************************************************
Detection time:38.2002 ms

INPUT1

OUTPUT1

**********************************************************************************

Detection time:38.4898 ms

INPUT2

OUTPUT2

**********************************************************************************

You are always welcome to download the code and modify it for better usages.Any contribution to the code is highly appreciated.

Thank you 🙂

See also :- http://blindperception.wordpress.com/

TRAINCASCADE AND CAR DETECTION USING OPENCV

AUTHORS:Abhishek Kumar Annamraju,Akashdeep Singh,Adhesh Shrivastava

Hello Friends

My last post explained how segmentation can be used to detect roads.
This post will explain the following things:
1.Optimum use of traincascade
2.Creating xml files for object detection
3.Using multiple xml files to detect object,here it is cars
4.Using multiple xml files without detecting a single object twice.

Haartraining is stated to provide better results than traincascade but it is extremely slow.Sometimes it may take one to two weeks to train a classifier.So we shifted our goal to traincascade.

The whole procedure from now will focus on car detection

STEP-1:
Download the image database from here : http://cogcomp.cs.illinois.edu/Data/Car/

STEP-2:
Inside a folder:
1.Copy all the positive images in a folder named pos.
2.Copy all the negative images in a folder named neg.
3.Create a folder named data to store cascade file generated later on.

STEP-3:
Open a terminal,navigate to the requires folder and type

1.find pos -iname “*.pgm” -exec echo \{\} 1 0 0 100 40 \; > cars.info

Result will be file like this :https://github.com/abhi-kumar/CAR-DETECTION/blob/master/cars.info

2.find neg -iname “*.pgm” > bg.txt
Result will be file like this :https://github.com/abhi-kumar/CAR-DETECTION/blob/master/bg.txt

3.opencv_createsamples -info cars.info -num 550 -w 48 -h 24 -vec cars.vec
(width and height parameters change with change of database,-num is the number of images in pos folder)

4.opencv_traincascade -data data -vec cars.vec -bg bg.txt -numStages 10 -nsplits 2 -minhitrate 0.999 -maxfalsealarm 0.5 -numPos 500 -numNeg 500 -w 48 -h 24
a)-numPos and -numNeg must have the number of photos you have in pos and neg folder respectively.
b)numPos < number of samples in vec
c)choosing minhitrate and maxfalsealarm

To understand how the parameters of training affect the output refer : http://scholarpublishing.org/index.php/AIVP/article/view/1152

For example you have 1000 positive samples. You want your system to detect 900 of them. So desired hitrate = 900/1000 = 0.9. Commonly, put minhitrate = 0.999^number of stages

For example you have 1000 negative samples. Because it’s negative, you don’t want your system to detect them. But your system, because it has error, will detect some of them. Let error be about 490 samples, so false alarm = 490/1000 = 0.49. Commonly,put false alarm = 0.5^number of stages

Note:

1)The number of negative images must be greater than the number of positive images.

2)Try to set npos = 0.9 * number_of_positive_samples and 0.99 as a minHitRate.

3)vec-file has to contain >= (npos + (numStages-1) * (1 – minHitRate) * numPose) + S, where S is a count of samples from vec-file.S is a count of samples from vec-file that can be recognized as background right away.

This will generates xml file in data folder.The face detction code in the opencv examples could be used but with change in the xml file name.

A better way of using the xml file is running it through the code I made for object detection using multiple xml file
Refer:https://abhishek4273.wordpress.com/2014/03/16/object-detection-using-multiple-traincascaded-xml-files/

Code and other xml files(do read README.nd):https://github.com/abhi-kumar/CAR-DETECTION.git

The cascade files used can be downloaded from here:https://github.com/abhi-kumar/CAR-DETECTION.git

These are sum of the results:
*******************************************************************************
Detection time:38.2002 ms

INPUT1

OUTPUT1

**********************************************************************************

Detection time:38.4898 ms

INPUT2

OUTPUT2

**********************************************************************************

You are always welcome to download the code and modify it for better usages.Any contribution to the code is highly appreciated.

Thank you 🙂

Science lovers must see this : http://tanvik3394.wordpress.com/

See also :- http://blindperception.wordpress.com/

FAST ROAD/PATH DETECTION USING OPENCV

AUTHORS:Abhishek Kumar Annamraju,Akashdeep Singh,Adhesh Shrivastava

Hello Friends

Today I would like to discuss with you how we are trying to implement road detection for our robotics project of developing a hexapod.

Map building,object detection,obstacle detection etc,all emerge out clearly when we successfully implement background detection.Robotics enthusiasts like to call this background detection as “basic path detection”.

OpenCV has always been our favourite tool for image processing,so again we are using the same for road detection.

Inspired by the great working of watershed algorithm for image segmentation(RESEARCH PAPER:http://www.ias-iss.org/ojs/IAS/article/viewFile/852/755),we wanted to implement it for detection of path.The main aim of this code is to segment out road as FAST as possible,but we also need to take care of the accuracy.

The code goes here.
You can download the code from https://github.com/abhi-kumar/Hexapod/blob/master/road_detection.cpp

Click once somewhere on the code and press ctrl+A to select whole code.You may not see the whole code so its better to copy the code and paste it in your favourite text editor and then go through it.

#include <opencv2/opencv.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/core/core.hpp>
#include <iostream>
#include <stdio.h>

using namespace std;
using namespace cv;

class findroad{                              //class that separates out roads from images
private:
Mat path;
public:
void setpath(Mat& image)
{
image.convertTo(path, CV_32S);
}

Mat getroad(Mat &image)                  //major working function attribute of the class
{
watershed(image, path);                                     //using watershed segmenter
path.convertTo(path,CV_8U);
return path;
}
};

int main( int argc, const char** argv )
{
Mat image1 = imread(argv[1],1);
Mat image;
resize(image1,image,Size(500,500),0,0,INTER_LINEAR);
Mat gray;
cvtColor(image, gray, CV_BGR2GRAY);
threshold(gray, gray, 100, 255, THRESH_BINARY);          //threasholding the grayscale image
double t = 0;
t = (double)cvGetTickCount();                                //setting up timer
Mat Erode;
erode(gray,Erode,Mat(),Point(2,2),7); //eroding image 7 times.
Mat Dilate;
dilate(gray,Dilate,Mat(),Point(2,2),7);    //dilating the image
threshold(Dilate,Dilate,1, 50,THRESH_BINARY_INV);
Mat path_trace(gray.size(),CV_8U,Scalar(0));
path_trace= Erode+Dilate;                                      //morphological addition
findroad road;                                             //creating an object for our class
road.setpath(path_trace);                                      //preparing the input
namedWindow("founded road");
namedWindow("input image");
Mat road_found = road.getroad(image);
road_found.convertTo(road_found,CV_8U);
imshow("founded road", road_found);
imshow("input image",image);
t = (double)cvGetTickCount() - t;                              //time of detection
printf( "road got detected in = %g ms\n", t/((double)cvGetTickFrequency()*1000.) );
cout << endl << "cheers" << endl;
imwrite("ROAD1.jpg",image);
imwrite( "ROAD1_DETECTED.jpg",road_found);
waitKey(0);
return 0;
}

********************************************************************************

We are still developing the code to get better and better results.
The folowing were our best outcomes

This result was obtained in 109.48 ms.

ROAD1

ROAD1_DETECTED

***************************************************************************************
This result was obtained in 114.475 ms

ROAD1

ROAD1_DETECTED
****************************************************************************************

This result was obtained in 98.91 ms

ROAD3

ROAD3_DETECTED

****************************************************************************************

This result was obtained in 110.336 ms

ROAD4

ROAD4_DETECTED

***************************************************************************************

But still as the image gets more complex,more crowded with objects the result fades away,for example the two following road detections:

******************************************************************************
This result was obtained in 107.042 ms

ROAD5

ROAD5_DETECTED

**********************************************************************************

This result was obtained in 114.656 ms

ROAD6

ROAD6_DETECTED

***********************************************************************************

Well with vehicles and other objects the code becomes more vulnerable.We are trying our level best to come up with an optimim solution.

Note:I have made the code open source so that everyone could benefit from it,also it can be modified by others to function in a better way.You can download the code from here https://github.com/abhi-kumar/Hexapod/blob/master/road_detection.cpp

Any modified code could be mailed to this address:
abhishek4273@gmail.com

Your contribution is highly appriciated.

Thank You 🙂

See also :- http://blindperception.wordpress.com/

USING C++ CLASSES IN OPENCV

Authors:Abhishek Kumar Annamraju,AkashDeep Singh,Adhesh Shrivastava

Hi there,

We have been working till now with small basic opencv programs,so continuing with them I would like to add some tinge of classes of c++ to it.As you are well aware of the fact that class helps us to give a systematic path for presenting the code.

In the following code

1)I have defined two classes one to store an image in it and another class which gets that image from first class for further manipulation.

2)Also I have fetched some image properties in main,passed them to class store and used them in class grayimage.

So,any changes in main wont directly affect the images or data in classes.To make changes in them you need to call them(classes and its objects) personally.

Again the code is well commented for your understanding

Click once somewhere on the code and press ctrl+A to select whole code.You may not see the whole code so its better to copy the code and paste it in your favourite text editor and then go through it.

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <cv.h>
#include <highgui.h>
#include <iostream>
#include<stdio.h>
using namespace cv;
using namespace std;

class store              
{
    public:             //Keep it public so that it can be used in other classes
        Mat image;
        int blue;
        int green;
        int red;



    public: 
    void getimage(Mat src) //storing image into a class variable named image
    {               

        if(! src.data )                              
        {
            cout <<  "src not filled" << endl ;
        }

        else
        {
            image = src.clone();
            cout << "got image" <<endl;
        }
    }

    void getdata(int bb,int gg,int rr)//storing data into class variables
    {
        blue =bb;
        green= gg;
        red=rr;
    }

    void showimage()//Display stored image using a class function
    {       
        if(! image.data )                              
        {
            cout <<  "Image not passed into this function" << std::endl ;

        }

        else
        {

            namedWindow( "Display window", WINDOW_AUTOSIZE );
            imshow( "Display window", image );
            cout << "image ready to get displayed" << endl;
        }   
    }

};  //first class ends here;

class grayimage   
//creating a class where i need to manipulate images and data obtained from another class
{

    private:
        Mat gray_image;
        int btrans,gtrans,rtrans;

    public:
        void setgray(Mat src) //converting the image to grayscale
        {
            if(! src.data )                              
            {
                cout <<  "src in second class not filled" << endl ;
            }

            else
            {
                cvtColor(src, gray_image, CV_BGR2GRAY );
                cout << "image conversion done" <<endl;
            }

        }

        void transferdata(int b,int g,int r)  //creating a transfer function
        {
            btrans = b;
            gtrans = g;
            rtrans = r;
        }       


        void showdata()
        {
            cout << "values of b,g,r before gray scale coversion are" << btrans << "," << gtrans << "," << rtrans << "," <<endl; 
        }

        void displaygray()
        {
            if(! gray_image.data )                              
            {
                cout <<  "Image not passed into this function" << std::endl ;

            }

            else
            {

                namedWindow( "gray image", WINDOW_AUTOSIZE );
                imshow( "gray image", gray_image );
                cout << "gray image ready to get displayed" << endl;
            }   
        }

};  

//main starts here

int main( int argc, char** argv )
{
    if( argc != 2)
    {
     cout <<" Usage: display_image ImageToLoadAndDisplay" << endl;
     return -1;
    }

    Mat img;
    img = imread(argv[1], CV_LOAD_IMAGE_COLOR);    

    if(! img.data )                              
    {
        cout <<  "Could not open or find the image" << std::endl ;
        return -1;
    }

    unsigned char *input = (unsigned char*)(img.data);
    int i,j,r,g,b;
    for(int i = 0;i < img.rows;i++)
    {
        for(int j = 0;j < img.cols;j++)
        {
                b = input[img.step * j + i ] ;
                g = input[img.step * j + i + 1];
                r = input[img.step * j + i + 2];
            }
    }

    store image1; //object to store image created
    image1.getimage(img);//image from main stored in object
    image1.getdata(b,g,r);//data from main stored in object
    image1.showimage();//displaying the stored image 

    grayimage image2;   //creating a manipulator object
    image2.setgray(image1.image); 
//fetches image from the first object that was stored and converts it into grayscale 
    image2.transferdata(image1.blue,image1.green,image1.red);
//fetches data from the first object that was stored 

    //This data transferred to the new object can be manupulated anytime later

    image2.showdata();
    image2.displaygray();          

    waitKey(0);                                         
    return 0;
}

This will help you to use classes in complex opencv c++ codes

Thank you 🙂

See also :- http://blindperception.wordpress.com/

CAMERA CALIBRATION USING OPENCV

AUTHORS:Abhishek Kumar Annamraju,Akash Deep Singh,Adhesh Shrivastava

Hello Friends

Camera Calibration as the name suggests, in layman’s language, is making your camera ready to deal with the conversion of pixel coordinates in an image to the real world coordinates. For any Image processing technique to work properly, scientists usually prefer a calibrated camera.

A very first question arises that what actually does calibrating a camera provides us?

To the answer it gives us the intrinsic parameters of the camera or the imaging device on a broader picture.

Lets see what are intrinsic parameters of a camera

Intrinsic parameters include the model of camera’s geometry and the distortion model of the lens.The camera’s geometry is described by four parameters fx,fy,cx,cy,where the first two parameters are focal lengths(in pixels) and the last two parameters adjust the displacement from optical axis.The distortion matrix includes {k1,k2,p1,p2,k3}.

x_{corrected} = x + [ 2p_1xy + p_2(r^2+2x^2)] \\<br /><br /> y_{corrected} = y + [ p_1(r^2+ 2y^2)+ 2p_2xy]

Also with camera calibration extrinsic parameters can be found out.These are the rotation and translation matrices.

The calibration process goes as follows:

Q = Transpose matrix of the matrix [X  Y  Z  1]  , where X,Y,Z are real world coordinates of a point object

q = Transpose matrix of the matrix [x  y  1]  , where x,y are pixel coordinates on the image plane of a point object present in the real world

M(A 3×3 Camera_matrix) =

[fx      0      cx
0        fy     cy
0        0        1 ]

W(contains the rotation and translation parameters) = [r1       r2          t]

s = scaling factor determined within the process

so,finally the relation comes out to be

q = sMWQ and all these Matrices are invertible so the process has a reciprocal.

Lets get into the process of camera calibration using opencv:

First copy the following code and save it in a file named calibration.cpp

Click once somewhere on the code and press ctrl+A to select whole code.You may not see the whole code so its better to copy the code and paste it in your favourite text editor and then go through it.

******************CODE STARTS HERE****************************

#include &lt;opencv2/core/core.hpp&gt;
#include &lt;opencv2/imgproc/imgproc.hpp&gt;
#include &lt;opencv2/calib3d/calib3d.hpp&gt;
#include &lt;opencv2/highgui/highgui.hpp&gt;

#include &lt;cctype&gt;
#include &lt;stdio.h&gt;
#include &lt;string.h&gt;
#include &lt;time.h&gt;

using namespace cv;
using namespace std;

const char * usage =
" \nexample command line for calibration from a live feed.\n"
"   calibration  -w 4 -h 5 -s 0.025 -o camera.yml -op -oe\n"
" \n"
" example command line for calibration from a list of stored images:\n"
"   imagelist_creator image_list.xml *.png\n"
"   calibration -w 4 -h 5 -s 0.025 -o camera.yml -op -oe image_list.xml\n"
" where image_list.xml is the standard OpenCV XML/YAML\n"
" use imagelist_creator to create the xml or yaml list\n"
" file consisting of the list of strings, e.g.:\n"
" \n"
"&lt;?xml version=\"1.0\"?&gt;\n"
"&lt;opencv_storage&gt;\n"
"&lt;images&gt;\n"
"view000.png\n"
"view001.png\n"
"&lt;!-- view002.png --&gt;\n"
"view003.png\n"
"view010.png\n"
"one_extra_view.jpg\n"
"&lt;/images&gt;\n"
"&lt;/opencv_storage&gt;\n";

const char* liveCaptureHelp =
"When the live video from camera is used as input, the following hot-keys may be used:\n"
"  &lt;ESC&gt;, 'q' - quit the program\n"
"  'g' - start capturing images\n"
"  'u' - switch undistortion on/off\n";

static void help()
{
printf( "This is a camera calibration sample.\n"
"Usage: calibration\n"
"     -w &lt;board_width&gt;         # the number of inner corners per one of board dimension\n"
"     -h &lt;board_height&gt;        # the number of inner corners per another board dimension\n"
"     [-pt &lt;pattern&gt;]          # the type of pattern: chessboard or circles' grid\n"
"     [-n &lt;number_of_frames&gt;]  # the number of frames to use for calibration\n"
"                              # (if not specified, it will be set to the number\n"
"                              #  of board views actually available)\n"
"     [-d &lt;delay&gt;]             # a minimum delay in ms between subsequent attempts to capture a next view\n"
"                              # (used only for video capturing)\n"
"     [-s &lt;squareSize&gt;]       # square size in some user-defined units (1 by default)\n"
"     [-o &lt;out_camera_params&gt;] # the output filename for intrinsic [and extrinsic] parameters\n"
"     [-op]                    # write detected feature points\n"
"     [-oe]                    # write extrinsic parameters\n"
"     [-zt]                    # assume zero tangential distortion\n"
"     [-a &lt;aspectRatio&gt;]      # fix aspect ratio (fx/fy)\n"
"     [-p]                     # fix the principal point at the center\n"
"     [-v]                     # flip the captured images around the horizontal axis\n"
"     [-V]                     # use a video file, and not an image list, uses\n"
"                              # [input_data] string for the video file name\n"
"     [-su]                    # show undistorted images after calibration\n"
"     [input_data]             # input data, one of the following:\n"
"                              #  - text file with a list of the images of the board\n"
"                              #    the text file can be generated with imagelist_creator\n"
"                              #  - name of video file with a video of the board\n"
"                              # if input_data not specified, a live view from the camera is used\n"
"\n" );
printf("\n%s",usage);
printf( "\n%s", liveCaptureHelp );
}

enum { DETECTION = 0, CAPTURING = 1, CALIBRATED = 2 };
enum Pattern { CHESSBOARD, CIRCLES_GRID, ASYMMETRIC_CIRCLES_GRID };

static double computeReprojectionErrors(
const vector&lt;vector&lt;Point3f&gt; &gt;&amp; objectPoints,
const vector&lt;vector&lt;Point2f&gt; &gt;&amp; imagePoints,
const vector&lt;Mat&gt;&amp; rvecs, const vector&lt;Mat&gt;&amp; tvecs,
const Mat&amp; cameraMatrix, const Mat&amp; distCoeffs,
vector&lt;float&gt;&amp; perViewErrors )
{
vector&lt;Point2f&gt; imagePoints2;
int i, totalPoints = 0;
double totalErr = 0, err;
perViewErrors.resize(objectPoints.size());

for( i = 0; i &lt; (int)objectPoints.size(); i++ )
{
projectPoints(Mat(objectPoints[i]), rvecs[i], tvecs[i],
cameraMatrix, distCoeffs, imagePoints2);
err = norm(Mat(imagePoints[i]), Mat(imagePoints2), CV_L2);
int n = (int)objectPoints[i].size();
perViewErrors[i] = (float)std::sqrt(err*err/n);
totalErr += err*err;
totalPoints += n;
}

return std::sqrt(totalErr/totalPoints);
}

static void calcChessboardCorners(Size boardSize, float squareSize, vector&lt;Point3f&gt;&amp; corners, Pattern patternType = CHESSBOARD)
{
corners.resize(0);

switch(patternType)
{
case CHESSBOARD:
case CIRCLES_GRID:
for( int i = 0; i &lt; boardSize.height; i++ )
for( int j = 0; j &lt; boardSize.width; j++ )
corners.push_back(Point3f(float(j*squareSize),
float(i*squareSize), 0));
break;

case ASYMMETRIC_CIRCLES_GRID:
for( int i = 0; i &lt; boardSize.height; i++ )
for( int j = 0; j &lt; boardSize.width; j++ )
corners.push_back(Point3f(float((2*j + i % 2)*squareSize),
float(i*squareSize), 0));
break;

default:
CV_Error(CV_StsBadArg, "Unknown pattern type\n");
}
}

static bool runCalibration( vector&lt;vector&lt;Point2f&gt; &gt; imagePoints,
Size imageSize, Size boardSize, Pattern patternType,
float squareSize, float aspectRatio,
int flags, Mat&amp; cameraMatrix, Mat&amp; distCoeffs,
vector&lt;Mat&gt;&amp; rvecs, vector&lt;Mat&gt;&amp; tvecs,
vector&lt;float&gt;&amp; reprojErrs,
double&amp; totalAvgErr)
{
cameraMatrix = Mat::eye(3, 3, CV_64F);
if( flags &amp; CV_CALIB_FIX_ASPECT_RATIO )
cameraMatrix.at&lt;double&gt;(0,0) = aspectRatio;

distCoeffs = Mat::zeros(8, 1, CV_64F);

vector&lt;vector&lt;Point3f&gt; &gt; objectPoints(1);
calcChessboardCorners(boardSize, squareSize, objectPoints[0], patternType);

objectPoints.resize(imagePoints.size(),objectPoints[0]);

double rms = calibrateCamera(objectPoints, imagePoints, imageSize, cameraMatrix,
distCoeffs, rvecs, tvecs, flags|CV_CALIB_FIX_K4|CV_CALIB_FIX_K5);
///*|CV_CALIB_FIX_K3*/|CV_CALIB_FIX_K4|CV_CALIB_FIX_K5);
printf("RMS error reported by calibrateCamera: %g\n", rms);

bool ok = checkRange(cameraMatrix) &amp;&amp; checkRange(distCoeffs);

totalAvgErr = computeReprojectionErrors(objectPoints, imagePoints,
rvecs, tvecs, cameraMatrix, distCoeffs, reprojErrs);

return ok;
}

static void saveCameraParams( const string&amp; filename,
Size imageSize, Size boardSize,
float squareSize, float aspectRatio, int flags,
const Mat&amp; cameraMatrix, const Mat&amp; distCoeffs,
const vector&lt;Mat&gt;&amp; rvecs, const vector&lt;Mat&gt;&amp; tvecs,
const vector&lt;float&gt;&amp; reprojErrs,
const vector&lt;vector&lt;Point2f&gt; &gt;&amp; imagePoints,
double totalAvgErr )
{
FileStorage fs( filename, FileStorage::WRITE );

time_t tt;
time( &amp;tt );
struct tm *t2 = localtime( &amp;tt );
char buf[1024];
strftime( buf, sizeof(buf)-1, "%c", t2 );

fs &lt;&lt; "calibration_time" &lt;&lt; buf;

if( !rvecs.empty() || !reprojErrs.empty() )
fs &lt;&lt; "nframes" &lt;&lt; (int)std::max(rvecs.size(), reprojErrs.size());
fs &lt;&lt; "image_width" &lt;&lt; imageSize.width;
fs &lt;&lt; "image_height" &lt;&lt; imageSize.height;
fs &lt;&lt; "board_width" &lt;&lt; boardSize.width;
fs &lt;&lt; "board_height" &lt;&lt; boardSize.height;
fs &lt;&lt; "square_size" &lt;&lt; squareSize;

if( flags &amp; CV_CALIB_FIX_ASPECT_RATIO )
fs &lt;&lt; "aspectRatio" &lt;&lt; aspectRatio;

if( flags != 0 )
{
sprintf( buf, "flags: %s%s%s%s",
flags &amp; CV_CALIB_USE_INTRINSIC_GUESS ? "+use_intrinsic_guess" : "",
flags &amp; CV_CALIB_FIX_ASPECT_RATIO ? "+fix_aspectRatio" : "",
flags &amp; CV_CALIB_FIX_PRINCIPAL_POINT ? "+fix_principal_point" : "",
flags &amp; CV_CALIB_ZERO_TANGENT_DIST ? "+zero_tangent_dist" : "" );
cvWriteComment( *fs, buf, 0 );
}

fs &lt;&lt; "flags" &lt;&lt; flags;

fs &lt;&lt; "camera_matrix" &lt;&lt; cameraMatrix;
fs &lt;&lt; "distortion_coefficients" &lt;&lt; distCoeffs;

fs &lt;&lt; "avg_reprojection_error" &lt;&lt; totalAvgErr;
if( !reprojErrs.empty() )
fs &lt;&lt; "per_view_reprojection_errors" &lt;&lt; Mat(reprojErrs);

if( !rvecs.empty() &amp;&amp; !tvecs.empty() )
{
CV_Assert(rvecs[0].type() == tvecs[0].type());
Mat bigmat((int)rvecs.size(), 6, rvecs[0].type());
for( int i = 0; i &lt; (int)rvecs.size(); i++ )
{
Mat r = bigmat(Range(i, i+1), Range(0,3));
Mat t = bigmat(Range(i, i+1), Range(3,6));

CV_Assert(rvecs[i].rows == 3 &amp;&amp; rvecs[i].cols == 1);
CV_Assert(tvecs[i].rows == 3 &amp;&amp; tvecs[i].cols == 1);
//*.t() is MatExpr (not Mat) so we can use assignment operator
r = rvecs[i].t();
t = tvecs[i].t();
}
cvWriteComment( *fs, "a set of 6-tuples (rotation vector + translation vector) for each view", 0 );
fs &lt;&lt; "extrinsic_parameters" &lt;&lt; bigmat;

}
//fs &lt;&lt; "rvecs" &lt;&lt; rvecs;
//fs &lt;&lt; "tvecs" &lt;&lt; tvecs;
if( !imagePoints.empty() )
{
Mat imagePtMat((int)imagePoints.size(), (int)imagePoints[0].size(), CV_32FC2);
for( int i = 0; i &lt; (int)imagePoints.size(); i++ )
{
Mat r = imagePtMat.row(i).reshape(2, imagePtMat.cols);
Mat imgpti(imagePoints[i]);
imgpti.copyTo(r);
}
fs &lt;&lt; "image_points" &lt;&lt; imagePtMat;
}
}

static bool readStringList( const string&amp; filename, vector&lt;string&gt;&amp; l )
{
l.resize(0);
FileStorage fs(filename, FileStorage::READ);
if( !fs.isOpened() )
return false;
FileNode n = fs.getFirstTopLevelNode();
if( n.type() != FileNode::SEQ )
return false;
FileNodeIterator it = n.begin(), it_end = n.end();
for( ; it != it_end; ++it )
l.push_back((string)*it);
return true;
}

static bool runAndSave(const string&amp; outputFilename,
const vector&lt;vector&lt;Point2f&gt; &gt;&amp; imagePoints,
Size imageSize, Size boardSize, Pattern patternType, float squareSize,
float aspectRatio, int flags, Mat&amp; cameraMatrix,
Mat&amp; distCoeffs, bool writeExtrinsics, bool writePoints )
{
vector&lt;Mat&gt; rvecs, tvecs;
vector&lt;float&gt; reprojErrs;
double totalAvgErr = 0;

bool ok = runCalibration(imagePoints, imageSize, boardSize, patternType, squareSize,
aspectRatio, flags, cameraMatrix, distCoeffs,
rvecs, tvecs, reprojErrs, totalAvgErr);
printf("%s. avg reprojection error = %.2f\n",
ok ? "Calibration succeeded" : "Calibration failed",
totalAvgErr);

if( ok )
saveCameraParams( outputFilename, imageSize,
boardSize, squareSize, aspectRatio,
flags, cameraMatrix, distCoeffs,
writeExtrinsics ? rvecs : vector&lt;Mat&gt;(),
writeExtrinsics ? tvecs : vector&lt;Mat&gt;(),
writeExtrinsics ? reprojErrs : vector&lt;float&gt;(),
writePoints ? imagePoints : vector&lt;vector&lt;Point2f&gt; &gt;(),
totalAvgErr );
return ok;
}

int main( int argc, char** argv )
{
Size boardSize, imageSize;
float squareSize = 1.f, aspectRatio = 1.f;
Mat cameraMatrix, distCoeffs;
const char* outputFilename = "out_camera_data.yml";
const char* inputFilename = 0;

int i, nframes = 10;
bool writeExtrinsics = false, writePoints = false;
bool undistortImage = false;
int flags = 0;
VideoCapture capture;
bool flipVertical = false;
bool showUndistorted = false;
bool videofile = false;
int delay = 1000;
clock_t prevTimestamp = 0;
int mode = DETECTION;
int cameraId = 0;
vector&lt;vector&lt;Point2f&gt; &gt; imagePoints;
vector&lt;string&gt; imageList;
Pattern pattern = CHESSBOARD;

if( argc &lt; 2 )
{
help();
return 0;
}

for( i = 1; i &lt; argc; i++ )
{
const char* s = argv[i];
if( strcmp( s, "-w" ) == 0 )
{
if( sscanf( argv[++i], "%u", &amp;boardSize.width ) != 1 || boardSize.width &lt;= 0 )
return fprintf( stderr, "Invalid board width\n" ), -1;
}
else if( strcmp( s, "-h" ) == 0 )
{
if( sscanf( argv[++i], "%u", &amp;boardSize.height ) != 1 || boardSize.height &lt;= 0 )
return fprintf( stderr, "Invalid board height\n" ), -1;
}
else if( strcmp( s, "-pt" ) == 0 )
{
i++;
if( !strcmp( argv[i], "circles" ) )
pattern = CIRCLES_GRID;
else if( !strcmp( argv[i], "acircles" ) )
pattern = ASYMMETRIC_CIRCLES_GRID;
else if( !strcmp( argv[i], "chessboard" ) )
pattern = CHESSBOARD;
else
return fprintf( stderr, "Invalid pattern type: must be chessboard or circles\n" ), -1;
}
else if( strcmp( s, "-s" ) == 0 )
{
if( sscanf( argv[++i], "%f", &amp;squareSize ) != 1 || squareSize &lt;= 0 )
return fprintf( stderr, "Invalid board square width\n" ), -1;
}
else if( strcmp( s, "-n" ) == 0 )
{
if( sscanf( argv[++i], "%u", &amp;nframes ) != 1 || nframes &lt;= 3 )
return printf("Invalid number of images\n" ), -1;
}
else if( strcmp( s, "-a" ) == 0 )
{
if( sscanf( argv[++i], "%f", &amp;aspectRatio ) != 1 || aspectRatio &lt;= 0 )
return printf("Invalid aspect ratio\n" ), -1;
flags |= CV_CALIB_FIX_ASPECT_RATIO;
}
else if( strcmp( s, "-d" ) == 0 )
{
if( sscanf( argv[++i], "%u", &amp;delay ) != 1 || delay &lt;= 0 )
return printf("Invalid delay\n" ), -1;
}
else if( strcmp( s, "-op" ) == 0 )
{
writePoints = true;
}
else if( strcmp( s, "-oe" ) == 0 )
{
writeExtrinsics = true;
}
else if( strcmp( s, "-zt" ) == 0 )
{
flags |= CV_CALIB_ZERO_TANGENT_DIST;
}
else if( strcmp( s, "-p" ) == 0 )
{
flags |= CV_CALIB_FIX_PRINCIPAL_POINT;
}
else if( strcmp( s, "-v" ) == 0 )
{
flipVertical = true;
}
else if( strcmp( s, "-V" ) == 0 )
{
videofile = true;
}
else if( strcmp( s, "-o" ) == 0 )
{
outputFilename = argv[++i];
}
else if( strcmp( s, "-su" ) == 0 )
{
showUndistorted = true;
}
else if( s[0] != '-' )
{
if( isdigit(s[0]) )
sscanf(s, "%d", &amp;cameraId);
else
inputFilename = s;
}
else
return fprintf( stderr, "Unknown option %s", s ), -1;
}

if( inputFilename )
{
if( !videofile &amp;&amp; readStringList(inputFilename, imageList) )
mode = CAPTURING;
else
capture.open(inputFilename);
}
else
capture.open(cameraId);

if( !capture.isOpened() &amp;&amp; imageList.empty() )
return fprintf( stderr, "Could not initialize video (%d) capture\n",cameraId ), -2;

if( !imageList.empty() )
nframes = (int)imageList.size();

if( capture.isOpened() )
printf( "%s", liveCaptureHelp );

namedWindow( "Image View", 1 );

for(i = 0;;i++)
{
Mat view, viewGray;
bool blink = false;

if( capture.isOpened() )
{
Mat view0;
capture &gt;&gt; view0;
view0.copyTo(view);
}
else if( i &lt; (int)imageList.size() )
view = imread(imageList[i], 1);

if(!view.data)
{
if( imagePoints.size() &gt; 0 )
runAndSave(outputFilename, imagePoints, imageSize,
boardSize, pattern, squareSize, aspectRatio,
flags, cameraMatrix, distCoeffs,
writeExtrinsics, writePoints);
break;
}

imageSize = view.size();

if( flipVertical )
flip( view, view, 0 );

vector&lt;Point2f&gt; pointbuf;
cvtColor(view, viewGray, CV_BGR2GRAY);

bool found;
switch( pattern )
{
case CHESSBOARD:
found = findChessboardCorners( view, boardSize, pointbuf,
CV_CALIB_CB_ADAPTIVE_THRESH | CV_CALIB_CB_FAST_CHECK | CV_CALIB_CB_NORMALIZE_IMAGE);
break;
case CIRCLES_GRID:
found = findCirclesGrid( view, boardSize, pointbuf );
break;
case ASYMMETRIC_CIRCLES_GRID:
found = findCirclesGrid( view, boardSize, pointbuf, CALIB_CB_ASYMMETRIC_GRID );
break;
default:
return fprintf( stderr, "Unknown pattern type\n" ), -1;
}

// improve the found corners' coordinate accuracy
if( pattern == CHESSBOARD &amp;&amp; found) cornerSubPix( viewGray, pointbuf, Size(11,11),
Size(-1,-1), TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 30, 0.1 ));

if( mode == CAPTURING &amp;&amp; found &amp;&amp;
(!capture.isOpened() || clock() - prevTimestamp &gt; delay*1e-3*CLOCKS_PER_SEC) )
{
imagePoints.push_back(pointbuf);
prevTimestamp = clock();
blink = capture.isOpened();
}

if(found)
drawChessboardCorners( view, boardSize, Mat(pointbuf), found );

string msg = mode == CAPTURING ? "100/100" :
mode == CALIBRATED ? "Calibrated" : "Press 'g' to start";
int baseLine = 0;
Size textSize = getTextSize(msg, 1, 1, 1, &amp;baseLine);
Point textOrigin(view.cols - 2*textSize.width - 10, view.rows - 2*baseLine - 10);

if( mode == CAPTURING )
{
if(undistortImage)
msg = format( "%d/%d Undist", (int)imagePoints.size(), nframes );
else
msg = format( "%d/%d", (int)imagePoints.size(), nframes );
}

putText( view, msg, textOrigin, 1, 1,
mode != CALIBRATED ? Scalar(0,0,255) : Scalar(0,255,0));

if( blink )
bitwise_not(view, view);

if( mode == CALIBRATED &amp;&amp; undistortImage )
{
Mat temp = view.clone();
undistort(temp, view, cameraMatrix, distCoeffs);
}

imshow("Image View", view);
int key = 0xff &amp; waitKey(capture.isOpened() ? 50 : 500);

if( (key &amp; 255) == 27 )
break;

if( key == 'u' &amp;&amp; mode == CALIBRATED )
undistortImage = !undistortImage;

if( capture.isOpened() &amp;&amp; key == 'g' )
{
mode = CAPTURING;
imagePoints.clear();
}

if( mode == CAPTURING &amp;&amp; imagePoints.size() &gt;= (unsigned)nframes )
{
if( runAndSave(outputFilename, imagePoints, imageSize,
boardSize, pattern, squareSize, aspectRatio,
flags, cameraMatrix, distCoeffs,
writeExtrinsics, writePoints))
mode = CALIBRATED;
else
mode = DETECTION;
if( !capture.isOpened() )
break;
}
}

if( !capture.isOpened() &amp;&amp; showUndistorted )
{
Mat view, rview, map1, map2;
initUndistortRectifyMap(cameraMatrix, distCoeffs, Mat(),
getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize, 1, imageSize, 0),
imageSize, CV_16SC2, map1, map2);

for( i = 0; i &lt; (int)imageList.size(); i++ )
{
view = imread(imageList[i], 1);
if(!view.data)
continue;
//undistort( view, rview, cameraMatrix, distCoeffs, cameraMatrix );
remap(view, rview, map1, map2, INTER_LINEAR);
imshow("Image View", rview);
int c = 0xff &amp; waitKey();
if( (c &amp; 255) == 27 || c == 'q' || c == 'Q' )
break;
}
}

return 0;
}

********************CODE ENDS HERE***********************************

Create a file named default.xml in the same folder that contains calibration.cpp and paste the following contents into it:

*******************CONTENT STARTS HERE****************************

<?xml version="1.0"?>
<opencv_storage>
<Settings>
<!-- Number of inner corners per a item row and column. (square, circle) -->
<BoardSize_Width> 9</BoardSize_Width>
<BoardSize_Height>6</BoardSize_Height>

<!-- The size of a square in some user defined metric system (pixel, millimeter)-->
<Square_Size>50</Square_Size>

<!-- The type of input used for camera calibration. One of: CHESSBOARD CIRCLES_GRID ASYMMETRIC_CIRCLES_GRID -->
<Calibrate_Pattern>"CHESSBOARD"</Calibrate_Pattern>

<!-- The input to use for calibration.
To use an input camera -> give the ID of the camera, like "1"
To use an input video  -> give the path of the input video, like "/tmp/x.avi"
To use an image list   -> give the path to the XML or YAML file containing the list of the images, like "/tmp/circles_list.xml"
-->
<Input>"images/CameraCalibraation/VID5/VID5.xml"</Input>
<!--  If true (non-zero) we flip the input images around the horizontal axis.-->
<Input_FlipAroundHorizontalAxis>0</Input_FlipAroundHorizontalAxis>

<!-- Time delay between frames in case of camera. -->
<Input_Delay>100</Input_Delay>

<!-- How many frames to use, for calibration. -->
<Calibrate_NrOfFrameToUse>25</Calibrate_NrOfFrameToUse>
<!-- Consider only fy as a free parameter, the ratio fx/fy stays the same as in the input cameraMatrix.
Use or not setting. 0 - False Non-Zero - True-->
<Calibrate_FixAspectRatio> 1 </Calibrate_FixAspectRatio>
<!-- If true (non-zero) tangential distortion coefficients  are set to zeros and stay zero.-->
<Calibrate_AssumeZeroTangentialDistortion>1</Calibrate_AssumeZeroTangentialDistortion>
<!-- If true (non-zero) the principal point is not changed during the global optimization.-->
<Calibrate_FixPrincipalPointAtTheCenter> 1 </Calibrate_FixPrincipalPointAtTheCenter>

<!-- The name of the output log file. -->
<Write_outputFileName>"out_camera_data.xml"</Write_outputFileName>
<!-- If true (non-zero) we write to the output file the feature points.-->
<Write_DetectedFeaturePoints>1</Write_DetectedFeaturePoints>
<!-- If true (non-zero) we write to the output file the extrinsic camera parameters.-->
<Write_extrinsicParameters>1</Write_extrinsicParameters>
<!-- If true (non-zero) we show after calibration the undistorted images.-->
<Show_UndistortedImage>1</Show_UndistortedImage>

</Settings>
</opencv_storage>

*****************CONTENT ENDS HERE***********************************

Now Lets compile it using the two commands:

1) chmod +x calibration.cpp

2) g++ -ggdb `pkg-config --cflags opencv` -o `basename calibration.cpp .cpp` calibration.cpp `pkg-config --libs opencv`

So,we will be using a chess board for calibration,Finally execute the program using

./calibration 0 -w 7 -h 7 -s 0.038 -o webcam.yml -op -oe

Where all the parameters I have set for the example execution should be changed according to the following rules of execution:

-w  — number of cells(squares)-1 horizontally

-h   —number of cells(squares)-1 vertically

-s — length of side of square in metres

webcam.yml is the output file

Before pressing “G” to start the execution make sure you get detection patterns periodically on the live-streaming video somewhat like the following image

So,finally you have calibrated the camera.

##################CHEERS###########################

Thank you 🙂

See also :- http://blindperception.wordpress.com/

CONTRAST AND BRIGHTNESS VARIATION OF IMAGES

AUTHORS:ABHISHEK KUMAR ANNAMRAJU,AKASH DEEP SINGH,ADHESH SHRIVASTAVA

Hi Friends,

Lets see how do you change contrast and brightness in opencv

Open a c++ file named contrast.cpp and paste the following code in it

Click once somewhere on the code and press ctrl+A to select whole code.You may not see the whole code so its better to copy the code and paste it in your favourite text editor and then go through it.

**********************CODE STARTS HERE*******************************

#include <cv.h>
#include <highgui.h>
#include <iostream>

using namespace cv;

double alpha; /**< Simple contrast control */
int beta;  /**< Simple brightness control */

int main( int argc, char** argv )
{

Mat image = imread( argv[1] );
Mat new_image = Mat::zeros( image.size(), image.type() );

std::cout<<" Basic Linear Transforms "<<std::endl;
std::cout<<"-------------------------"<<std::endl;
std::cout<<"* Enter the alpha value [1.0-3.0]: ";std::cin>>alpha;
std::cout<<"* Enter the beta value [0-100]: "; std::cin>>beta;

for( int y = 0; y < image.rows; y++ )
{ for( int x = 0; x < image.cols; x++ )
{ for( int c = 0; c < 3; c++ )
{
new_image.at<Vec3b>(y,x)[c] =
saturate_cast<uchar>( alpha*( image.at<Vec3b>(y,x)[c] ) + beta );
}
}
}

namedWindow("Original Image", 1);
namedWindow("New Image", 1);

imshow("Original Image", image);
imshow("New Image", new_image);

return 0;
}

**************************CODE ENDS HERE*******************************

Compilation:

1. chmod +x contrast.cpp
2. g++ -ggdb `pkg-config --cflags opencv` -o `basename contrast.cpp .cpp` contrast.cpp `pkg-config --libs opencv`

Usage:

./contrast IMAGE.EXTENSION CONTRAST_INCREASE_VALUE BRIGHTNESS_ADDING_VALUE

(eg: – ./contrast cat.jpg 2 100)