当前位置：网站首页>Opencv programming: opencv3 X trains its own classifier

Opencv programming: opencv3 X trains its own classifier

2022-07-19 10:49:00 【InfoQ】

One 、 Introduction to the environment

operating system :

windows10 64 position

QT edition :

5.12.6 ( My program is mainly QT+OpenCV Realize image processing and display )

OpenCV edition :

OpenCV3.4.7

Two 、 Download and install OpenCV

windows Don't download the source code , You can download the compiled file directly on the official website and decompress it to use .

OpenCV Official website download address :

https://opencv.org/releases/

After downloading, unzip it to the specified directory , I decompress it directly to C Discoid .

Because the version downloaded on the official website is VC edition , And my QT It uses MinGW compiler , The libraries in the installation package downloaded from the official website above and below can't be used , Need to download another one MinGW edition . Download address ：https://github.com/huihut/OpenCV-MinGW-Build

Why do I need to download two versions ？ In fact, it's mainly MinGW Version of OpenCV Two training classifiers in (opencv_traincascade.exe) The file of cannot be used on my computer , Possible library conflicts , The specific problems have not been thoroughly studied , Just download another one VC The version is OpenCV,VC In the version opencv_traincascade.exe Files can be used normally . Actually downloaded VC edition OpenCV Mainly to use these two files （opencv_traincascade.exe、opencv_createsamples.exe）

3、 ... and 、 test OpenCV Its own classifier

3.1 Introduction to the built-in classifier file

OpenCV The official has provided many trained classifier files , stay OpenCV The installation directory of is .

Common Face detection 、 Eye detection 、 Cat face detection 、 Pedestrian detection, etc , see XML You can know the name of the file .

Write below QT Program , call OpenCV Cascade classifier for testing .

3.2 QT Example code for

Below QT The interface is very simple , Mainly for testing classifier files .

xxx.cpp File code ：

#include &quot;widget.h&quot;
#include &quot;ui_widget.h&quot;

Widget::Widget(QWidget *parent)
 : QWidget(parent)
 , ui(new Ui::Widget)
{
 ui->setupUi(this);
 dir=&quot;C:/&quot;;

 ui->label_source->setAlignment(Qt::AlignVCenter);
 ui->label_2->setAlignment(Qt::AlignVCenter);
}

Widget::~Widget()
{
 delete ui;
}


QImage Widget::Mat2QImage(const Mat& mat)
{
 // 8-bits unsigned, NO. OF CHANNELS = 1
 if(mat.type() == CV_8UC1)
 {
 QImage image(mat.cols, mat.rows, QImage::Format_Indexed8);
 // Set the color table (used to translate colour indexes to qRgb values)
 image.setColorCount(256);
 for(int i = 0; i < 256; i++)
 {
 image.setColor(i, qRgb(i, i, i));
 }
 // Copy input Mat
 uchar *pSrc = mat.data;
 for(int row = 0; row < mat.rows; row ++)
 {
 uchar *pDest = image.scanLine(row);
 memcpy(pDest, pSrc, mat.cols);
 pSrc += mat.step;
 }
 return image;
 }
 // 8-bits unsigned, NO. OF CHANNELS = 3
 else if(mat.type() == CV_8UC3)
 {
 // Copy input Mat
 const uchar *pSrc = (const uchar*)mat.data;
 // Create QImage with same dimensions as input Mat
 QImage image(pSrc, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
 return image.rgbSwapped();
 }
 else if(mat.type() == CV_8UC4)
 {
 // Copy input Mat
 const uchar *pSrc = (const uchar*)mat.data;
 // Create QImage with same dimensions as input Mat
 QImage image(pSrc, mat.cols, mat.rows, mat.step, QImage::Format_ARGB32);
 return image.copy();
 }
 else
 {
 return QImage();
 }
}

// Open local picture 
void Widget::on_pushButton_open_clicked()
{
 filename=QFileDialog::getOpenFileName(this,&quot; Select the open file &quot;,dir,tr(&quot;*.bmp *.jpg *.png&quot;));
 if(filename.isEmpty())return;
 QFileInfo info(filename);
 dir=info.path(); // Save the current path 


}

// Face detection code 
void Widget::opencv_face(QImage qImage)
{
 QTime time;
 time.start();

 // Define cascading classifiers 
 CascadeClassifier face_cascade;
 // Load classification file 
 //
 if( !face_cascade.load(&quot;C:/OpenCV_3.4.7/OpenCV-MinGW-Build-OpenCV-3.4.7/etc/haarcascades/haarcascade_frontalcatface.xml&quot;) )
 {
 qDebug()<<&quot;haarcascade_frontalface_alt.xml  Classifier loading error &quot;;
 return;
 }
 Mat frame=QImage2cvMat(qImage);
 cvtColor( frame, frame, COLOR_BGR2GRAY );// Convert the image to grayscale 

 std::vector<Rect> faces;
 // Face detection 
 face_cascade.detectMultiScale(frame,faces);
 qDebug()<<tr(&quot; Time consuming :%1 ms  distinguish :%2  Number :%3\n&quot;).arg(time.elapsed()).arg(faces.size()).arg(faces.size());

 for ( size_t i = 0; i < faces.size(); i++ )
 {
 #if 1
 Point center(faces[i].x + faces[i].width/2, faces[i].y + faces[i].height/2);
 ellipse(frame, center, Size( faces[i].width/2, faces[i].height/2 ), 0, 0, 360, Scalar( 255, 0, 255 ), 4 );
 rectangle(frame,
 cvPoint(cvRound(faces[i].x), cvRound(faces[i].y)),
 cvPoint(cvRound((faces[i].x + faces[i].width-1)),
 cvRound((faces[i].y + faces[i].height-1))),
 Scalar(255, 255, 255), 3, 8, 0);
 #endif

 // Extract the recognition results 
 Mat frame1;
 for(size_t i=0;i<faces.size();i++)
 {
 Point center(faces[i].x + faces[i].width / 2, faces[i].y + faces[i].height / 2);
 frame1= frame(Rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height));
 }
 /* Display the recognition result on the control */
 ui->label_2->setPixmap(QPixmap::fromImage(Mat2QImage(frame1)));
 }

 /* Display... On the control */
 QImage display_image=Mat2QImage(frame);
 ui->label_source->setPixmap(QPixmap::fromImage(display_image));
}

Mat Widget::QImage2cvMat(QImage image)
{
 Mat mat;
 switch(image.format())
 {
 case QImage::Format_ARGB32:
 case QImage::Format_RGB32:
 case QImage::Format_ARGB32_Premultiplied:
 mat = Mat(image.height(), image.width(), CV_8UC4, (void*)image.constBits(), image.bytesPerLine());
 break;
 case QImage::Format_RGB888:
 mat = Mat(image.height(), image.width(), CV_8UC3, (void*)image.constBits(), image.bytesPerLine());
 cvtColor(mat, mat, CV_BGR2RGB);
 break;
 case QImage::Format_Indexed8:
 mat = Mat(image.height(), image.width(), CV_8UC1, (void*)image.constBits(), image.bytesPerLine());
 break;
 }
 return mat;
}


void Widget::on_pushButton_start_clicked()
{
 opencv_face(QImage(filename).scaled(ui->label_source->size(),Qt::KeepAspectRatio));
}

xxx.h File code ：

#ifndef WIDGET_H
#define WIDGET_H

#include <QWidget>
#include &quot;opencv2/core/core.hpp&quot;
#include &quot;opencv2/core/core_c.h&quot;
#include &quot;opencv2/objdetect.hpp&quot;
#include &quot;opencv2/highgui.hpp&quot;
#include &quot;opencv2/imgproc.hpp&quot;

#include <QFileDialog>
#include <QTime>
#include <QDebug>

using namespace cv;

QT_BEGIN_NAMESPACE
namespace Ui { class Widget; }
QT_END_NAMESPACE

class Widget : public QWidget
{
 Q_OBJECT

public:
 Widget(QWidget *parent = nullptr);
 ~Widget();
 QImage Mat2QImage(const Mat& mat);
 void opencv_face(QImage qImage);
 Mat QImage2cvMat(QImage image);
 QString dir;
 QString filename;
private slots:
 void on_pushButton_open_clicked();

 void on_pushButton_start_clicked();

private:
 Ui::Widget *ui;
};
#endif // WIDGET_H

xxx.pro file

QT += core gui

greaterThan(QT_MAJOR_VERSION, 4): QT += widgets

CONFIG += c++11

# The following define makes your compiler emit warnings if you use
# any Qt feature that has been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS

# You can also make your code fail to compile if it uses deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0

SOURCES += \
 main.cpp \
 widget.cpp

HEADERS += \
 widget.h

FORMS += \
 widget.ui

# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target

#linu Path setting of the platform 
linux {
message(' function linu edition ')
# add to opencv The path to the header file , You need to modify it according to your own header file path 
INCLUDEPATH+=/home/wbyq/work_pc/opencv-3.4.9/_install/install/include\
 /home/wbyq/work_pc/opencv-3.4.9/_install/install/include/opencv\
 /home/wbyq/work_pc/opencv-3.4.9/_install/install/include/opencv2

LIBS+=/home/wbyq/work_pc/opencv-3.4.9/_install/install/lib/libopencv_*
}

win32
{
 message(' function win32 edition ')
 # add to opencv The path to the header file , You need to modify it according to your own header file path 
 INCLUDEPATH+=C:/OpenCV_3.4.7/OpenCV-MinGW-Build-OpenCV-3.4.7/include \
 C:/OpenCV_3.4.7/OpenCV-MinGW-Build-OpenCV-3.4.7/include/opencv \
 C:/OpenCV_3.4.7/OpenCV-MinGW-Build-OpenCV-3.4.7/include/opencv2
 LIBS+=C:/OpenCV_3.4.7/OpenCV-MinGW-Build-OpenCV-3.4.7/x86/mingw/bin/libopencv_*.dll
}

xxx.ui file ：

3.3 Test the effect of face classifier

Replace the classifier file in the code with ：haarcascade_frontalface_alt2.xml

This QT The code is just for simple testing , There is no thread to identify , If recognition takes a long time , In the process of identification UI The interface will get stuck , Just a moment .

3.4 Test the effect of cat face classifier

Replace the classifier file in the code with ：haarcascade_frontalcatface.xml

3.5 Test the effect of pedestrian detection classifier

Replace the classifier file in the code with ：haarcascade_fullbody.xml

Four 、 Train your classifier

4.1 Preface

If the object you actually want to detect is OpenCV There is no classifier in it , perhaps OpenCV The recognition accuracy of the built-in classifier does not meet the requirements , You can use OpenCV The self-contained classifier program trains itself . Training methods there are many online tutorials , The following is a simple description of the training process of building wheels repeatedly .

explain : Because the following content is mainly a simple description of the training process , So I prepared fewer samples , If you need to see the following instructions for actual training, increase the number of samples .

4.1 Description of positive and negative sample materials for training

Want the calculator to recognize the specified object , First of all, let the calculator know what the object you want to recognize looks like , Need to learn in advance . Learning process , A positive sample and a negative sample need to be prepared . A positive sample is the object to be identified ; Negative samples are used to compare with positive samples , Negative samples do not contain pictures in positive samples or similar pictures , But you can't choose randomly , It's best to view the positive sample in the same environment , It works best , Reduce false identification .

It's best to use grayscale images for sample pictures （ That's black and white ）; The more samples, the better , Try to be higher than 1000, The greater the difference between samples, the better , The proportion of positive and negative samples can be 1：3, The best size recommended by the official training sample is 20x20, Do not use special characters in the naming of sample pictures , Just use a normal name .

All pictures of the positive sample must have the same size , If inconsistent or larger , You can scale all samples to 20*20.

Recommend a simple image processing tool ：

https://blog.csdn.net/xiaolong1126626497/article/details/106085795

Size determines the length of training , Large size can also be trained , If the picture is too small, many details will be lost , The size can be weighed according to the actual situation , But too large image sample training may lead to insufficient memory , The specific situation can be adjusted according to the training effect and situation .

Be careful , In order to improve the accuracy of training , Negative samples cannot be randomly selected .

such as : The positive sample tested is the wheel , Then the best negative sample is the car body , road 、 guardrail 、 Wait, environment .

4.2 Positive sample image example

The following is a positive sample image of the car , A positive sample image can be created PositiveSample Folder storage .

4.3 Negative sample image example

A negative sample image can be created NegativeSample Folder storage . Negative sample images do not require sample size , But it should be greater than or equal to the size of the positive sample ; And negative samples cannot be repeated , To increase the difference of negative samples . Negative samples should also be grayed , Same as positive sample operation .

4.4 Create a working directory

In any directory on the computer , Create a working directory OpenCV_TrainingData, Copy the directory storing positive and negative samples to OpenCV_TrainingData Under the table of contents , Create another XML Catalog , Used to store the generated training files .

3.2 Create a positive sample description file

Open the computer command line terminal .

Use cd The command enters the directory of positive samples .

Execute the order as follows ：

 command 1- Enter the positive sample directory ：cd /d D:\linux-share-dir\OpenCV_TrainingData\PositiveSample

 command 2- Output the names and paths of all pictures in the directory to pos.txt file ：dir /b/s/p/w *.jpg > pos.txt

Open generated pos.txt The contents of the document are as follows .

Slightly modify the contents of the document , add

Number of detection targets 、 The upper left position coordinate of the target image 、 Picture width and height parameters

The size of the sample pictures I prepared here are 40x40, So fill in the code ：1 0 0 40 40

The modification effect is as follows ：

If the number of pictures is large , Manual modification is troublesome ,

Use the text editor directly , Search and replace .

3.3 Create a negative sample description file

The creation method of negative sample description file is the same as that of positive sample description file , Enter the directory of negative sample pictures , Generate neg.txt file , The code is as follows ：

 command 1：cd /d D:\linux-share-dir\OpenCV_TrainingData\NegativeSample

 command 2：dir /b/s/p/w *.jpg > neg.txt

Be careful ： Negative sample neg.txt The file does not need to be modified , The following is the final document .

3.4 Generate positive samples .vec file

For the convenience of filling the path , The generated positive and negative sample description file pos.txt and neg.txt Copy to the upper directory .

Positive sample .vec File generation , Execute the order as follows ：

 command 1：cd /d D:\linux-share-dir\OpenCV_TrainingData

 command 2：C:\OpenCV_3.4.7\opencv-vc-3.4.7\build\x64\vc15\bin\opencv_createsamples.exe -vec pos.vec -info pos.txt -num 54 -w 40 -h 40

Parameter Introduction ：

opencv_createsamples.exe: Executable program that generates the sample description file (opencv Bring their own ), In front is the path on my computer .

-vec pos.vec Specify the generated vec file

-info pos.txt Specify the description file of the source sample

-num 54 Specify the total number of calibration target samples , That is, all the sections in the sample description file 2 The sum of the numbers in the column .

-w 40 Specify the width of the sample after scaling , If the previous picture is not 40, Then this will be scaled to 40, With this parameter, the previous image processing process can be omitted .

-h 40 Specify the height of the sample after scaling , If the previous picture is not 40, Then this will be scaled to 40, With this parameter, the previous image processing process can be omitted .

On my computer OpenCV Installation path for ：

The results are as follows ：

After successful execution, generate pos.vec file .

explain ： Negative samples do not need to be generated vec file .

3.5 Start training samples

 command 1：cd /d D:\linux-share-dir\OpenCV_TrainingData
 command 2：C:\OpenCV_3.4.7\opencv-vc-3.4.7\build\x64\vc15\bin\opencv_traincascade.exe -data XML -vec pos.vec -bg neg.txt -numPos 50 -numNeg 133 -numStages 20 -w 40 -h 40 -mode ALL

Parameter Introduction ：

-data Specify output directory , Training produces xml The files are in this directory

-vec Specify the vec file

-bg Specify the negative sample data file , That is, the previously generated neg.txt file

-numPos Specify the number of positive samples , This value must be less than the number of positive samples prepared , Otherwise it will be reported can not get new positive sample.

Reference reason ：minHitRate： Affect each strong classifier threshold , When set to 0.95 If the number of training samples is 10000 individual , So one of them 500 One is likely to betray. Don't be a negative sample , When choosing the second time, you must choose the following 500 individual , According to this law, we add more for each level behind numPos*minHitRate A positive sample , According to the training series, the following formula can be obtained

numPos+（numStages-1）

numPos

（1-minHitRate）《= Training samples prepared

The above formula only sets the number of positive samples participating in the training according to the training level and the sum of the prepared positive samples , It can only be used as an estimate , Less than the calculated number may be no problem , But there must be a problem if it is greater than that number

Now explain ” There may be a problem “ How to understand ： Because we always add a fixed number of positive training samples every time by default , But sometimes there may be samples that do not meet the conditions in the following fixed number of positive training samples , These samples are similar to the samples we excluded , So for example, we plan to add 500 One sample is enough , But it actually needs to be added 600 individual , Then there is a problem .

From the results of the above example, we can see , We are allowed to throw away at each level 12000*0.001 A positive sample =12, It should be noted that in case 11 One or more 10 One follows the first 12 The threshold of is the same , So we lost the front 10 A or 9 A just , Therefore, the number of each increase may be less than 12 individual , Greater than 12 The situation of one is the above ” There may be a problem “.

-numStages Specify the training level

-numNeg Specify the number of negative samples

-w 40 -h 40 Specify sample drawing size

-mode Appoint haar Types of features ,basio Use only vertical features ,al1 Indicates the use of vertical and 45 Degree rotation feature

Start training ：

After successful training, in XML The directory will generate cascade.xml file , This document is the final successful training document , You can test in the above code instead .