Human motion recognition evaluation summary learning

Study paper ：

[1] Song Zhen , Zhang Yushu , Yang Gang . A review of human motion recognition and evaluation [J]. Journal of Communication University of China ( Natural science edition ),2021,28(03):58-65.DOI:10.16196/j.cnki.issn.1673-4793.2021.03.009.

1. Action recognition and action evaluation

1.1 Action recognition

Definition ： It refers to the given action sequence data （ Video or 3D action sequence ） Analyze , Identify and judge the action category it contains

1.2 Action Evaluation

Definition ： It is to evaluate the completion quality of a certain standard action , It is mostly used in gymnastics 、 rowing 、 Dance and other professional areas of action evaluation and action training . It often needs to be carried out on the basis of action recognition , Standardization of actions in professional fields through expert knowledge 、 Fluency 、 Judge artistically .

1.3 The difference

Action recognition can be regarded as a multi classification problem , It is mainly to quantitatively compare the similarity between the input data and the standard reference , Then output the label of the action type ; In action evaluation , It is not simply to judge the data similarity , It focuses on the guidance of expert knowledge , Standardization of actions in professional fields 、 Fluency 、 Artistry, even the analysis and evaluation of the strength of human muscles , It measures a deeper level 、 More professional similarity

1.4 Basic technical route

Data objects 、 Data preprocessing 、 feature extraction 、 Research on classification method and motion evaluation of motion recognition

2. technology roadmap

2.1 data type

The two main types of data used are video data and 3D Bone data

2.1.1 Video data

Compared with image data , There is one more video data Temporal dimension

Two ways to extract features ：

1. A method of directly extracting and classifying the spatiotemporal features of sequences

2. Extract bone information (2D or 3D Bone information ) Training

2.1.2 3D Bone data

Capture directly through specific motion capture settings 3D Bone data , That is, bone animation data

2.2 Data preprocessing

2.2.1 Denoise

about 3D Bone data , It is basically not affected by the acquisition environment , Low noise , Basically, there is no need to denoise , For video data , The information is unstable or redundant , So it must be handled .

Common methods ：

1. Cavity repair
2. Image smoothing （ Markov random Airport ）

2.2.2 Space time alignment

Space time alignment problem ： Different people have different speed when exercising , To make the key frames of comparison correspond , The space-time of the two videos must be aligned .

Bone standardization ： In space , Different people have different bone sizes , This will interfere with the comparison of some parameters , Such as joint angle 、 Angular velocity, etc , So we should first carry out bone standardization

1. Align time series ： Fixed sliding window , Introduce the concept of action point of time anchor

2. Align space bones

3. Space action alignment ： Between bodies （ By shoulders and torso 3D Rotation offset of position extraction ）

3. Action recognition feature description method

3.1 Video data features

3.1.1 Local feature description

A bottom-up description , It is to extract useful geometric regions from around feature points , And generate an identifying vector to represent the characteristics of this region . Local features are not susceptible to environmental noise 、 The influence of object occlusion or human motion changes , Zoom 、 Translation and rotation operations also have good stability

3.3.2 Global feature description

Describe the identified target as a whole , It covers human body information , Represents high-level features or semantics .

3.2 3D Feature description of bone data

3.2.1 Joint based descriptor

Joint based descriptors are designed to establish correlations between the positions of body joints , Consider all 3D The pairwise distance between bones . Each individual eigenvalue passes K‑means Cluster as 5 One of the groups , Binary vectors are used to represent each cluster index .（ there 5 A group should be corresponding to the body 5 A part ）

shortcoming ： This descriptor lacks time information , The description of the action is not accurate enough .

3.2.2 Mining based descriptors

Mining based descriptors refer to the classification of actions according to the participation of body parts in actions , Similar to data mining , Associate with related actions through partial joint subsets of actions （ Usually, the subset of joints involved in the action is similar ）

3.2.3 Dynamics based descriptor

Dynamics based descriptors focus on representing actions as collections of joint 3D trajectories , It can more clearly and intuitively describe the characteristics of bones .

Calculate the joint position of the current joint and the differential characteristics of the speed and acceleration of the human joint to represent the local three-dimensional human posture

4. Action recognition classification method

4.1 traditional method

1. The hidden Markov model (HMMS)

   A sequence related , Stochastic model based on transition probability and transmission probability , The probability of the current state of the system is only related to the state of the previous moment , It has nothing to do with other historical state conditions

2. nonlinear SVM

4.2 Deep learning method

4.2.1CNN

4.2.2 Dual stream networks

Adopt a two branch network architecture , Capture the spatial and temporal information of the video respectively .

airspace utilize RGB Image as input to extract appearance features

Time domain Using optical flow information as input to extract temporal features

And then through Multitasking training Two behavior recognition data sets are classified by the method of , Remove over fitting , And get better results

5. Action Evaluation

5.1 Feature description

1. For different professional movements , Each body joint plays a different role , You can assign weights to each joint according to expert knowledge , Lay the foundation for action evaluation
2. Combine action evaluation with big data , thus , There is a basis for how to evaluate each action detail , It greatly increases the reliability of action evaluation
3. For some corrective research , You can customize the rules and label all possible errors , Evaluate the correctness of action execution