Design and implementation of a gesture control system for tablet computer based on gaze

Introduce a Gazture, New system , It has a two-tier structure ,1： Focus on real-time gaze estimation through acceptable tracking accuracy , At the same time, the cost is small .2： A robust gesture recognition algorithm is realized , While compensating for gaze estimation errors .

、gaze Widely applied

Traditional shortcomings

Our introduction to this thing

The equipment we use

There are two kinds of tracking technologies     Desktop computers and mobile terminals     What are desktop companies like , What kind of tracking and detection does everyone use , A development .

  The development of tracking and detection of mobile devices .

Why use gestures to interact  

Introduce the two-tier structure

1. Look at the tracking layer ：  Three challenges （ Can cheap equipment work , The head moves , People have different attitudes The distance between eyes and computers is different ）. Main task （ Low overhead , Estimate with acceptable accuracy , Solve the influence of head movement and posture changes ）.

        First , Extract meaningful Eye characteristics , （ Compare some classifiers , And then use SDK 了 ）, Extract eye features （ The extraction method is existing , Features include Pupil , Eyelids up and down , Left and right canthus ）.

Collected E《L,Ri》 Is a variety of i Look at the left and right eye features of the reference , G Namely Ei Corresponding gaze position .

  Currently, what is newly captured is e ,N(e,E) Is the predefined distance Is to find the eye features you want to refer to , Take the current eye feature as the same as the reference , That's the reference gaze position , It is equivalent to calculating the gaze position of the current eye

 2 Step Mapping transfer ： Through a transfer function Active eye features --》 Reference ,,,

        Collect gaze information ： Collected are eye features E' and Corresponding gaze position G’    These can be collected explicitly or implicitly on the computer , Implicit is that, for example, people may keep staring at that place when looking at the software on the computer , This can be collected implicitly .

    about G’ Find nearby G（ In the initial mapping G in ）,G Find the corresponding E（ It is also the beginning ）,E By calculating the weighted average E The waves .

  The user changed his posture , We are going to construct a transfer function Namely E Waves and E' Of . Linear transfer function E Wave vector size 20.

  Look at the formula （3）：S It's a diagonal matrix （ The scaling factor ）,T It's a vector （ Offset factor ）.

The goal is formula （4）： Find S and T The value of .min（v） It's a vector v The minimum value per dimension of .

 E’ and E The wave is transformed by the transfer function It has become or said that we have used it as a reference eye feature , Then use the formula 1 You can calculate the position ; Formula 1 is the formula calculated between the newly captured image and the reference image .

Gesture recognition layer ！！！！！！！！！！！！！！！、

  Tradition is based on location Then we should pay attention to three aspects For example, how to drop

our Gazture It is composed of a series of gaze moving directions derived from gaze positions . Yes 8 A direction How to use it Look at Figure 2 ！！！！！

  After obtaining the gaze position , To recognize gestures , But there are difficulties ： There is an error in the estimated gaze position , The number of staring positions in each direction is different , 2 The combination of the two directions is unknown .

    solve ： Direction calculation and gesture extraction .

Direction calculation ： Design the direction calculation method of sliding window . There will be gaze estimation errors or even outliers in each window .  ------- therefore The slope of gaze in each window is calculated by robust fitting method .

            Using adjacent gaze positions xy The change trend of coordinates , Determine the direction yes k still k+180 degree .

Last , We map the calculated angle to the nearest predefined direction in the gesture （ Does it mean See which gesture direction is more similar ？？？？）. We call the direction obtained in this step the sliding direction . Pictured 3 Shown , The sliding direction is from the first sliding window w1 Calculated

Gesture extraction ： You can ask questions ： The direction of calculation is still wrong ; Even if it's right , Maybe the combination of the two directions in the last gesture is wrong

    solve ：    Design a second level gesture extraction method .

          For each window Determine his direction

（ The most frequent sliding direction and predefined threshold th decision , If the most frequent >th  The window direction is the most frequent sliding direction ; otherwise , The current window direction is ignored ）

        Aggregate continuous same direction .

（ If the current direction is different from the previous direction , We add it as the direction of the current gesture ）

Algorithm is given Pseudo code

  Initial mapping ： Use random points on the computer , Then people look To collect .

      Use some functions to detect eye features Get the touch position

Suppose the user looks at these positions when touching Then the touch position is regarded as the gaze position

  Two mapping transfer modes can be supported ： Explicit and implicit   、

Gesture recognition ： Two kinds of support mode realtime and batch The batch

      Real time is to process the captured image immediately Calculate eye characteristics

      Batch processing is to get all photos with a gesture first Then take some time to deal with To get the gesture

  Real time use time is short All accuracy is not as high as batch processing

Like a game Time sensitive You can use real-time ; Like unlocking devices Sensitive to accuracy   Select batch .

  The experimental environment   Choice of volunteers   Evaluate the performance from the following aspects

The experiment is divided into two parts Calibration and gaze tracking

calibration ： There is a point People see , If you capture two consecutive pictures If you don't move your eyes, you are watching . Recording point location and eye characteristics

Watch and track ：

Eye tracking point 50 Time , The system will record the position of the stimulus and the estimated gaze position . The accuracy of gaze tracking can be expressed by the Euclidean distance between the stimulus position and the estimated gaze position . The smaller Euclidean distance , The higher the accuracy . The tracking speed is the reciprocal of the frame processing time .