Perceive the attention status of users on smart phones

In the device 、 Applications and web The ubiquitous computing era with increasing services , Users' attention has become the new neck of Computing .

This paper presents a new middleware Attelia, The middleware does not need any special psychology - Physiological sensor , It can sense the user's attention state on the user's smartphone in real time .

In order to better find the delivery time of interrupt notification from various applications and services to mobile users ,Attelia Detect the breakpoint of the user's activity on the smartphone [16], Use our novel “ Applications act as sensors ”(AsaS) Methods and machine learning techniques .

In the era of Ubiquitous Computing , User “ attention ” remain unchanged , The amount of information provided is increasing . such as ：

Number of multifunctional network devices , Applications used by users 、web The number of service and communication channels is also increasing .

In this context , Limited human attention resources have become the new bottle neck of Computing .

From the perspective of human users , This excess information is often referred to as “ Information overload ”.

Especially in this study , We will focus on interrupt overload , The interruption caused by too many notifications from the computing system and improper delivery interferes with users .

Interrupt overload ：

      The main reason is from the computing system “ notice ”.

    A typical instant notification system will have a variety of negative effects on users' work efficiency and even their emotions .

  Several significant features were also observed in the recent notice .

     • Increase the diversity of notification types and sources

Adaptive processing support for such notifications , It includes dynamically adjusting the notification time according to the user's current attention state and the information to be notified 、 Media or content , Obviously, it is necessary to alleviate the interrupt overload of users .

The different requirements for this support are as follows .

• The feasibility of mobile devices

• Real time sensing

• Applicable to different types of notification sources

• Compatibility for all-weather use     Is playing mobile phones all day Then this function is not stuck

To achieve this adaptive notification support , We propose a new middleware , Perceive users' attention state on users' smartphones .

 A Detect breakpoints ：  The earliest concept in the field of Psychology , As Perceive the time goal of appropriate interruption time .

                      A breakpoint is the boundary between two adjacent actions ,  Human's perception system divides users' goal oriented activities .

We use a method to perceive coarse-grained and easy to measure indicators ,  Just use the current equipment , To determine the appropriate notification time .

The specific methods ： A Introduce a new AsaS Method ,    User applications （app） The usage mode will play   Sensors and machine learning technology .

app As a sensor , Running on the phone , Achieve affinity with mobile devices .

Through machine learning technology ,Attelia Realize perception . As a middleware under the application , Enter... From the application of the current operation UI event , Realize applicability and easy deployment for different applications . Yes app Don't modify . No special sensors are required ,so All day use OK.

 Attelia System architecture   ： What does the whole system look like Pictured above 1  What it contains is as follows

UIEventLogger,UI Event recorder

BreakPointLogger, Breakpoint recorder

FeatureExtractor, Feature extractor

 Weka[15] machine learning engine, Weka[15] Machine learning engine ,

 GroundTruthAnnotator application, Manually annotated small data set annotator   Applications

off-line components for model training. Offline component of model training

GroundTruthAnnotator There are many explanations ......？？？？？

Attelia It can be implemented on Android platform , You can enter and record UI Flow of events （ knock , Click on , Scroll or modify UI Components ）

It can be distributed on Google platform , It is helpful for system deployment .

The whole project ：：

1. Collection Real label Stage ：

During application use , The real tag value of the breakpoint time is ： Collect through user's voluntary manual annotation .

（ Where is the person on this application page It's the real label ）

chart 2 Shows a screenshot ： our Annotation Applications Floating on the phone screen .

In the operation of ordinary Android Application time , The user presses the float button , Time to press the breakpoint .

Attelia The service will continue UI Flow of events ( Events from the Comment button are not included ) And breakpoint timestamp ( Time to press the Comment button ) Record to local storage .  （UI Flow of events （ knock , Click on , Scroll or modify UI Components ））

2. In the offline model training stage ：

Every 3 A period of seconds   Will extract 45 Defining features , Pictured 3 Shown ,

And input it into Weka To train the classifier model .

3. Testing phase , Service captured UI The event flow will be input into the model in real time on the mobile device , The service will dynamically detect breakpoints

assessment ：    Find some people It is required to use our system on Samsung mobile phones Use ten randomly and naturally app Use for five minutes ,              Look at the breakpoint by feeling .

Data speak ： On two models And unified model   The results are very good

summary Conclusion ：

    This paper presents a new middleware ：Attelia

    middleware ： It can sense the attention state of users on smart phones in real time , There is no need for any special psychology - Physiological sensor .

                  Take advantage of our novel “ Application as a sensor AsaS” Methods and machine learning techniques , Detect breakpoints in user activity on smartphones .

AsaS The specific methods ： User applications （app） The usage mode will play   Sensors and machine learning technology .

                Through machine learning technology ,Attelia Realize perception . As a middleware under the application , Enter... From the application of the current operation UI event , Realize applicability and easy deployment for different applications . Yes app Don't modify . No special sensors are required ,so All day use OK

Finally received the support of the Association ！

    Our preliminary assessment shows that the accuracy rate is quite optimistic 80-90% about . This study was supported by the National Institute of information and communication technology (NICT) Partial support for .