Electroencephalography and Eye Gaze Movement Signals ’ Usage for Estimation of User Interface Usability

Task execution time is one of components used to estimate system’s usability. Since information search is one of the most important tasks in any computer system, the eye movement should be taken into account while designing system’s interface. Fitts’ law allows to estimate probable eye movement time based on the distance between objects and their size. User’s motivation can significantly influence the eye movement speed. In this paper we analyze experimentally how Fitts’ law parameter (slope) changes when user is motivated and when there is less motivation to execute the task. Our experiments confirm that gamma brainwave activity can be used as a metric to estimate the motivation level. This allows a more objective estimation of the motivation level as compared to situation when motivation level is estimated by an expert’s observation. It is also demonstrated that the slope in Fitts’ law is up to five times smaller when children of age two to four years are motivated to reach the object and the motivation can be raised by presenting new, unseen objects. Obtained results allow a more precise estimation of the eye movement task execution time as well as provide additional insights for system interface designers aiming to attract small children. DOI: http://dx.doi.org/10.5755/j01.eee.21.5.12511

1 Abstract-Task execution time is one of components used to estimate system's usability.Since information search is one of the most important tasks in any computer system, the eye movement should be taken into account while designing system's interface.Fitts' law allows to estimate probable eye movement time based on the distance between objects and their size.User's motivation can significantly influence the eye movement speed.In this paper we analyse experimentally how Fitts' law parameter (slope) changes when user is motivated and when there is less motivation to execute the task.Our experiments confirm that gamma brainwave activity can be used as a metric to estimate the motivation level.This allows a more objective estimation of the motivation level as compared to situation when motivation level is estimated by an expert's observation.It is also demonstrated that the slope in Fitts' law is up to five times smaller when children of age two to four years are motivated to reach the object and the motivation can be raised by presenting new, unseen objects.Obtained results allow a more precise estimation of the eye movement task execution time as well as provide additional insights for system interface designers aiming to attract small children.

I. INTRODUCTION
The design of system usability is a very complicated, multidimensional task.According to J. Nielsen [1] system usability aims to identify problematic places rather than to find universal rules for perfect system design.This is highlighted by other authors [2]- [8].In these papers ergonomic properties of various systems are analysed and specific recommendations provided.It is pointed out that the usability is strongly dependent on user's properties as well.In the standard of the International Organization for Standardization and the International Electro technical Commission ISO/IEC 9126-1, 2001 [9], software quality is divided into seven main categories with one called "Quality in Use" as described in part 4 [10] and depending on the system's user properties.
One of metrics in this standard is Task time (Ta) -how long does it take to complete a task.Task type and complexity can vary depending of the type of the system; however the most common task in an information system is to find some information or even to notice some information on the user interface (UI).It takes time to move the eye gaze from one point in the UI to another and just then certain actions are done to finish the task.The eye gaze movement as a usability characteristic is important as current tendencies implies the eye movement can be used as input device instead of mouse or other device [11]- [13].Therefore eye gaze movement could be used for complex UI task execution rather than visual search only.
Fitts' law was published on 1954 in paper "The information capacity of the human motor system in controlling the amplitude of movement" [14] and revealed how the speed and accuracy of muscle movement depends on each other.The author presented the idea that the movement is not straight to the centre of the target: first of all there is a jump close to the object (see Fig. 1) with some circle of error for the first try and then it is corrected with a smaller circle of error for the second try until the target is reached.Step-wise movement towards target [12].
Despite the fact that Fitts did not present a formula by himself, his idea was formalized by other authors [15] and is presented in (1) where t is the mean time to hit the centre of a target and There were multiple experiments with Fitts' law, when it was adapted to different Human Computer Interaction (HCI) devices [16]- [22].Depending on the used HCI the slope parameter is different; however Fitts' law can be applied to most of them.This also is important in task execution time calculation as we can define how long the user will need to see the object as well as how long it will take to reach the object, using different HCI devices, as while the object is not reached usually any actions can be done with it.
To get the task execution time we can divide certain task into small steps and with the help of Fitts' law to calculate how long it will take for the user to get from one place in the UI to another (by seeing it or by reaching it with some other HCI device).The movement time together with additional time needed to execute certain activity after the object is reached can help to identify the task execution time with more precision.As well it can be used to analyse UI layout efficiently in a sense of task execution time (to identify which layout is more appropriate for a certain task execution, when the most important criteria is task execution time).
Even though existing discussions on whether the Fitts' law is suitable for eye movement [23] and limitations of Fitts' law to achieve accurate results when ID is close to zero [24], [25], this law is still actually used in system usability testing and the only usability related quantitative law proved by experiments.However there are just a few papers on how Fitts' law parameter is related to user's properties (usually medical condition and no published papers on estimation of motivation level to the user, its eye movement slope value b) [26]- [28].
By grooving up the motivation can be trained even to an unconscious level.It means when an adult gets some task a certain level of motivation automatically exists in the execution of the task [29].However it does not mean all adults are motivated all the time.Usually there are situations when persons are using different applications with no reason, to spend some time only or do automatic actions as UI is known very well for him.Meanwhile during usability test adults get a certain task and most of them are willing to finish it as quick as possible.Automatic prejudice comes into play, when after giving some task to the user he or she might guess the purpose of the task and unconsciously starts acting differently comparing to normal work with the system.All these reasons must be taken into account in order to execute a clear experiment to identify the impact of motivation level to the slope of eye gaze movement.
One of way to monitor a person's emotions is to analyse electroencephalography (EEG) signals.During the last few years more brainwave research devices appeared [30].This allowed investigating user's emotional properties, brainwave activity, to use EEG in HCI [31]- [33].The analysis of brainwave is not a simple task as different types of brainwaves exists [34] (see Table II) and it is still a discussion how to interpret these signals correctly.Previous researches found delta waves are primarily associated with deep sleep and the waking stage.As the signal frequency is very low sometimes it might be confused with signals of bigger muscles of the neck or jaw.Theta waves usually indicate emotional discomforts such as stress, frustration, disappointment as well as creative inspiration or deep meditation.Alpha waves are indicating a relaxed awareness and inattention, while beta waves usually associated with active thinking, active attention, and focus on the outside world or solving concrete problems.Comparing these to brainwaves alpha indicates a mindless state rather than a passive one, and can be reduced or eliminated by hearing unfamiliar sounds, anxiety or mental concentration.Gamma waves have the biggest frequency.Its activity is related to subjective awareness, attention.Some brainwave association is still discussed as there are many different arguments to what these waves could mean, how it could be caused, how different combinations of brainwave activity should be interpreted.One of ideas is that beta and gamma waves usually are associated with attention while the combination of both of them is correlated to focused attention [35].This means EEG and betas as well as gamma brainwave analysis can be a suitable tool to identify is the child motivated during some task or no, as his or her brain activity might say more on this question comparing the child.
The aim of this work is to analyse how the slope parameter changes in Fitts' law when different user's motivation level exists.This will provide data whether the user's motivation influences the eye gaze movement speed, depending on whether is he or she motivated or not.This analysis will be on answering two research questions: 1. How to measure person's motivation level or its changes? 2. Does the motivation level influences the eye gaze movement speed and should motivation level be taken into account for eye gaze movement time prediction in Fitts law?By defining the difference between motivated and less motivated users, system UI designers will be able to define the expected task execution time even more precisely, taking into account the motivation level of the user.While motivation level estimation process will allow gathering data on slope value range in different motivation levels.

II. EXPERIMENT METHODOLOGY
In this work we chose to analyse small children from ages 2 to 4. Children of this age are capable to choose freely and have minimal level of prejudice [36].This allows us to get a real expression to different UI rather than impact of addiction to some kind of UI.In addition small children are easier to motivate for short time period [29] therefore it is easier to achieve significant change in motivation levels during the experiment.However when using children in this of experiment rather than adults it is difficult to ask the child how motivated they were during a certain task -the dictionary of children is not enough to express his or her emotions, children might not be aware of the term motivation, how to compare it etc.Therefore additional methods for motivation level estimation have to be used.
To investigate the impact of motivation on eye gaze movement slope we recruited fourteen participants, as recommended by Microsoft for usability testing experience with children [36].All of them were children from 2 to 4 years old.Current research states children in this age group are too young to clearly express their satisfaction levels [37].Therefore there are no unified methods to test system usability with this age group.All of them participated in this experiment with one of his or her parents.Parents did not participate in the experiment but were used to keep the child calm and relaxed.Before the experiment started, all parents had to sign an agreement and to provide information about his children's daily/weekly activity with computer.According to our pre-experiment questionnaire all participants used computers more than once per week, but not more than 20 min per day; their daily/weekly computing activities were YouTube movies, mostly series "Masha i Medved".
As all children in the experiment were familiar with YouTube and series of "Masha i Medved" movie, all situations were created to imitate UI for YouTube movie view.Children were asked to sit by the computer and to choose which movie they want to see.Children were allowed to click on the picture by themselves, however as some children did not know how to play the they were helped by the personnel -a child had to look at the movie scene and to tell which of these movies he or she wanted to see.
There were 5 different UI, which had multiple starting scenes of the movie on each of them, located in different places of the screen.The number of 5 experiments was chosen to make sure the child will not get bored.As well the size, dimension of the movie scene were constant for all pictures.Changes were made only in location of the pictures (random position with no overlapping pictures) as well as two tasks out of five had one new scene as all other were always the same.The additional, unseen scene was added to increase the motivation of the child, as all scenes for him or her are already seen and the new one will be interesting and exciting.All five situations were presented for child in the same order: 1. 6 pictures (all the same) of a starting scene of a wellknown animation movie.2. 6 pictures (all the same) of a starting scene of a wellknown animation movie in different screen locations.3. 5 pictures (all the same) of a starting scene of a wellknown animation movie in different screen locations.4. 4 pictures (all the same) and 1 new of starting scene of a well-known animation movie in different screen locations.5. 4 pictures (all the same) and 1 new of starting scene of a well-known animation movie in different screen locations.The choice to use multiple pictures was done to model situations when a user has a choice which object they want to pick, which way is more suitable for them to do a task they want etc. rather than executing automatic actions to look at one specific object.As the size of the pictures was constant we changed the number of pictures to force a change of distances from one picture to another.As well the number of 6 or 5 pictures was chosen to fill the screen with pictures, distancing each other approximately the same length as the pictures' width.This allowed us to make sure there will be enough space between each object.
All experiments were executed separately for each child to make sure there will be no distractions.As well the experiment room was locked to prevent distractions from outside.To make sure the child will be calm he or she had time to get to know the room and the equipment.Then step by step the EEG and eye gaze tracking systems were set to track child's brain activity and eye gaze movement.EEG was analysed using EPOC Emotive system where all types of brainwave were recorded for later analysis.Mirametrix eye tracking system was used to track eye gaze movements in order to find out what the child saw, where his eyes were looking at.The system logged monitored coordinates were the child was looking.By synchronizing in time and combining data of brainwave activity and eye gaze movement tracking we were able to identify the correlation between these two sources.
As well all experiments were filmed (both laboratory and working monitors, with all acting systems and executed actions) and summarized by independent observer.The task for independent observer was to log information what the child did, how he acted, etc.The data was useful to analyse possible causes of changes as well as correlation to the child's actions and brainwave activity as well as eye movement.
To ensure the research is compatible with ethics and law in Lithuanian in April, 2014 Lithuanian Bioethics Committee gave their consent to the study, which was done using special EEG, Eyes Motion tracker and Video hardware and software for 2-4 year old children.The Committee decided that the measures are compatible with research for children's rights and is not forbidden.

III. RESULTS OF THE EXPERIMENT
Synchronization of EEG signal data, eye movement tracking logs, experiment video records and paper notes of independent observer is required to get a full view of the research.It was done by using the same global time in all of these parts, therefore synchronization at millisecond level was achieved (except the observers data, it has approximately at 3 seconds accuracy).
In the first phase of result analysis the child's motivation level was observed by analysing Beta and Gamma brainwave activity.For each child's experiment a chart of brainwave activity changes in time was made and all times were marked when the eye movement to the object started.Moreover observer's notes were marked to identify additional child's activities during the experiment.
The data cannot be averaged for all 14 children, as there is a big variety in time when child decided to see the movie, what he or she does between different situations etc. Therefore example data of one child brainwave activity in our experiments is presented in Figure 2 and presents main tendencies, noticed in data of all children.
In Fig. 2 vertical lines with numbers from 1 to 5 identify the task execution start time.Usually after the task was given and before the task was done the child's beta and gamma brainwave activity increases and starts to decrease after the task was done.It shows that a child concentrates on the task.
Children think on what they should do, while after the task is done allow a loss of concentration.This increase of both beta and gamma brainwave activity matches the theory of focused attention in the literature [35].
As the research used 2 year-4 year old children, additional activities were noticed during the brainwave monitoring.As the example in Fig. 2 shows, the child after task 1 up to task 2 had more jumps in brainwave activity (area A).In this case it was a child's discussion why after choosing the movie was not working.Area B shows a situation when a child was distracted by other activities, usually a conversation with parents etc.In this case the beta brainwave activity decreases as a child knows he or she does not need to do any tasks, while concentration might increase.
Task 4 included a new scene.In this case all children showed an increase of gamma brainwave activity.Usually this activity does not decrease instantly (area C) as the child really wants to see a new series of the movie and starts interaction with surroundings on why it's not shown.After some time of the task 4 the gamma brainwave activity decreases as children usually lose hope to see the movie.So the task 5 increases both beta and gamma brainwave activity, however the focus is not as high as in task 4, despite the same new scene of the movie that was shown in the task again.
According to the results of brainwave activity analysis we classified all tasks for each child into no aimed (where gamma frequency increased less than 2 Hz) and aimed (where gamma frequency increased more than 2 Hz).The Non-aimed object represent starting movie scenes which were selected by children, however with not such an immense interest (just because they had to choose one of them).Meanwhile aimed objects were those scenes, which were selected by children with a big interest, revoked by themselves.Therefore usually only task 1 and task 4 was classified as aimed.
After all tasks were classified into aimed, no aimed and not executed (children refused to continue or did not execute the test as requested), the analysis of eye gaze movement was executed to find out how the motivation changes the eye gaze slope in Fitts' law.This was done analysing the eye gaze movement logs.As the log stored data of all eye movements in the screen, we were able to identify not only the beginning of eye movement but the concentration on certain object (end of the eye movement to a specific target) too.First of all eye jumps were classified into two categories: those who stopped on the area of object, picture; those who did not reach, overlapped with any object.Then according to eye gaze movement data for each jump which reached the movie scene picture the jump duration as well as distance to the object's centre and its width was determined.The data were enough to calculate the index of difficulty (2) and to get the slope value (b from ( 1)) as we assume there is no reaction time a.This assumption was made as specific time or event was not given for the child to start the task -children decided by themselves when they want to look at a certain movie scene.
One of the most obvious results we obtained according to these data -if the task is aimed children always reach the object within target's area.Therefore if a new scene of a well-known movie was noticed, the motivation level rises and eyes always go to the new object (regardless of whether it is at a greater distance comparing to other pictures).Meanwhile if the task is non-aimed, same child looks over a couple of targets, one after another and even 17 % of eye movement is close to the target, however they do not reach it.This shows the child might just overlook, not concentrating on one target.
To analyse eye gaze slope, according to Fitts' law we took all data of eye movement from any source area (blank area or another picture) to the target area (picture of movie scene).In total there were 8 aimed and 35 non-aimed targets (see Fig. 3).By applying all eye movement data of reached targets to Fitts' law calculated slope for each of them and found out that the average slope for non-aimed tasks is 1.07 (std.0.68) and for aimed tasks -0.21 (std.0.12).The difference of slope value for aimed and non-aimed tasks can be seen in Fig. 4. It showed that the eye movement in aimed tasks is approximately 5 times faster, comparing to non-aimed tasks.In our experiment all movie scenes were of the same dimension, therefore the index of difficulty mostly depends on the amplitude A (the width of the target might vary as well as the objects were rectangular, however the standard deviation of the width W is 14 % of radius a circle would have with the same area all rectangles had).Analysing the correlation between index of difficulty and task type (aimed or non-aimed) we do not want to raise any significant results as the position of movie scenes were randomized, however the minimum value of index of difficulty is greater in aimed tasks, comparing to non-aimed (see Fig. 4).It indicates the same tendency we noticed while analysing the filmed material of all experiments -if a child sees a new scene he is motivated to reach the exact object, despite the fact there is another one within a smaller distance.
Even though there are a deviations on eye movement speed, the line between aimed and no aimed tasks is clearly distinguished according to the slope value b.This allows raising a hypothesis that the motivation level can be calculated according to the eye movement slope.However it can depend on the situation and for some tasks, which are not as different as in our case, the line can be unclear for clustering only on eye movement, rather than both brain activity, eye movement and human supervision.

IV. CONCLUSIONS
As some researchers state [31] and our experiment shows with different tasks, the motivation level can be identified according to the beta and gamma brainwave activity.Experiments with children from 2 to 4 years show their brain activity is not constant and changes many times (in this experiment variation for beta brainwave were 11-16, for gamma -30-50), probably the influence of many different factors, not only the task itself.The fact eye movement slope is significantly different in aimed and unaimed tasks, shows we decided correctly to indicate motivated tasks if the gamma brainwave increase is more than 2 Hz.
The motivation level is a big factor for eye movement of 2 to 4 years old, as aimed objects are reached even up to 5 times faster by the eye comparing to unaimed tasks, where lack of motivation exists and overview method is sometimes used rather than straight look into one object.
For children of age 2 to 4 years new objects increase the motivation, however just temporary as the same object, shown second time, can be threaded as known an uninteresting for the child.Therefore the graphical user interface should always change in some manner to attract small children more.
Experiments with age of 2 to 4 years are complicated as only 54 % of tasks were executed properly.Therefore to get more accurate data in experiments with children of this age, the number of children should be increased or executed multiple times to get more data to analyse.

Fig. 2 .
Fig. 2. Example of one child's beta and gamma brainwave activity during experiment with 5 different situations.

Fig. 3 .
Fig. 3. Summary of successful and unsuccessful tasks with children in the experiment.

Fig. 4 .
Fig. 4. Slope dependency on tasks index of difficulty and motivation level.
Electroencephalography and Eye Gaze Movement Signals' Usage for Estimation of User Interface Usability Egle Radvile 1 , Antanas Cenys 1 , Simona Ramanauskaite 2 constants a and b are estimated from a particular experiment.Constant a is called the no informational part or reaction time.Constant b is the informational part in seconds per bit or a slope.There distance from start point to the centre of the target is called amplitude A while the tolerance of the jump is called width W (radius or width of the target).