Home

D3.1 User requirements report with observations of

1. All Devices 1 5 A a 10 4 3 S 2 3 p e uw 0 5 3 3 2 2 2 LL 2 et T T l T Hand Head Eye Head Eye Dwell Click Device Figure 6 6 Composition of device task quality 20 10 2005 Key 1 Incorrect commands 2 Target misses 3 Control corrections 42 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Usability variation in particular environments A method of improving the usability of the eye control modality was found by effectively removing the control corrections to the pointer position for eye control This was implemented by using a screen magnification system that temporarily increased target objects under the eye control pointer This is in effect slightly modifying the particular environment that eye control is using This zoom improvement thus greatly increased the efficiency of eye control to parity with head based control for both the dwell and click modalities even when the same facility was available with head based control Figure 6 7 Task Efficiency by Device All Devices 100 T T T 1 c 90 la moi l l loa b 8 amp 4 h l o l l l D l l l l l l l un 70 l l l l l 3
2. Potential eye control users Suggestions relating to their additional requirements of eye control technology other than those covered in the questionnaire included e Portability Portable and connected to a laptop so that the user wasn t tied to a PC indoors e Control of positioning Operation of an indoor static chair for changes in position Changing the position of a profiling bed e All aspects of daily living Professionals Suggestions relating to their additional requirements of eye control technology that they felt were not being provided at the moment included e Leisure Playing PC games e Arithmetic for children e Alarms and assistance devices Final additional comments Respondents were invited to make any further comments they wished relating to issues that had not been covered in the questionnaire Existing eye control users An existing user of an eye control system left no doubt as to its importance to them e would have no desire to live without this eye control system Potential eye control users Additional comments included e A request for information How do we find out about the suitability of different products Professionals Additional comments included e The need to overcome potential technical problems including conflicts between Windows and the eye control system software involuntary movement problems caused by natural and certain artificial 20
3. For example Stewart is able to control the movement of his electric wheelchair using head switches He uses his switches to move independently from communication and environmental control to wheelchair control and back again using a DX device positioned at the side of his Cameleon below Stewart s speech can only be understood by familiar listeners For this reason Stewart sometimes chooses to communicate using a speech output device a Cameleon This is in effect a tablet PC with speech output which is positioned on his wheelchair With a communication program called EZ Keys Stewart is able to express himself clearly even though because he uses a switch communication in this way does take time and effort for him Stewart also uses the Cameleon for writing using the EZ keys scanning keyboard with a predictor With this he can write directly into a word processor email application etc For environmental control Stewart also uses switches positioned in his headrest He is able to independently switch from communication writing on the Cameleon using the EZ Keys software 4 with that head control pod to try something amp Figure 7 12 Stewart using EZ Keys on his Cameleon PC based speech output device for communication Stuart was one of the users we involved in the brief user trials testing out an eye control application during a single session When Stuart used eye control for writing with the Tobii syst
4. e Can you imagine what it must be like to have something to say and no way to say it If you can communicate you can take part and have some control over your life e I would like to be able to participate in conversation spontaneously and be able to communicate with my children effectively Professionals Of the 5 professionals all 5 thought that the facility to communicate socially using eye control was very beneficial Their reasons for it being very beneficial included e Medically necessary to communicate physical needs 20 10 2005 104 106 Communication by Gaze Interaction COGAIN IST 2003 511598 e Communication is a fundamental need for everyone Additional requirements of eye control technology Existing eye control users The existing users of eye control systems are clearly very positive and enthusiastic about what they are able to achieve with their eye control systems overall The following comments were made in response to a question relating to any additional requirements they might have of eye control technology over and above they are achieving already e Environmental control This eye control user acknowledges that this could be done via eye control but it hasn t been purchased yet e Move from social communication application into eye control application myself e Play more video games turn the volume up and down e Tf anything I would like it to be more portable
5. Their shape size and the pressure required to activate them depends on the requirements and abilities of the user Some for example provide tactile or auditory feedback These are a few examples from a huge range of choices e Pillow Switch which is suitable for activating with the jaw e Floor Foot Switch e Mercury switch that is activated by tilting 5 degrees from horizontal position e Grasp Switch The last one is designed to be activated with a gentle squeeze or pinch The user must be capable of a well controlled grasping movement and able to release the grip within a short period of time One of the special facilities of the switch is that if it is held for over 2 seconds then the switch will latch or lock on If the user does not have a controlled grasping movement therefore then the Grasp Switch is not the appropriate switch for them Below is an example of a jaw activated switch Here movements of the jaw are used to activate the switch Fig 5 1 The switch also has a headset for auditory feedback Figure 5 1 J aw activated switch 1 2 earpieces 3 head set support 4 chin strap 5 adjustable securing means 6 oval chin cup 7 chin bar 8 fastening means 9 chin bar electrode 10 conductive wire 20 10 2005 26 106 Communication by Gaze Interaction COGAIN IST 2003 511598 An example of a dual switch is a Sip Puff switch that is activated by inhalation exhalation The Origin Instruments Corporatio
6. Figure 6 3 Often these tests only assess one type of control such as typing on an on screen keyboard that occurs in a real environment The data from these experiments is usually determined by the nature of the assessment task for example words per minute for a typing task but other metrics such as cursor paths eye scan paths or user subjective reaction to the test are often recorded giving a richer data set for usability assessment The rationale behind these potentially complex real world tests is that although frequently time consuming and laborious to conduct the true usability of a device cannot be known unless that device is actually tested on such a real world complex environment Hence these tools will be better suited to usability assessment than tools or techniques aimed at simply measuring device performance Complex interface kulta ja hopea ovat jalometalleja Eye scan path overlaid for analysis VALMIS kulta ja hopea ovat jafometalleja Figure 6 3 Example Real World typing testi Reviewing literature there appears to be no standard or commonly accepted test for assessing real world usability on an environment for any pointing device Typically real world evaluation is designed to test or assess a particular element of interaction with specific interest rather than the full range of interaction that is possible on an environment This is acceptable if the user
7. In offline analysis data from the most recent sessions are used to determine the best location and frequency band for cursor control for the next sessions Figure 5 8 Placement of electrodes in a cap for EEG measurement Millan 2003 The Cyberlink Interface is a commercially available system that enables hands free control of computers and electrical devices The Cyberlink system detects brain and body signals by the sensors in the headband Figure 5 9 The Cyberlink system detects brain and body signals by the sensors in the headband These are amplified digitized and transmitted to the computer in order to control a mouse or an interactive video game navigate a business application use a web browser control almost any Windows application play musical synthesizers activate peripheral devices adjust environmental controls etc It has been the experience of the ACE Centre that not all users experience success with system and those who do would find it hard to match the speed of an effective eye control system 5 2 Information from stakeholders In the questionnaire potential eye control users were asked how they felt eye control would compare with their existing access method in general Where an opinion was expressed 6 out of 8 they felt that eye control would be better or much better Their reasons given included e His eye pointing is very good so I would hope that it would be easier and quicker e Les
8. Low cost eye tracking has been investigated for some time by for example Hansen et al 2002 and Corno et al 2002 Several students are picking up the challenge to design a low cost tracker see e g Kreutz 2003 With the rapid improvement in tracking hardware video boards digital camera and CPU s a break through may happen within the next few years even without high investments in engineering efforts Second area of possible improvement Reliability and robustness of tracking According to Jakob and Karn 2004 10 to 20 of all subjects cannot be tracked reliably for continuous interaction While this may differ from system to system the robustness of most systems leaves much to be desired they claim Detection of objects can be a difficult task to perform and in particular detection of the human eye is difficult as the contrast between eye and skin may be poor The appearance of the pupil is heavily influenced by occlusions from the eyelids and will be totally covered for every eye blink The effects of occlusion and illumination changes are also related to the ethnic origin of the user Both Caucasian and Asian people have facial features that may make iris tracking difficult The eyelids of Asians are generally close together and may thus result in less boundary information On the other hand the nasal bone and superciliary arch of Caucasians are usually more pronounced and therefore casts more shadows on the eye In addition the
9. Pennsylvania United States Salvucci D D and Anderson J R 2000 Intelligent gaze added interfaces Proceedings of CHI 2000 2 ACM Press pp 273 280 Shaw R Crisman E Loomis A and Laszewski Z 1990 The Eye Wink Control Interface Using The Computer To Provide The Severely Disabled With Increased Flexibility And Comfort Third Annual IEEE Symposium on Computer Based Medical Systems Chapel Hill NC USA 1990 Shein G F 1997 Towards Task Transparency in Alternative Computer Access Selection of Text Through Switch Based Scanning Ph D Thesis Dept of Industrial Engineering University of Toronto Shein G F Treviranus J Brownlow N D Milner M and Parnes P 1992 An overview of human computer interaction techniques for people with physical disabilities Int Journal of Industrial Ergonomics vol 9 issue 2 February 1992 pp 171 181 Sibert L E and Jacob R J K 2000 Evaluation of eye gaze interaction Proceedings of CHI 2000 2 ACM Press pp 281 288 Smith W J 1996 ISO and ANSI Ergonomic Standards for Computer Products A Guide to Implementation and Compliance Prentice Hall New York Spaepen A Edwards K Verheust M Van Mele I Wauters F Schampheleer W Ketelaer P Jacobs R Hendrickx E and van Steenberghe D 1996 Access design for all a study on learning curves during training on devices for textual input to telecommunications equipment for the motor impaired CCHP 96
10. distance between the two eyes also varies across ethnic backgrounds Once the eye is located the iris is a prominent and reliable feature within the eye region because of its high contrast Eye tracking often includes information about the history of previous positions whether the eye is open or closed reflection properties and other information describing the appearance of the eye In particular information about previous positions may save computations and reduce the amount of false detections In addition although the appearance of the eye of a single user may change dramatically over time tracking a known person s eye with known dynamics is expected to be more reliable and fast than tracking someone with unknown characteristics Prior information of this kind could for example be obtained by collecting sufficient statistics to learn the possible appearances and the dynamics over time for a particular user Blake and Isard 1998 Methods used for eye detection and tracking rely on different assumptions on the image data and in general two classes of approaches exist One common and effective class of eye tracker exploits active illumination from infrared IR light emitters Through novel synchronization schemes and by using the reflective properties of the pupil when exposed to near infrared light dark or bright pupil effects the eye can be detected and tracked effectively and reliably In addition to controlling the light conditions IR also
11. double vision Diplopia may be the result of an injury to the brain or a side effect of many commonly prescribed medications and may make it difficult for the user to fix his gaze on a given point see strabismus C Blurred vision Another occurrence associated with some brain injuries as well as a side effect of medications see below Blurred vision of screen elements decreases the accuracy of eye fixations D Cataracts clouding of the lens of the eye If a cataract has formed on the portion of the lens that covers the pupil it may prevent light from passing through the pupil to reflect off the retina Without a good retinal reflection eye tracking systems based on the bright pupil method cannot accurately predict the user s eye fixations The clouded lens may also make it difficult for a user to see text on the screen clearly Surgical removal of the cataracts will normally solve the problem and make the use of eye tracking possible E Homonymous hemianopsia blindness or defective vision in the right or left halves of the visual fields of both eyes This may make calibration almost impossible if the user cannot see calibration points on one side of the screen 20 10 2005 18 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Ability to maintain a position in front of the Eyegaze monitor It is generally easiest to run an eye tracking system from an upright seated position with the head centred in front of the Eyegaz
12. for head and 28 for gaze In this experiment subjective scale ratings of the efficiency found the users to be most pleased with the hand mouse input The satisfaction with gaze input was lower than for mouse and head while the efficiency ratings of head and gaze were almost identical What improvements need to be made to existing eye control technology and applications First area of possible improvement Reducing the Price Eye tracking systems available for people with special needs cost more than some people can afford often more than 5000 and authorities that would normally supply citizens with communication aids may be reluctant to invest this high amount of money if the final effectiveness is uncertain In a recent paper Jakob and Karn 2004 explained why eye trackers are so costly The Eye tracker industry only sell tens or hundreds of systems in a year which makes it difficult to invest in the large engineering effort that would be required to develop a really good inexpensive unit But without such a unit the market will continue to be limited to tens or hundreds per year a chicken and egg problem 20 10 2005 62 106 g COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 Therefore the first area of improvement is definitely to increase the market for gaze trackers by improving them and by informing the authorities about their potential This is one of the main objectives of COGAIN
13. 1 Possible eye control efficiency metrics Bevan et al 1995 Satisfaction assessment areas and factors Area Workload Comfort Ease of use Physical effort Headache Accuracy of pointing Mental effort Eye discomfort Speed of pointing Factors Time pressure Facial discomfort Accuracy of selection Frustration Mouth discomfort Speed of selection Performance Neck discomfort Ease of system control Table 6 1 Possible eye control satisfaction metrics Bates 2005 Thesis ISO 1998 Smith 1996 Douglas etal 1999 This idea of usability is well suited to examining the use of eye controlled pointing devices by motor impaired users of computer based systems Specified users can refer to the level and type of motor impairment of the users Specified goals can refer to activities associated with the operational aims of the user for example using common software such as word processing web usage email as well as operation of the equipment 20 10 2005 37 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction associated with the pointing device itself such as calibration The particular environment refers to the physical environment in which the system will be used as well as the support and social context available for use the former includes use of wheelchairs or beds while the latter includes the provision of human helpers These metrics all help to set and define the scenarios in w
14. 10 2005 105 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction lighting robustness snagging cables difficulties in the setting up and calibration process the need for gradual implementation to avoid eye strain training and practice for the user Its not a natural way of looking e A need for the user to be able to calibrate and re calibrate independently as and when the user wishes 20 10 2005 106 106
15. 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Figure 7 9 QualiCHESS application Importantly many PC computer games can be played using eye control without modification if the user and their system are sufficiently accurate In summary people with disabilities want to be able to communicate play games be independent have mobility communicate socially whether in the home the office on the move etc like everyone else The range of access technology currently available does not always enable all users with disabilities to realise these objectives COGAIN provides an excellent opportunity to discover the extent to which eye control technology can reduce the shortfall 7 2 Information from stakeholders The questionnaire returns provided an indication of the range of software and applications that can be used both with eye control and with other access technology The eye control users who responded use their systems for Word processing social communication e g Speaking Dynamically Pro emailing Internet Instant messaging games The users of other access methods who responded use their systems for Word processing 1 Dasher user social communication e g EZ Keys emailing Internet Instant messaging games activating toys powered mobility environmental control The non eye control users who found their current access methods very difficult felt that use eye control would enable them to e Ac
16. 3 3 Eyeglasses may person is able bodied and has normal vision Indeed Goldberg and cause reflections that prevent Wichansky 2003 estimate that up to 20 of randomly selected subjects fail eye tracking to get a good calibration Eye tracker manufacturers typically report higher success rates Lower success rates in real life conditions may be caused by ambient light or reflections from the environment The user may not be sitting still enough for the system to be able to track him her Most systems also have an optimal distance and location concerning the positioning of the person in from of the eye tracking device The problems are exacerbated when people who have severe involuntary head or eye movements try to use the systems The person may also be lying on his or her back or sit in a divergent position e g her head rotated sideways The current eye tracking systems have some ways of trying to cope with these problems Most manufacturers provide accessories for positioning the eye tracking device and the computer screen so that the distance and the angle of the tracker are adjustable e g by using an adjustable arm If the camera loses the eye some of the systems automatically start searching for it For example the Metrovision VISIOBOARD system s camera automatically moves and scans the surroundings trying to relocate the lost eye Whilst this can be very effective to re locate the eye of a user who has moved away from the V
17. 6 Low 1 12 3 4 5 12 345 12 3 4 Workload Comfort Ease of Use Device Questionnaire Category a Communication by Gaze Interaction Key Hand Head dwell Eye dwell Head click Eye click AUN Notes Data sets with the same letter are not significantly different p gt 0 05 Figure 6 5 Device satisfaction Low Workload and High Comfort and Ease of Use for highest Satisfaction The usefulness of this usability approach was that it now allowed close examination of the individual elements that constitute the objective and subjective metrics Examining the quality metric in particular Figure 6 6 showed that making control corrections to the pointer position Factor 3 Figure 6 6 particularly when under eye control accounted for a large proportion of the non productive time in the trials and was the most influential factor on eye modality usability Not surprisingly this proportion increased for the smaller interface targets If this could be reduced then the eye modality usability might be improve significantly Examining the individual satisfaction ratings Figure 6 5 showed that the eye modality required more workload than the head and hand modalities and was less comfortable to use The eye modality in dwell selection mode had low ease of use but the eye modality in click selection mode was as easy to use as the head modality Quality Components by Device
18. ANONYMOUS AND WILL BE KEPT ON OUR OWN RECORDS ONLY 20 10 2005 85 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Please choose the statement that best describes you A I have a disability and am have been a regular user of an eye control system or a helper on your behalf B I have a disability and do not use an eye control system or have had only a little experience with one or a helper on your behalf C I am a parent professional e g teacher carer health professional assistive technology specialist who supports a person people who have a disability who I feel might benefit from eye control technology Please Note We are keen to hear the views of both disabled users and those professionals carers who support them For this reason if you are a professional carer who supports an individual with a disability we would welcome a response from both your own perspective Section C Page 11 as well as on behalf of the user Section A Page 3 if they require your support in doing this 20 10 2005 86 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Section A Interaction I have a disability and am have been a regular user of an eye control system Name Email Address Telephone Number Organisation Company if applicable Ala Question Why do you need or choose to use eye control to access the computer E g due to Motor Neurone D
19. Bayesian Methods 2004 MacKenzie LS 1991 Fitts law as a performance model in human computer interaction Doctoral dissertation University of Toronto MacKenzie LS 1992 Fitts law as a research and design tool in human computer interaction Human Computer Interaction 7 pp 91 139 20 10 2005 81 106 Communication by Gaze Interaction COGAIN IST 2003 511598 MacKenzie LS and Buxton W 1992 Extending Fitts Law to Two Dimensional Tasks Proceedings of CHI 1992 MacKenzie I S and Soukoreff R W 2002 Text entry for mobile computing Models and methods theory and practice Human Computer Interaction 17 pp 147 198 Majaranta P Aula A and R ih K J 2004 Effects of feedback on eye typing with a short dwell time Proceedings of the Eye tracking research amp applications symposium on Eye tracking research amp applications San Antonio Texas pp 139 146 Majaranta P MacKenzie I S A Aula and R ih K J 2003a Auditory and visual feedback during eye typing Extended Abstracts of the ACM Conference on Human Factors in Computing Systems CHI 2003 New York ACM pp 766 767 Majaranta P MacKenzie I S and R ih K J 2003b Using motion to guide the focus of gaze during eye typing Abstracts of ECEMI2 12th European Conference on Eye Movements Dundee Scotland August 2003 Majaranta P and R ih K J 2002 Twenty Years of Eye Typing Systems and Design Issues Proceedings of the
20. Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction Allb Al2a Please give the reason s why you think that eye typing is would be either very beneficial quite beneficial not beneficial Whether or not you use the computer for emailing how beneficial do you think this is would be for you i e please answer either very beneficial quite beneficial not beneficial A12b Please give the reason s why you think that emailing is would be either very beneficial quite beneficial not beneficial Al3a Whether or not you use the computer for Internet access how beneficial do you think_this is would be for you i e please answer either very beneficial quite beneficial not beneficial A13b Alda Please give the reason s why you think that Internet access is would be either very beneficial quite beneficial not beneficial Whether or not you are able to control a wheelchair with eye control how beneficial do you think this would be to you i e please answer either very beneficial quite beneficial not beneficial Al4b Please give the reason s why you think that wheelchair control would be either very beneficial quite beneficial not beneficial Al5a Whether or not you are able to control your environment e g TV Radio Telephone etc with eye control how beneficial do you think this is would be to you i e please answer either very bene
21. Proceedings of the 5th International conference on Computers helping people with special needs Part I pp 277 282 Stampe D M and Reingold E M 1995 Selection by Looking A Novel Computer Interface and its Application to Psychological Research In Findlay J M Kentridge R W and Walker R Eds Eye Movement research Mechanisms processes and applications Elsevier Tobii 2003 User Manual Tobii eye tracker and ClearView analysis software Tobii Technology AB December 2003 Velichkovsky B M Pomplun M and Rieser H 1996 Attention and communication Eye movement based research paradigms In Zangemeister W H Stiel S and Freksa C Eds Visual attention and cognition Amsterdam New York Elsevier Science Publisher Ward D J and MacKay D J C 2002 Fast hands free writing by gaze direction Nature vol 418 number 6900 p 838 20 10 2005 83 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Wiklund M E Dumas J S and Hoffman L R 1987 Optimizing a portable terminal keyboard for combined one handed and two handed use Proceedings of the Human Factors Society 31st Annual Meeting Santa Monica CA Human Factors Society pp 585 5809 Zhai S Hunter M and Smith B A 2000 The Metropolis keyboard An exploration of quantitative techniques for virtual keyboard design Proceedings of the UIST 2000 Symposium on User Interface Software and Technology Zhu Z Fujimura K
22. Symposium on ETRA 2002 Eye Tracking Research amp Applications Symposium 2002 New Orleans Louisiana pp 15 22 Matias E MacKenzie I S and Buxton W 1993 Half QWERTY A one handed keyboard facilitating skill transfer from QWERTY Proceedings of the ACM Conference on Human Factors in Computing Systems INTERCHI 93 New York ACM pp 88 94 Matsumoto Y Ino T and Ogasawara T 2001 Development of Intelligent Wheelchair System with Face and Gaze Based Interface Proceedings of 10th IEEE Int Workshop on Robot and Human Communication ROMAN 2001 pp 262 267 Millan J R 2003 Adaptive Brain Interfaces Communications of the ACM vol 46 no 3 pp 75 80 Miniotas D Spakov O and Evreinov G 2003 Symbol Creator an alternative Eye based Text Entry Technique with Low Demand for Screen Space Proceedings of INTERACT 2003 September 2003 pp 137 143 Miyoshi T and Murata A 2001 Input Device Using Eye Tracker in Human Computer Interaction JEEE International Workshop on Robot and Human Interactive Communication pp 580 585 Murata A 1991 An experimental evaluation of mouse trackball joystick joycard lightpen trackball and touchscreen for pointing basic study on human interface design In Bullinger H J ed Human aspects in computing design and use of interactive systems and work with terminals Elsevier Nguyen K Wagner C Koons D and Flickner M 2002 Differences in the infrared bright pupi
23. UTA Isabella Signorile POLITO Revision History Date Author s Comments 12 3 2005 P ivi Majararta UTA Replaced Figure 9 5 Lisa Oosthuizen ACE and Paivi 20 6 2005 Majararta UTA Replaced Figure 9 8 and updated text related to the figure 30 8 2005 Henna Heikkila UTA General cheking for consistency referencing spell checking etc 20 10 2005 2 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Table of Contents 1 EXECUTIVE SUMMARY usccsssssssssssesssssscesssensssessseesaeeraceesnessaeesueesseeseseseseenaceeaceesaeesaeesueeeseeesceeeaeesaceeeaeenaceeaieesaeesaeeeaeeseseenieenass 5 End users eye control hardware requirements 1 2 0 1 retreat reir cnn e ener ri db rc 5 End Users eye control software fequlr merrts s pce rtt bert dee tr hee dde apetece c 6 2 INTRODUCTION WHAT IS THE PURPOSE OF THIS USER REQUIREMENTS DOCUMENT nnne 7 2 1 Putting the Useratthe Cerlfe serioses eee nea irino Eee e arri teg eR 1 2 2 Data collection methods and background information sese nnnnnennnnnnnnnes 9 2 9 SUMMANY silts actae neca excu A ri ca ru rum dr od cd re ap e re n rer En Hr Dg 10 3 WHOIS CURRENTLY ABLE TO USE EYE CONTROL TECHNOLOGY sseeeeeeeen inneren nre 11 3 1 Whatthe literatur amp Says icc rece tre te e ehe terrm a n in an dened no d i 11 3 2 Information from stakeholders 3 cioe peterent den rrr tertie re pe d n o
24. VisionKey system uses a EEEN JLILJE sequential scanning and requires only coarse vertical eye movements mH a for selection Kahn et al 1999 SESS It must be noted however that in practice scanning can be Figure 3 6 Sequential row column considerably slower and therefore considerably more frustrating than Scanning Shein 1997 direct eye control for the user so that given a choice it is likely that 20 10 2005 13 106 Communication by Gaze Interaction COGAIN IST 2003 511598 the vast majority of users would choose eye control as opposed to eye blink Most eye tracker manufacturers report that the spatial accuracy of the system is about from 0 5 to 1 degree Even if the calibration succeeds and the measured point of gaze is fairly accurate right after the calibration the tracking may fail after a while because of the deterioration of a calibration that occurs over time the calibration drifts The drift may be caused by the change of pupil size the eyes may become dry or a considerable change of the angle position and distance of the person s eyes in relation to the eye tracking device Systems that track both eyes cope with the drift better because most drift effects are inversely symmetrical between the eyes Tobii 2003 There are also algorithms that can dynamically correct the drift Stampe and Reingold 1995 The accuracy of the calibration is checked on every successful selection and can then b
25. a high jaw fatigue reported by this group especially in the beginning of the experiment Saepan et al 1996 concluded the power of gaze interaction to be the minimal motor ability required In a recent experiment Hansen et al 2004 compared the speed of text production of 12 Danish and 15 Japanese non disabled subjects They were writing either by hand controlled mouse head tracking the NaturalPoint system and gaze tracking Quick Glance system on a Danish and Japanese on screen keyboard with very large buttons only 12 buttons on a full screen The hand controlled mouse was the fastest input on day two for both interfaces 7 5 wpm and 16 1 cpm respectively whereas the head input yielded 6 1 wpm and 12 3 cpm respectively on day two Gaze input was found to be 6 3 wpm and 11 4 cpm respectively on the second day The difference between head and gaze input was not significant in any of the experiments by Hansen et al 2004 Accuracy was measured by the number of sentences that either were erroneous or had been corrected by the subjects in percentages of all sentences typed Errors were very rare in the Japanese experiment for all three input devices Only 3 of the sentences typed by hand mouse were erroneous or had been corrected whereas 5 of the head typed sentences and 6 of the gaze typed sentences had errors or corrections The Danish subjects were much less accurate They made errors in 14 of the sentences for hand mouse 14
26. acquire the target move the scene right or left all with your eyes Or place the bet pull the handle or deal yourself three cards The door is just opening on the potential for using eye tracking technology in a broad range of gaming venues http www eyegaze com SOLUTIONS htm video confirmed 2005 02 22 The two computer games that they include in their system are Paddle games and Score four Other games that work well with head tracking systems are also likely to work with gaze control systems A Game Accessibility Special Interest Group works under The International Game Developers Association IGDA http www igda org accessibility confirmed 2005 02 22 One of their aims is to improve hardware support for miscellaneous special devices that may be used as an alternative to joystick control A gaze controller would be an obvious candidate to work within this group One representative from a US company Brain Actuated Technologies that manufactures an EEG EMG EOG control device is included in the special interest group and COGAIN might consider joining it as well The head mouse producer Natural Point runs a forum for discussions of headmouse gaming http forums naturalpoint com cgi bin ultimatebb cgi They also have a rather long list of games that they recommend for headmouse interaction including flight simulators and racing games When investigating games that would work well with gaze control it may b
27. alterations in operational characteristics environments and arrangements for users that optimise the usability of eye control 6 2 Information from stakeholders As described above Usability is generally defined as the effectiveness efficiency and satisfaction with which specified users can achieve specified goals in particular environments ISO DIS 9241 11 Two case studies follow where users are making clear choices and decisions in relation to usability issues Case study Sarah Sarah has a high spinal lesion that for her means that she cannot move her body below the neck and has to rely on a ventilator to breathe She has very good head control however and is able to use a mouthstick very effectively for a range of activities that she enjoys 20 10 2005 44 106 Communication by Gaze Interaction COGAIN IST 2003 511598 For example using her mouthstick Sarah is able to make friendship bracelets by weaving them and she can also use a mobile phone to send text messages She can also access the computer using the mouth stick She uses the mouth stick to push the mouse around the mat with great accuracy accurately enough indeed to be able to design websites a skill she recently acquired during a specialised two year training course For Sarah this method of using the mouth stick to control the mouse keyboard gives her the high level of accuracy she requires for web design However this method requires a great de
28. be useful to be able to do each of the things you d like them to C7a Do es he she they write word process with his her their access method If Yes please answer C7b and C7c below If No please explain why not 20 10 2005 97 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction C7b If you answered Yes to C7a Which software do es he she they use C7c If you answered Yes to A7a About how many words per minute can he she they type with his her their access method If more than one person please give examples C8 If he she they were able to use eye control instead of their current access method s how do you think it would compare Please choose Not as good better much better C9 If you think it would be better much better than their current access method why do you think this i e Is it quicker Less effort Etc C10 Clla If you think it would be not as good why do you think this e g difficult to use out of doors etc If he she they were able to use the computer for writing using eye control sometimes called eye typing how beneficial do you think_this would be i e please answer either very beneficial quite beneficial not beneficial Clib Please give the reason s why you think that writing using eye control sometimes called eye typing would be either very benefic
29. bed Some gaze tracking systems therefore offer a lightweight laptop model e g Quick Glance LC Technologies and Tobii However even with the size and weight of laptop computers decreasing the system still has to be positioned in front of the user Consequently it occupies some space and may not be convenient e g at a dinner table Some people who use a text to type system have full mobility They are definitely more obstructed by the constant need to have a computer monitor in front of them than they would be if the system were completely wearable Recently GazeTalk software has been used with a low weight VGA monitor that can be clipped on to a normal pair of glasses The monitor model CO 3 from Microoptical has 640 x 480 pixel resolution and 24 bit colour 60 HZ see Figure 8 3 Field of view is approximately 16 degrees horizontally 20 10 2005 65 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Figure 8 3 A researcher from ITU typing with GazeTalk by mouse on a micro display Typing speed was 12 8 wp Average typing speed with a hand controlled mouse across 90 sentences yielded 12 8 wpm and 8 6 wpm for two medium trained subjects This performance is compatible with the one found among experimental subjects who had a full size monitor available in fact one of the micro typing subjects was just as fast as the fastest subject in the experiment reported by Hansen et al 2004 This was regarded as a v
30. c 60 4 l llo T ES I Nu l l 50 4 i i o i 4 l l l d 4 l l E 30 i Notes l l l a F 99 l l Data sets with the 1 i i l same letter are 0 jy i i l i not significantly 0 _ l l different p gt 0 05 Hand Head Eye Head Eye Head Eye Head Eye Dwell Olick Dwell Click v Normal e Zoom Device Enhancement Figure 6 7 Modified device efficiency Hence Figure 6 7 the objective usability of eye control was improved to equal head based objective usability This benefit was also shown for subjective usability Table 6 2 with the satisfaction rating for both head and eye modalities increased due to the improvement although the eye modality was still less satisfying than the head and hand modalities Cost benefit of enhancement and subjective device satisfaction Satisfaction per device Change due to zoom 7 rati enhancement Device 1 7 rating Standard Enhanced devices devices 1 7 rating o Hand 6 20 Head Dwell 4 36 4 10 0 34 8 Eye Dwell 2 93 3 47 0 54 18 Head Click 4 13 4 83 0 10 2 Eye Click 3 90 4 50 0 60 15 Table 6 2 Modified device satisfaction 20 10 2005 43 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Usability variation in specified users A second change to eye control was now attempted by changing the specified users of eye based
31. character twice Experienced users of Erica an early gaze typing system developed by Hutchinson et al 1989 could produce a full page in 85 minutes which is approximately 6 wpm assuming a page contains 2400 characters Spaepen et al 1996 found performance to be approximately 7 wpm and Stampe and Reingold 1995 obtained similar results on their gaze typing system Speed is not the only objective to consider when designing a user friendly gaze communication system Hansen et al 2001 referred to additional user requirements for a system to be satisfying The system should be easy to install maintain and update It should consist of standard consumer hardware components that can be replaced immediately when something breaks down Calibrations should be performed easily and quickly Tracking should be sufficiently robust to allow for mobile use with occasional changes in light conditions use of glasses and minor changes in head position Prolonged use should not cause fatigue or cause the eyes to dry out Ideally the price of the system should not be prohibitively high Finally the system should not make the disabled person look awkward For instance members of an ALS user group have told us that they would 20 10 2005 61 106 Communication by Gaze Interaction COGAIN IST 2003 511598 prefer not to wear any kind of peculiar equipment on their head and that the tracking hardware should either look familiar or be invisible When
32. comparing gaze interaction to other access methods used for selection by pointing several issues should be considered First of all gaze may be the only viable interaction form for some users e g in a full locked in situation Secondly gaze interaction may be one of several possible input methods used during the day to ease the burden of muscles in the neck that are used for head based interaction to example Finally comparisons of gaze interaction with other forms have often been done with previous generations of tracking technology that have now improved considerably For this reason the comparisons cited below should not be considered as representative of performance potentials of future systems Spaepen et al 1996 compared gaze dwell time typing to a chin operated trackerball After 90 minutes of training gaze typing was a bit faster than trackerball typing 35 characters per minute as against 30 characters per minute similar to 7 wpm and 6 wpm respectively for able bodied students Subjects with multiple sclerosis achieved typing speeds of 5 wpm and 2 wpm respectively Error rates were higher for gaze interaction compared to chin 2 4 versus 1 0 data only provided for able bodied subjects Saepan et al 1996 proposed that the higher speed with the gaze typing system might have encouraged the subjects to proceed at the expense of accuracy The low typing speed for the disabled group using a chin track ball was presumably related to
33. control using eye control As described earlier facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to control their environment Eye control users Of the 3 eye control users one did not comment Another who already uses eye control for their environment feels it is beneficial The other who does not have eye control over their environment feels it would be very beneficial Their reasons included the following e Freedom e Im doing things on my own 20 10 2005 103 106 q COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 Potential eye control users Of the 8 users of other forms of access technology 4 thought it would be very beneficial 2 not benefical one was able to use standard technology already and the other did not comment Reasons for considering it to be beneficial included e Increased independence privacy e He has more control over his eyes than his hands so it is likely to be easier quicker and more accurate It would increase the options available to him The reservations related to e potential difficulties with reliability and reduced portability e Tt would be a lot easier but reliability would be a big concern My independence is very important and I simply would not trust it My environmental control system is extremely portable I cannot see an eyegaze system ever being as portable
34. courtesy of Dixon and Nancy Cleveland 20 10 2005 17 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction Nystagmus This is a constant involuntary horizontal vertical or rotary movement of the eyeballs It may lead to the inability of users to fix their gaze long enough to make selections The problem here is that simply reducing the fixation dwell time threshold can increase the risk of involuntarily making selections see Midas Touch problem below Strabismus This visual difficulty also called heterotropia or tropia results in an inability to attain or maintain binocular vision due to some type of muscle imbalance in one eye causing that eye to turn in out up or down relative to the other eye Strabismus can be intermittent occurring sometimes constant occurring all the time and or alternating occurring sometimes with one eye and sometimes with the other eye whether intermittently or constantly The problem for eye tracking arising from alternating strabismus is that for monocular systems it is impossible to make sure that the tracked eye is the one not deviating from the line of sight One solution to this problem is putting a partial eye patch over the nasal side of the eye not being observed by the camera that often solves this tracking problem Since only the un patched eye can see the screen it will continuously focus on the screen By applying
35. enjoy keeping in touch with friends and family especially when she is staying overnight at school The benefits and perceived potential benefits of access to the Internet using eye control As described earlier facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to access the Internet Eye control users 2 of the 3 eye control users use their systems to access the Internet One feels it is beneficial the other very beneficial Their reasons included the following e Its my way of keeping up with what s going on in the outside World Potential eye control users Of the 8 users of other forms of access technology 4 thought it would be very beneficial 1 quite beneficial 1 not very beneficial 1 not benefical and 1 did not comment Reasons for considering it to be beneficial included e accessing the news taking up hobbies news groups chat groups and distance learning 20 10 2005 102 106 Communication by Gaze Interaction COGAIN IST 2003 511598 The one reservation was from a headmouse user who thought that eye control might be slower for Internet access Professionals Of the 5 professionals one thought that the facility to access to the Internet using eye control was quite beneficial and the other 4 thought it would be very beneficial Their reasons included e on line banking shopping accessing information sp
36. expression as a function of manual tasks Proceedings of the International Conference on Rehabilitation Engineering RESNA pp 83 87 Frey L A White K P Jr and Hutchinson T E 1990 Eye Gaze Word Processing IEEE Transactions on Systems Man and Cybernetics 20 4 pp 944 950 Gips J Olivieri C P and Tecce J J 1993 Direct control of the computer through electrodes placed around the eyes In Smith M J and Salvendy G eds Human Computer Interaction Applications and Case Studies Elsevier Amsterdam pp 630 635 Goldberg J H and Wichansky A M 2003 Eye tracking in usability evaluation A practitioner s guide In Hy ni J Radach R and Deubel H eds The Mind s Eyes Cognitive and Applied Aspects of Eye Movements Oxford Elsevier Science Grossens C A and Crain S S 1987 Overview of Nonelectronic Eye Gaze Communication Techniques AAC Augmentative and Alternative Communication 3 2 77 89 Hansen D W Hansen J P Nielsen M Johansen A S and Stegmann M B 2002 Eye Typing using Markov and Active Appearance Models IEEE Workshop on Applications on Computer Vision pp 132 136 Hansen D W and Pece A E C 2005 Eye tracking in the wild Computer Vision and Image Understanding 98 1 April 2005 pp 155 181 20 10 2005 79 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Hansen J P Johansen A S Hansen D W Itoh K and Mashino S 2003a Language Techno
37. formal carers family members service providers whose needs must also be taken into account User characteristics will affect users abilities to use a system and will influence the form that a product or system takes e g nystagmus or involuntary head move Design that takes as its starting point the needs of the user rather than something that is bolted on later The process of gathering information through User Analysis and Activity Analysis What is known about the characteristics of users The activities that users need to perform The process by which text is produced using eye control A system with which a user can independently move from one from of assistive technology to another independently e g between the following assistive technology environmental control social communication writing wheelchair control Software that enables an end user to access the computer with more efficiency effectiveness or satisfaction 77 106 Communication by Gaze Interaction COGAIN IST 2003 511598 11 References AbleNet Inc homepage Available at http www ablenetinc com Accot J and Zhai S 1997 Beyond Fitts Law Models for Trajectory Based HCI Tasks Proceedings of CHI 97 pp 295 302 Barea R Boquete L Mazo M and Lopez E 2002 System for assisted mobility using eye movements based on electro oculography IEEE Trans Neural Syst Rehab Eng 2002 Dec 10 4 pp 209 18 Bates R 1999
38. hence could not allow comparison between devices and modalities and differing tasks unless the tasks were identical However the typing rate was common to papers assessing keyboards and this could be used to compare differing eye based pointing devices if the same keyboards and text entry tasks were used or if the same device was used and differing keyboards assessed for their efficacy However this is limited to assessing a single task type not the diverse range of tasks a user may wish to perform in everyday life The objective usability metrics found are often supported by subjective metrics of user reaction to the modality or device This is essential as to better gain a full understanding of the usability of a modality it is regarded as not adequate to simply measure the objective performance of a modality without also assessing the subjective reaction of the user when using the modality Bevan et al 1991 1995 Perhaps a modality performed well objectively with low task times and error rates but the user worked hard to control the system or the modality was uncomfortable to use Would this modality be more suitable to the task than a modality that objectively performed less well but required less work from the user or was more comfortable to use It is suggested from this study that perhaps what users require is a method of assessing the usability of modalities in a more standardised objective and subjective way with a common measureme
39. if they cannot independently move about within their whole environment However good a social communication program they have there are limited benefits if they can only communicate in a limited number of locations 20 10 2005 56 106 Communication by Gaze Interaction COGAIN IST 2003 511598 8 What can people with disabilities currently achieve using eye control technology specifically and how well 8 1 What the literature says What according to the literature are users currently able to achieve successfully with eye control The literature on eye gaze control has reported on several areas in which this control mode has been used type to talk environmental control and leisure activities Non electronic gaze communication The most common use of eye control is for type to talk systems Actually gaze typing has been possible for more than 20 years see Majaranta and R ih 2002 for an overview AAC Gaze communication without computers has an even longer history Users would gaze at an alphabet printed on a board see Grossens and Crain 1987 for an overview Using an e tran frame characters or symbols can be displayed on a transparent sheet of plastic in a matrix and the communication partner holds the sheet in front of the user By maintaining eye contact through the sheet it is easier for the communication partner to estimate at which character the user is looking Drinker and Kropoff 1981 In the 1980 s a non elect
40. important therefore that those who are involved with it have a clear understanding of where it stands in relation to the very wide range of hardware and software available to the disabled community Without such an understanding there is a danger of going to all the trouble and expense of implementing the use of Eye Control Technology with a disabled person when perhaps a cheaper simpler more reliable or indeed more effective control method could be provided It is not within the remit of this document to provide information on every single access device every single piece of Access Technology and every single piece of access software These amount to many thousands However it is important that COGAIN partners are aware of the categories and purposes of access methods and devices that might provide alternatives to Eye Control Technology An overview of the range of alternatives to Eye Control Technology is now provided in order to provide a clearer understanding of the issues involved in considering the potential benefits and limitations of Eye Control Technology in relation to other types of Access Technology 5 1 What the literature says There are many alternative ways in which assistive technology devices can support people with disabilities Most of these require a controlled physical movement Some people might use a part of their body such as a fist finger or foot Some might use the blink of an eye or head movement Obviously this needs to b
41. into what can currently be achieved and what might be achieved in the future Contact has already been established with a very severely disabled eye control user called Keith The information acquired from him provides a valuable insight into the impact of eye control technology on his quality of life Case Study Keith Keith has ALS He is completely paralysed and is only able to move his eyes He can no longer move his head at all He cannot blink This means that he experiences great difficulty and discomfort and it is necessary to keep clearing his eyes in order to prevent a film building up on them He has been using an LC technologies Eyegaze system for 2 years and he controls it from an upright position in his wheelchair He uses the system for about 12 hours daily He runs his laptop through the LC system He would also like to be able to use the Eyegaze to take control over his environment so I could be more independent to change TV channels turn lights on and answer phone He says he could do this but he just has not purchased that system Whilst he has not timed the number of words per minute he achieves but the eye response control or dwell select is set to 0 20 seconds which suggests an extremely proficient user In the experience of The ACE Centre achieving such a speed with any other pointer method than eye control e g a headmouse would be very difficult indeed Eye writing is quicker for him than when he was a
42. modality but efforts could be made to make the operation of the modality more usable To assess what modalities and methods are most usable for a user a definition of what constitutes usable and usability is required together with suitable methods of measuring usability in relation to eye based and other control modalities Knowledge of what available modalities in what configurations would give the most usable eye or other modality control for the user is vital for those using these systems and also for those helping others to use these systems Usability A widely accepted definition of usability in the context of computer applications is the Degree to which specified users can achieve specified goals in a particular environment with effectiveness efficiency and satisfaction ISO DIS 9241 11 From this the notion of effectiveness refers to the completeness and accuracy with which goals can be achieved The notion of objective efficiency refers to the effort or resources necessary to achieve these goals Figure 6 1 and subjective satisfaction refers to the subjective assessment by the user of factors such as workload comfort and ease of use whilst achieving these goals Table 6 1 What is measured Derived metric Derived metric Quantity of goal achievement Effectiveness ES Efficienc Quality of goal achievement y Task time Oo N Productive period Unproductive time Figure 6
43. synthetic voice output via a wireless network or Bluetooth connection to a headset worn by the communication partner The communication partner may then speak back at a normal level of voice in the headset s microphone transmitting the reply to the speakers of the computer in the living room Pointing at video images Hansen et al 2001 imagined a user scenario year 2010 in which a disabled person points with his gaze at an object on a video recording of the contents of his bag in order to get a bypassing person to fetch it for him Dahl an ALS patient once suggested having a video camera attached to his wheelchair He would like it to be mounted in a normal eye height position to provide him a normal standing view and he would like to be able to control zooming and movements of the camera by moving his eyes on a monitor on his wheelchair This he imagined could become an efficient pointing stick for him For instance he would drive around a self service table and zoom in on the food that he would like the communication partner s to give him Henning Dahl personal communication 2000 Fifth area of improvement Increase mobility Some of the present gaze tracking systems are quite heavy up to 10 kilos and only meant for indoor use at a fixed location The user has to be seated in front of it and head movements are constrained Low weight laptop computers can be placed on a wheelchair table in front of the user They may even be taken to
44. system are sufficiently accurate ell alc ed xh aT FRE Prsc setae ni EHE Figure 7 5 SofType onscreen keyboard EE SofType has a word prediction system to allow a reduction in the number of keys that the user must press and thus increase the writing speed The most important features of a good assistive system of course is that it allows the user to do things more easily faster and without effort Today young people are accustomed to communicate by SMS it s a quick way to send and receive greetings invitations news etc If a young person with severe physical difficulties wants to send SMS messages but is unable to use a mobile telephone QualiSMS can solve this problem This is a software application that allows the user to send SMS using the PC with their own specialised access device With QualiSMS it is possible to send and manage SMS personalise the folders and create a personal address book However the difference is that the user does not need to have a mobile phone 20 10 2005 50 106 Communication by Gaze Interaction COGAIN IST 2003 511598 whew Corta he Messages HS Amo 7 595 nam LETS 4 Ode Dantei tem dram un ki qe gE ty d Ww Figure 7 6 QualiS MS application for sending text messages via a PC Controlling the Environment using Assistive Technology Technology is available to enable people with disabilities to control their environment whether it is a single room t
45. that can be achieved with the buttons of an ordinary mouse Initially he kept the same on screen keyboard and predictor as before which was helpful because his predictor s vocabulary had gradually evolved to become increasingly personalised over several years About a year ago Paul decided to change to Dasher as his preferred method of text entry With this he estimates that he is up to about four times quicker than with his grid based text entry system Figure 5 13 The Penfriend on screen keyboard and predictor with Dragger for dwell select Apart from having an on screen method of text entry k5 2005 02 26_15 27 33 txt Dasher however the headmouse combined with Dragger is used to Fie Edit View Options Help perform exactly the same functions as an ordinary mouse As D x i amp amp a result Paul has quick and efficient access to the full range This is Dasher of Windows software At present Paul is using his headmouse very effectively However from past experience it is considered possible that head movement might become increasingly difficult and tiring for him For this reason it is important for him to consider eye control as his next method of accessing and controlling the computer Maximum Speed res Figure 5 14 With Dasher Paul can just drive through the letters to write and does not need to either press a switch or use a dwell select 5 3 Summary There is a range of ac
46. very beneficial and the other 1 said they thought it was quite beneficial Their potential benefits included control of the TV the ability to keeping up with events allow the user to be wireless and enable them to call their caregiver if required At present as far as we are aware powered mobility using eye control is not an option that is commercially available However the views of respondents on its potential use were extremely informative Of the 3 eye 20 10 2005 31 106 Communication by Gaze Interaction COGAIN IST 2003 511598 control users one did not comment one feels it would be beneficial the other very beneficial Their reasons included the following e Freedom from always asking others for help e Iam not moving around a lot in my chair Of the 8 users of other forms of access technology 2 thought it would be very beneficial one not very beneficial 3 not beneficial one was unsure and one did not comment Reasons for considering it to be beneficial included increased independence Their reservations related to safety issues for example problems with looking at the computer and direction at the same time problems due to a visual impairment problems with outdoor use Of the 5 professionals 4 thought that the facility to control a wheelchair using eye control would be not beneficial and the other 1 said they did not know Their reasons for it not being beneficial included safe
47. view was surprising as Ahmar has been a successful user of his SofType Headmouse combination for many years and because it was only a trial layout the on screen keyboard he tested out for eye control was nowhere near as efficient a keyboard layout as SofType gt AU Figure 6 12 Ahmar using a headmouse to play a Space Invader game His reason for choosing eye control technology for eye writing though was simple When using the headmouse for computer access because of his cerebral palsy Ahmar had to work at stiffening his whole body in order to keep his head still enough to achieve the level of accuracy he required in order to access the onscreen keyboard With eye control however using the Tobii system it was different This particular eye control system was able to ignore Ahmar s involuntary head movement and track the movement of his eyes alone As a result Ahmar was able to relax his whole body With this access method he felt regardless of whether it was quicker or not he would be able to eye write for far longer far more comfortably Therefore he felt he would gain far more user satisfaction overall when using this method for writing e Given a free choice Ahmar feels that he would find eye control more efficient effective and satisfying for writing on the computer On the other hand he prefers to use the headmouse for those applications where pinpoint accuracy is required On these occasions he
48. vistas pir rd re EE TA 16 10 GLOSSARY sissssssiecisscciscessvervisesastesitsscsssessuscsiateivuesdvsancuseeusrsnisaznvasessauoosicesuessatdesutederesvsiaesisusadusrsinzsiearoiivsusvasuseesavivaauediieeat 77 11 REFERENCES m 78 APPENDIX A EYE CONTROL QUESTIONNAIRE creen sss rei tte ttes tt ttti 85 Questionnaire n User Nes i casero rennen nente A a RE fede ccn rd ra ids 85 APPENDIX B EYE CONTROL QUESTIONNAIRE SUMMARY ernst en 100 20 10 2005 4 106 Communication by Gaze Interaction COGAIN IST 2003 511598 interaction 1 Executive Summary From the literature and data collected it seems that at present eye control can only be used effectively to meet a limited range of user requirements Furthermore it can only be used effectively by a limited number of people with disabilities who might benefit greatly from it To address these issues a number of recommendations are made in this document for consideration by COGAIN partners 1 1 End users eye control hardware requirements It is recommended that a good starting point would be to e Measure how effectively the eye control technology available can meet the needs of the full range of users who might benefit from it To achieve this aim it is recommended that WP3 User Involvement should e Trial as many specialist Eye Control systems as possible This will provide an opportunity to e Feed back to Eye
49. 003a 2003b Extensive use of trivial undo functions e g Jacob 1991 Increasing tolerance of noise on the gaze input by using large and or well separated selection areas e g Hansen et al 2003 Miniotas et al 2003 Fourth area of improvement Allow for multi modal in and out put on gaze controlled systems Ideally the user should be able to use the same system through all stages of their disability Many ALS patients for instance have little or no previous experience with computers and are quite busy adapting to the severity of their situation keyboards may be the only input form that they are familiar with Several communication partner s must be able to help the user complete letters edit text and use other functions in the program without having to learn an unfamiliar interaction method Limited time resources among the specialists responsible for introducing and configuring the system mean that the duration and quality of user training is often severely limited The progression through the stages of disabilities such as ALS is gradual and the fatigue factor often makes it necessary for the user to switch to a less efficient input method during the day They may become familiar with the system by using a well known keyboard When they lose the ability to use their fingers they may begin to use a mouse trackerball or joystick Head tracking may then be the most convenient control that they master when they lose the ability to
50. 1995 or similarly people who may have some reluctance difficulty or discomfort moving their hands or arms in Europe alone 6 6 of the working population aged 16 to 64 suffer from some form of arm or hand disability or related problem EUROSTAT 2002 to people who have little if any bodily movement Chapman 1991 such as Amyotrophic Lateral Sclerosis or Motor Neurone Disease which causes in later stages a form of locked in syndrome nearly 350 000 people suffer from these disabilities worldwide International Alliance of ALS MND Associations Between these extremes lie diverse ranges of motor disabled user groups who may benefit to a greater or lesser extent from using their eyes or head to interact with an interface These include any disabilities that cause paralysis or impairment of motor function at a high level on the body Examples include cerebral palsy brain injury resulting in locked in syndrome multiple sclerosis musculoskeletal diseases polio Parkinsonism and injuries to the cervical spinal column causing tetraplegia currently there are between 30000 and 40000 people in the UK alone with tetraplegia or paraplegia of varying levels of injury As the level of motor disability increases so the number of possible usable computer input devices or input modalities such as eye head or hand pointing decreases dramatically with the majority of input devices becoming unusable once hand function is lost As the level o
51. 3 511598 9 Discussion and Recommendations Eye control is a comparatively recent development as a method of access to technology for people with disabilities From the literature and data collected it seems that at present eye control can only be used effectively to meet a limited range of what users with disabilities require of it Furthermore it can only be used effectively by a limited number of people with disabilities see diagram below Diagram to illustrate the current limited use of Eye Control Technology Limited number of end users with limited number of applications Figure 9 1 Illustration of the current limited use of eye control technology This chapter therefore will discuss the key issues that have arisen from this document relating to what users require of Eye Control Technology that they cannot already achieve To address these issues a number of recommendations are made for consideration by COGAIN partners 9 1 Issues relating to users eye control hardware requirements Whilst many of those considered in this document already successfully use Access Technology to achieve many if not all of the things they wish to without using Eye Control their expectation is that it could provide a more effective efficient and satisfying form of access Their current method s of access to technology might be slow and or effortful for them whereas Eye Control Technology might offer a much more direct and efficient form o
52. 9 pp 474 481 COGAIN Annex 1 2004 Communication by Gaze Interaction COGAIN 511598 Annex 1 p 6 Corno F Farinetti L and Signorile I 2002 A Cost Effective Solution for Eye Gaze Assistive Technology ICME2002 IEEE International Conference on Multimedia and Expo Lausanne Switzerland Donegan M and MacKay D J C 2004 DECO The Dasher Eye Control Optimisation Project Proceedings of the Eleventh Biennial International Conference of International Society for Alternative and Augmentative Communication ISAAC 2004 Natal Brazil Douglas S A Kirkpatrick A E and MacKenzie LS 1999 Testing Pointing Device Performance and use Assessment with the ISO9241 Part 9 Standard Proceedings of CHI 99 ACM Press Drinker P A and Kropoff S 1981 Eye link for non vocal communication Direct selection by eye contact Paper presented at the 4th Annual Conference on Rehabilitation Engineering Washington DC Duchowski A T 2003 Eye Tracking Methodology Theory amp Practice Springer Verlag London UK 2003 EUROSTAT 2002 http europa eu int comm eurostat 2002 Evans D G Drew R and Blenkhorn P 2000 Controlling Mouse Pointer Position Using an Infrared Head Operated Joystick IEEE Trans Rehab Eng vol 8 no 1 pp 107 117 Findlay J M and Gilchrist I D 2003 Active vision The psychology of looking and seeing Oxford University Press Oxford Fould R 1980 Communication rates for nonspeech
53. 95 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction Section C I am a parent professional e g teacher carer health professional assistive technology specialist who supports a person people who have a disability who I feel might benefit from eye control technology Name Email Address Telephone Number Organisation Company if applicable Question Please write answer in this column Cla Do you support one person or more than one person Clb If more than one person about how many Clc What is the disability range of disabilities of the person people you work with e g due to Motor Neurone Disease Head Injury ME Cerebral Palsy Cld What age group do es he she they belong to Please choose Under 5 years 5 11 years old 11 18 years Over 18 Way over 18 Cle Are you a parent or professional Clf If professional what profession C2 Is Are he she they able to control assistive technology in any way e g Access the computer Operate a wheelchair Control their environment Yes No If Yes please answer C3 C4 and C5 If No please Go to C6a C3 If he she they are able to control assistive technology which device s do es he she they use e g Keyboard and mouse joystick headmouse switch es trackerball keyboard and guard etc C3a How is are he she they pos
54. Browser is already in use by a wide range of people using different access methods and can be controlled using other applications such as SAW from the ACE Centre gt wae singin an SAM eels o a ha 778 a a Figure 9 8 The WWAAC browser shown here with a simple on screen interface designed with SAW ACE Centre to enable users with physical and or communication difficulties to access the Internet more effectively There is no reason why a powerful interface design tool such as SAW could not be used to design any sort of symbol or text based user interface e To enable users to make an informed choice of which software to use for eye control a matrix should be set up on the COGAIN website relating to features of different software that can or could be used for eye control The comparison could be based on features such as those described above such as choice of output methods range of access methods range of multi modal access etc As a result of collating and publishing this information users and those supporting them will be able to help to steer the work of researchers and developers both inside and outside the COGAIN project 9 3 Issues relating to safety and reliability It is essential that Health and safety issues are uppermost in the minds of COGAIN partners when working with end users Infrared exposure Obviously users require the technology they use to be both safe and reliable In terms of t
55. COGAIN IST 2003 511598 e Potential difficulties in ensuring the correct positioning of the eye control device e Outdoor use The potential benefits of writing specifically using eye control for potential eye control users Of the 8 potential eye control users who expressed an opinion 2 thought it would be beneficial 4 thought it would be very beneficial and one thought it would be not very beneficial he already accesses the computer effectively with mouse and keyboard Their reasons were related to enabling greater choice ease of use independence and speed Comments included e It would allow her to have complete flexibility over what she wants to say e It is likely to be easier quicker and more accurate The benefits and perceived potential benefits of using eye control for email As described earlier facilities exist to enable users of both eye control systems and other forms of access technology used by respondents to use email Eye control users The reasons why emailing was important to them included the following from eye control users e I keep in contact with people daily and it gives me an outlet to feel I can still make a difference in somebody s life e Because I m giving you my thoughts Potential eye control users Reasons given why emailing was important to potential eye control users included e Indispensable for running a business from home e T feel she would
56. COGAIN IST 2003 511598 Examining text entry metrics for these studies were typing rate and subjective like or dislike of the overall system Stampe and Reingold 1995 typing rate error count task time gaze scan paths of the eye on the interface and subjective like or dislike of the system Majaranta et al 2004 and typing rate and user subjective qualification of typing efficiency and satisfaction with the system Hansen et al 2004 Task times and error rates are commonly used for eye based pointing usability studies together with task success or failure and task typing rates Although adequate with a modality or device that has a shorter task completion time and a lower error rate higher quality of interaction during the task almost certainly being more suitable for the task than a device with a longer task time and higher error rate these metrics are quite crude and do not offer great insight into the detailed usability of a modality or device In addition perhaps a device has a shorter task time but higher error rate than another device with a longer task time but lower error rate which device is most suitable for the task In addition from these findings it is evident that there is no commonly used usability assessment method for eye based pointing Common metrics between differing works were found with task time task typing rate and task error rate used however these depended highly on the nature of the tasks performed and
57. COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 light reflects directly back from the eye to the camera similar to the red eyes effect in photographs taken with flash If the light source is located off axis it the camera sees a dark pupil There are variations in how well each eye tracking system can track the user s pupil The size of the pupil has an affect on how well the system is able to track the person s eye According to Nguyen et al 2002 there are fairly big differences in the infrared bright pupil response of human eyes The ethnic background has on effect on how the light is reflected from the retina On the other hand in techniques based on the dark pupil affect the colour of the iris matters the contrast between the iris and the pupil must be distinguishable from the video image The camera must have an unobstructed view to the person s eye especially the pupil to be able to track it accurately Eyelids or eyelashes may partially cover the pupil The frames of the user s glasses may also obstruct the view to the eye and the frames or the lenses may cause extra reflections When contact lenses are used the reflection is obtained from the surface of the contacts instead of the cornea itself Small hard contact lenses may sometimes cause problems if the lenses move around considerably on the cornea LC Eyegaze 2001 For all the above mentioned reasons the calibration may fail even if the Figure
58. Communication EJ Na Information Society Lanha by Gaze Interaction IST 2003 511598 NoE COGAIN Communication by Gaze Interaction Network of Excellence Information Society Technologies D3 1 User requirements report with observations of difficulties users are experiencing Due date of deliverable 28 02 2005 Actual submission date 01 03 2005 Last modified 30 08 2005 Start date of project 1 9 2004 Duration 60 months ACE Centre Project co funded by the European Commission within the Sixth Framework Programme 2002 2006 Dissemination Level Pe Restricted to other programme participants including the Commission Services nad Restricted to a group specified by the consortium including the Commission Services Lo Confidential only for members of the consortium including the Commission Services 5d COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction Donegan M Oosthuizen L Bates R Daunys G Hansen J P Joos M Majaranta P and Signorile I 2005 D3 1 User requirements report with observations of difficulties users are experiencing Communication by Gaze Interaction COGAIN IST 2003 511598 Deliverable 3 1 Available at http www cogain org results reports COGAIN D3 1 pdf Main Author Mick Donegan ACE Contributors Lisa Oosthuizen ACE Richard Bates DMU Gintautas Daunys SU John Paulin Hansen ITU Markus Joos TU DRESDEN P ivi Majaranta
59. Control System developers how effectively their technology is meeting the needs of the full range of existing and potential users and e Make observations and suggestions relating to any potential modifications to their systems and or software that might make it more accessible and or more effective for more users e As the above information is acquired to enable users to make an informed choice of which hardware to consider for their eye control needs it is recommended that e WP3 should add the information gathered from the above investigations to the WP5 catalogue of currently available eye trackers The emphasis of the information provided by WP3 should be specifically related to usability issues related to the requirements of end users with disabilities e g environmental control portability issues mounting issues choice of output methods range of access methods etc It is anticipated that this will be carried out in collaboration with other Work packages e g WP5 Eye Tracker Development It is anticipated that this will be carried out in collaboration with other Work packages It is anticipated that this will be carried out in collaboration with other Work packages It is anticipated that this will be carried out in partnership with WP4 Tool development 20 10 2005 5 106 Communication by Gaze Interaction COGAIN IST 2003 511598 1 2 End users eye control software requirements Featu
60. G based Human Computer Interaction system is an alternative to eye tracking systems and when combined with an on screen keyboard is fully operational without the user having to initiate a physical movement The system transforms biosignals into controls for two dimensional cursor movement In contrast to eye tracking systems the HCI system has the potential to be relatively inexpensive and portable and eliminates some of the concerns related to eye tracking systems such as dwell time user training and loss of calibration Current approaches to EEG based communication can be divided into two groups those that use time domain EEG components and those that use frequency domain components Frequency domain methods use spectral analysis and focus on specific frequencies at specific scalp locations The BCI developed at the Wadsworth Center uses mu 8 12 Hz and or beta 18 25 Hz rhythms recorded over sensorimotor cortex to control a cursor on a video screen In the simplest case the amplitude of a single spectral band at a single location on the scalp is used to determine one dimensional 1 D cursor movements The user learns to control this The BCI research was presented at the Joint Research Centre of the European Commision in Ispra Italy Millan 2003 20 10 2005 29 106 Communication by Gaze Interaction COGAIN IST 2003 511598 amplitude Cursor movements are calculated 10 times per second by an empirically derived linear equation
61. He operates an adapted electric wheelchair using thumb control At present producing speech is not an easy process for him making voice recognition inappropriate at this stage He has a limited range of neck movement due to the metal pins used to fuse the break but he does have very good head control Though it is difficult for him to move his arms he is able to hold a mouse in one hand and operate the mouse buttons He has visual difficulties in one eye that make it necessary to wear a strong prismatic lens over the right eye which consequently appears opaque This made the Tobii system currently designed for use with both eyes simultaneously difficult for him to use successfully For Julius whilst he has tried eye control and a headmouse as alternatives his preference is for a CameraMouse system One important reason is that the system offers him a high level of Figure 5 10 Using head movement to control CameraMouse software via a webcam 20 10 2005 32 106 Communication by Gaze Interaction COGAIN IST 2003 511598 accuracy in directly accessing all of the Windows applications he needs to use Combined with a special on screen keyboard with built in prediction Skeleton Keys Julius is quick and accurate throughout the day and able to access all the functions available to every other computer user He describes himself as able to work at the speed of a medium paced user SER UT This is the skeleton key on sc
62. ISIOBOARD and then returns to it it is not designed to work at the speed necessary to accommodate the head movement of a user who has ongoing jerky involuntary head movement The Tobii eye tracking device tracks both eyes and is able to temporarily continue with one eye s data if the other eye is lost Tobii 2003 Whilst all current eye trackers have problems in tracking the eye s if the user s eyes have a tremor or if the user has involuntary head movements initial trials by The ACE Centre have suggested that the extent to which each system can cope can vary considerably No comparisons between the different systems in terms of their efficacy in this respect are available For this reason it is important for COGAIN to compare different systems with users who experience these difficulties If the calibration fails for the user some systems can be used with a default calibration and a special filtering of eye movements can be applied if the user has eye movement disorders Charlier et al 1997 For example in the Metrovision VISIOBOARD system allows to use 3 x 2 matrix for people for whom the calibration fails Naturally the accuracy of the measured point of gaze is very rough in such cases and the objects on the screen must be large e g only 6 9 large buttons visible on the screen at a time If only a few buttons are visible at a time normal on screen keyboards cannot be used The characters or commands must then be organized hierarchical
63. MUNICA has 4 800 different symbols to create personal dynamic communication tables It is a very powerful communication 20 10 2005 49 106 Communication by Gaze Interaction COGAIN IST 2003 511598 device for disabled people who either cannot or prefer not to use speech or written language Actually it s not possible use COMUNICA with an eye control system This is because it is a dedicated device that only runs its own software and not PC software This is an important issue as for people to need to use eye control as an access method the software they use would obviously need to be PC based to be able to be run on a PC based eye control Nonetheless there is a range of Symbol based software that could be used with eye control such as the Grid SAW Speaking Dynamically Pro etc Software for written communication For written communication on a PC a grid based system is often used This might involve a virtual keyboard that can be easily modified for the user For example the key size the keyboard layout the gap between the keys etc can be changed to suit the user A range of devices can be used to access on screen keyboards including switch access and pointing devices including joysticks headmice etc Importantly these keyboards can also be accessed using eye control SofType is an example of a PC based on screen keyboard that can be accessed by any pointing device including eye control if the user and eye control
64. Multimodal Eye based Interaction for Zoomed Target Selection on a Standard Graphical User Interface Proceedings of Interact 99 vol II British Computer Society pp 7 8 Bates R 2002 A Computer Input Device Selection Methodology for Users with High Level Spinal Cord Injuries Proceedings of the Ist Cambridge Workshop on Universal Access and Assistive Technology CWUAAT Trinity Hall University of Cambridge 25th 27th March 2002 Bates R 2005 Doctoral Thesis Enhancing the Performance of Eye and Head Mice A Validated Assessment Methodology and Investigation into the Performance of Eye and Head Based Assistive Technology Pointing Devices De Montfort University 2005 Beukelman D R and Mirenda P 1992 Augmentative and Alternative Communication Management of Severe Communication Disorders in Children and Adults Paul H Brooke Publishing Co Baltimore Bevan N Kirakowski J and Maissel J 1991 What is usability Human Aspects in Computing Design and Use of Interactive Systems and Work with Terminals Elsevier Science Bevan N Rengger R Macleod M Bowden R Drynan A Blayney M and Cooper D 1995 The Performance Measurement Handbook Version 3 National Physical Laboratory Teddington Middlesex United Kingdom TW11 OLW NPLUS PMH v3 0 Dec 95 Blake A and Isard M 1998 Active Contours The Application of Techniques from Graphics Vision Control Theory and Statistics to Visual Tracking of Shapes in Mo
65. P re 15 ecETN NPEP tiadoniaandidnuinaniddn dnautawnavandnunda mel aia anda Andon aaiiies 16 4 WHO IS CURRENTLY UNABLE TO USE EYE CONTROL TECHNOLOGY ener trees 17 4 What the literature Says nnne rentre inier neret iiiter 17 4 Information from stakeholders sse nennen eret inne 19 AS SUMMAN EORR E 23 5 WHAT POTENTIAL ALTERNATIVES TO EYE CONTROL ARE THERE nennen rere 25 5 1 Wiiatithe literature Says cese foci ener e err ted ERR v He EMO rei d FH iv v ox 25 5 2 Information from stakeholders cere rem rr enden oe tege ver nr Ee ir rr T gc 30 5 3 SUMMALY c iecore eco eon qe cv ve pen nod e o n RECEN vn ERE a RE C ta Fe P HL REX HE 34 6 CHOOSING BETWEEN EYE CONTROL AND OTHER ACCESS METHODS THE CONCEPT OF USABILITY 36 6 1 Whatthe literature Says nonni trate Chremes ert vec eb Dp vera tee rade 36 6 2 Information from stakeholders sesssessessessessessessessessussvsssssssssesscsseserssessnsssssnsessansesssuserssussssssnsusssseerssesnssssansessansessens 44 5 37 SU lideres ereiecre nie trenta serpere mat enr Pe ERE NET RCHE SER IRURE TER EXER UEBER ER A UE eigen atime 4 7 WHAT CAN PEOPLE WITH DISABILITIES CURRENTLY ACHIEVE WITH ACCESS TECHNOLOGY OVERALL 48 73 What the literature SayS ice oci ae AAE EEEa e creer d ud avian 48 L2 Information from stakeliolders iiiter ertnh e err ene rg eret ipi rar He 53 DEEP Na EE APANE AAEE 56 8 WHAT CAN PEOPLE WITH DISABILITIES C
66. Professionals Of the 5 professionals 4 thought that the facility to control the environment using eye control is very beneficial and the other 1 said they thought it was quite beneficial Their reasons for it being beneficial included e Control of the TV the ability to keeping up with events allow the user to be wireless and enable them to call their caregiver if required The benefits and perceived potential benefits of social communication using eye control As described earlier facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to communicate socially Eye control users Of the 3 eye control users one did not comment The two others already use their systems for social communication One feels it is beneficial and the other feels it would be very beneficial Their reasons included the following e It allows me to still be a part of my family s lives Plus I can still give advice and help others e Communication is a daily need because I need to say things on my mind Potential eye control users Of the 8 users of other forms of access technology 6 thought it would be very beneficial 1 was unsure because it depended on how their child s verbal skills developed and the other did not comment Reasons for considering it to be beneficial included e Greater freedom e Improve communication with the family less dependence on the caregiver
67. URRENTLY ACHIEVE USING EYE CONTROL TECHNOLOGY SPECIFICALLY AND HOW WELL eeeenreenrennrrenren resin rei rentrer iiit ttt tet e 2 5 8 3 Whatthe literature SayS eto ht hind ditare tht hene ttt vc ha cra th e d tbe 5 8 2 Information from stakeholQers iode deterret hcec dei eet tere e hn 66 8 3 SUMMALY Sica rata ie ene trop ca tb ir Gn Fd pode EHE ek HP E n ER ERE FERRE Ein eene d E XR Pe ERE 67 9 DISCUSSION AND RECOMMENDATIONS rnnt rns rre trt rei iiti iii eto 68 9 1 Issues relating to users eye control hardware requirements ssssseseeeeennnnnn nne 68 9 2 Issues relating to users eye control Software requirements 69 20 10 2005 3 106 Communication by Gaze Interaction COGAIN IST 2003 511598 A wide choice of on screen software interfaces sse eene eene tnn 70 A range of NputMethOds iuiciae ncaa mein cadence eon cado a ende n ice Fm a er rd de na a dca rns 70 Awide choice of output metliods 5 n ricco ence nate ccrtc ege caet dct aa ira eras 7 A choice of languages acea ce dire erede ed a n nenne rer rd aea od t PR e FC CA D rcr is 72 Summary of software requirements ssssseseseeeeneeennenn terne nter tentent rne rentre rne rin 72 9 3 Issues relating to safety and reliability 15 Infrar amp d eXpoSUFe iade odi tnc ete E d en RO Rc ie dedic d e eU v E Gra Hebe RE 75 Positioning and mounting ISSU85 center nre cnni nhe inniti erred nia
68. a Which software do you use A7Tc If you answered Yes to A7a About how many words per minute can you type with your eye typing system A8a Do you have any other way of controlling the computer that you could or do use to write with If Yes please answer A8b and A8c below A8b What other form of computer access do you or could you use e g single switch headmouse etc A8c How does your speed with eye control compare with your other forms of writing on the computer e g With eye control I can write about words per minute With my switch access I can write about Words per minute A9 Overall in comparison with other ways of controlling the computer what are the main advantages of eye control for you e g Do you find eye control requires less effort to use Do you find you can use eye control for longer periods A10 Overall in comparison with other ways of controlling the computer what are the main disadvantages of eye control for you e g Do you find eye control requires more effort to use Do you find you can only use eye control for shorter periods than your other access method if any Alla Whether or not you use the computer for eye typing how beneficial do you think this is would be for you i e please answer either very beneficial quite beneficial not beneficial 20 10 2005 89 106 Communication by
69. a trivial exercise It 20 10 2005 58 106 Communication by Gaze Interaction COGAIN IST 2003 511598 may not be desirable from an aesthetic point of view to have electrodes placed on the face while driving around see Figure 8 2 Jordan 2000 warns strongly against ignoring the socio pleasure when doing inclusive design The results of such attitudes can be seen in some of the hideous looking products that are designed to meet disabled people s physiological needs p 918 This will draw attention to the user s disability and because they embody aesthetics that few would accept given a choice immediately label the user as disabled and unempowered Jordan says Figure 8 2 Electrodes placed on a user s face for EOG gaze tracking The electrodes may draw more attention to the user than s he may like Entertainment and work Advanced gaze tracking systems provide dwell time selections of icons menus and links Users can operate standard browsers some popular games e g Solitaire and a range of normal Windows software In combination with virtual on screen keyboards text and numbers may be entered in most standard software While it is obviously a great advantage to have control of a normal PC the cost of the high precision systems and the restrictions they put on the mobility of the user are still an issue of some concern Web browsing is a very rich source of information and a leisure activity Unfort
70. adual implementation to avoid eye strain training and practice for the user Its not a natural way of looking Finally they felt there was a need for the user to be able to calibrate and re calibrate independently as and when the user wishes It is acknowledged of course that certain systems already meet at least some of the above requirements Case Study Stewart Stewart s case study provides an indication of the range of user needs that can be met through access technology overall Stewart has athetoid cerebral palsy This means that it is difficult for him to make any controlled movement Attempts at a deliberate movement can result in a certain amount of involuntary movement For this reason if Stewart wishes to keep still enough to control the head switch he uses he fixes himself by pressing his arms against the side of his wheelchair and pressing his head firmly against his headrest Stewart uses head switches to control a wide range of the Figure 7 10 Stewart using eye control and headswitch to access the Tobii Technology eye tracking system 20 10 2005 54 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction Figure 7 11 Stewart s DX wheelchair control device right with which he independently switches from communication and environmental control to wheelchair control technology he needs for leisure and daily living using an integrated system
71. al of effort and difficulty due to the need to transfer her gaze from mouse to keyboard to screen and back again For this reason she is extremely interested in using eye control for whichever computer applications she can Having trialled it over several days eye control Sarah feels potentially offers her a more direct comfortable and quick method of carrying out certain tasks on the computer even more so than the SmartNav headmouse which she sometimes uses Eye control in her opinion is more natural and requires less physical effort than either the mouthstick or headmouse With the Quick Glance II SH in fact Sarah is able to achieve a high enough level of accuracy to carry out a range of computer related activities that she enjoys such as computer games web browsing etc However at least for the present she prefers to continue to use the mouthstick for web design due to her requirement for a higher level of accuracy with her web design application Figure 6 8 Website designed by Sarah using mouthstick and ordinary mouse and keyboard For Sarah in relation to usability issues she finds eye control more effective efficient and satisfying for certain low accuracy level applications whereas she prefers to use a mouthstick when pinpoint accuracy is required Figure 6 9 Sarah using a Quick Glance Il SH on a laptop system Case Study Ahmar Ahmar is a young man with cerebral palsy who uses an Origin Ins
72. alibration Window GRecalibration Window Figure 4 9 Before and after the calibration on the left was good enough for Michael to eye write using a large grid but with a later calibration right he was unable to use the same grid effectively Two other eye gaze systems were tried with Michael on separate occasions an eye analysis system under development by QinetiQ and the Quick Glance II SH Despite three specialists from QinetiQ working with Michael for several hours it proved impossible to achieve a satisfactory calibration The Quick Glance too was not particularly successful Even though a calibration could be achieved albeit with difficulty the level of accuracy was very poor indeed only good enough for example to very roughly target each area of a 2 x 2 full screen grid Even a 2 x 2 grid however could not be used functionally because the mouse movement was not sufficiently dampened or filtered For example it was impossible to achieve any kind of dwell select because the pointer was jumping about so much The user trial with Michael gave the following indications At present whichever eye control system of those available was tried it would be difficult to achieve an effective calibration for Michael when his nystagmus is at its most severe Even if a successful calibration can be achieved account must be taken of the fact that the nature of Michael s visual difficulties fluctuates and a suc
73. and Ji Q 2002 Real time eye detection and tracking under various light conditions Symposium on ETRA 2002 Eye Tracking Research Applications Symposium New Orleans Louisiana pp 139 144 20 10 2005 84 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Appendix A Eye Control Questionnaire 11 1 Questionnaire on User Needs COGAIN www cogain org is a 5 year European Project that is dedicated to making progress with Eye Control Technology An essential part of this project is to actively involve people with disabilities and those who support them in order to bring about improvements with this exciting new technology By responding to this questionnaire you will be playing a significant part in ensuring that the aims of the project remain closely linked to what people with disabilities actually need rather than what others might think they need The following questionnaire is divided into separate sections aimed specifically at different groups Please answer only those questions that relate to you and return as soon as possible We would be grateful if you could return this questionnaire either as an email attachment to oosthuizen ace centre org uk or by post to Lisa Oosthuizen The ACE Centre 92 Windmill Road Headington Oxfordshire OX3 7DR Any queries can be sent to oosthuizen ace centre org uk or you can phone Lisa on 44 0 1865 759 800 NOTE THE INFORMATION YOU PROVIDE AND YOUR DETAILS WILL REMAIN
74. arly the way in which the software interface is designed e g magnification larger targets etc offers opportunities to enable more users with accessing difficulties e g those with involuntary head and eye movement to access this technology effectively A range of hardware and software will need to be trialled under COGAIN for comparison in relation the efficacy of its use for users with a wide range of needs Eye tracking systems aimed for people with disabilities include e EagleEyes http www bc edu schools csom eagleeyes e Eye Response Technologies ERICA http www eyeresponse com e EyeTech Digital Systems Quick Glance http www eyetechds com e H K EyeCan VisioKey http www eyecan ca e LC Technologies Eyegaze http www eyegaze com e Metrovision VISIOBOARD http www metrovision fr e TechnoWorks TE 9100 Nursing System http www t works co jp page01 1 html e Tobii Technologies MyTobii http www tobii com It is important that as many such potentially beneficial systems and appropriate software are trialled by a cross section of users and potential users as possible and the information gathered made available to all users and potential users 20 10 2005 16 106 Communication by Gaze Interaction COGAIN IST 2003 511598 4 Who is currently unable to use eye control technology At present Eye Control Technology finds it difficult to cope with users who have certain physical or visual difficulties These i
75. aze Duchowski 2003 This requires taking into account the head movements in addition and relative to the eye movements At least two reference points are required for the gaze point calculation sometimes called the glint pupil vector By measuring the corneal reflection s relative to the centre of the pupil Figure 3 1 In EagleE yes electrodes measure the the system can compensate for a certain degree of head EOG Gips etal 1993 movement Before the system can calculate the direction of gaze it must be calibrated This is done by showing a few e g 9 points on the screen and asking the person to look at the points one at a time If the person is not able to direct his or her gaze and focus on the point e g due to eye tremor the accuracy of the calibration suffers or may become totally impossible The duration needed to look at each calibration point may vary For example the LC Eyegaze s calibration procedure simply waits for a good fixation before moving to the next calibration point LC Eyegaze 2001 us o CS Ce wO wO Ce wO WO The corneal reflections are typically from an infrared IR or near Figure 3 2 Corneal reflections as seen infrared NIR LED light source Because IR is invisible to the eye by eye tracker s camera it does not distract the user If the light source is located on axis with the camera and the eye it causes a so called bright pupil effect The 20 10 2005 11 106 C
76. be an effective method of control for some motor impaired users Bates 2005 thesis Here objective efficiency was measured as 20 10 2005 40 106 Communication by Gaze Interaction COGAIN IST 2003 511598 described Figure 6 1 with measured quantities of quality and quantity of goal achievement Quality of goal achievement per task is measured as a deviation from a perfect score of 5 A number of pointing and selection actions between the start and finish of a task are classed as errors As the quality of interaction during the task is degraded by the weighted counts of the error types so the rating is reduced until either the task is completed or the quality rating is reduced to 1 at which point the current task is regarded as failed In this measuring scheme users are required to complete all of the tasks set which effectively removes variability in the quantity of goal achievement The task efficiency metric shown on the right hand of the diagram Figure 6 1 can be used to compare device performance and to enable the separate components of efficiency to be examined in detail These comparisons are complimented by measures of satisfaction Table 6 1 This approach has been used to quantify differences between head based and eye based pointing using a standard set of real world tasks The task sets are designed to include proportions of interaction objects and interaction techniques that are representative of typical word
77. bility of eye based control fall into two areas abstract target acquisition tests for example MacKenzie 1992 MacKenzie 1991 MacKenzie and Buxton 1992 Accot and Zhai 1997 Sibert and Jacob 2000 Douglas et al 1999 Murata 1991 Istance and Howarth 1993 Bates 1999 Radwin et al 1990 and simulated real world interaction sequences on user interfaces for example Istance et al 1996a Jacob 1993 Hansen et al 2004 Majaranta et al 2004 To define these two areas abstract target acquisition test tools are based on presenting the user with a sequence of targets of varying size and spatial separation on an otherwise blank screen and asking the user to rapidly point to targets in turn Figure 6 2 Typically the data collected from these experiments is sparse with the time taken to select targets and the number of errors being recorded as usability or performance metrics Test targets Simple interface vevelrres wela desi m Jim dew imes coe le lows iat IF at Figure 6 2 Example abstract target acquisition test Target acquisition test from Accot and Zhai 1997 20 10 2005 38 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction In contrast simulated real world test tools are typically based on the user performing a small set of tasks or interaction sequences on either a real environment or a simulated and simplified version of a real environment
78. ble to use his fingers to type I am faster with my eyes than I ever was when my fingers used to work 20 10 2005 15 106 Communication by Gaze Interaction COGAIN IST 2003 511598 He regards the process of eye control of the computer as essential to his quality of life I would have no desire to live without this eyegaze system and uses it for a range of activities He uses it for social communication writing emailing and access to the Internet Eye writing is his only way of communicating It enables him to still be a part of other people s lives Plus I can still give advice and help others Through emailing he keeps in daily contact with people It gives me an outlet to feel like I can still make a difference on somebody s life The Internet is his only way of keeping up with what s going on in the outside world If it were possible he feels that wheelchair control using his eyes would be very beneficial as it would provide him with freedom from always having to ask others for help 3 3 Summary At present whilst certain systems have features that are intended to accommodate certain accessing difficulties in practice their effectiveness will vary for different users Because of the lack of comparative information it is important that under COGAIN the efficacy of different systems with different types of users is compared in order to provide them with the information they need to make a well informed choice Simil
79. cess Emails Internet communication present methods are incredibly slow e Indicate choice have some degree of independence e Would like to make it easier for him to access all the available technology access is exhausting for him to do very much For both eye control and non eye control users their reasons for wanting to use additional applications related primarily to the need for independence but also included privacy and to reduce the shortfall between potential and performance The existing users of eye control systems are clearly very positive and enthusiastic about what they are able to achieve with their eye control systems overall The following comments were made in response to a question relating to any additional requirements they might have of eye control technology over and above they are achieving already e Environmental control This eye control user acknowledges that this could be done via eye control but it hasn t been purchased yet 20 10 2005 53 106 Communication by Gaze Interaction COGAIN IST 2003 511598 e Move from social communication application into eye control application myself e Play more video games turn the volume up and down e Tf anything I would like it to be more portable From all of the respondents with disabilities additional comments relating to their requirements of eye control technology other than those covered in the questionnair
80. cess devices available to for people with disabilities to use as alternatives to eye control including switches pointer based interfaces e g trackerballs joysticks headmice and electrical interaction e g brain computer interface and muscle EMG The factors involved in choosing an appropriate access device or devices are complex and depend on a range of issues including safety reliability independence ease of use etc For certain other people with disabilities however eye control is their only method of 20 10 2005 34 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction independent control of technology It is important that for these people especially efforts are made to explore the full range of applications used by people with disabilities that can increase their independence and improve their quality of life including social communication powered mobility and environmental control 20 10 2005 35 106 Communication by Gaze Interaction COGAIN IST 2003 511598 6 Choosing between Eye Control and other access methods the concept of Usability 6 1 What the literature says Introduction There is a wide and expanding range of user groups who may benefit from eye or head based pointing direct interaction with an interface These user groups range from people with no disabilities who may have hands occupied with other tasks and wish to point with head or eye Jacob
81. cessful calibration at one moment in time might not work effectively for him on another occasion On the occasion when Michael did achieve a successful calibration the significant increase in speed that he was able to achieve in comparison with switches combined with his greater comfort and satisfaction that the Tobii gave him meant that he was extremely Figure 4 10 On this particular occasion enthusiastic about eye control as an access method Michael was able to use the system very effectively though his performance varied His enthusiasm emphasised the importance of the need for depending on his physical condition developers to try to accommodate people like Michael by taking his kinds of accessing difficulties involuntary head movement and fluctuating visual difficulties into account 4 3 Summary As described above the questionnaire response from existing users of eye control was small However it was interesting to find a long term user with athetoid cerebral palsy amongst those who are already an eye control system successfully This is encouraging as many of those with this condition have a certain amount of involuntary head movement Obviously the visual and physical abilities and difficulties of any person with 20 10 2005 23 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction any given condition is different form any other person whether they have the same
82. condition or not As a result the amount of involuntary movement that results from having athetoid cerebral palsy for example will vary considerably Therefore if developers wish to meet the needs of any group of users with complex physical or visual difficulties therefore they will need to ensure that their systems are flexible enough to accommodate a wide range of differences It is important for COGAIN partners to remember that despite the benefits that some people with disabilities are already enjoying though the use of eye control technology many of those who would gain most benefit from its use are still excluded from using it It is hoped that by working closely with these potential users many more of them will be able to use eye control technology by the time the project is completed 20 10 2005 24 106 Communication by Gaze Interaction COGAIN IST 2003 511598 5 What potential alternatives to eye control are there This chapter describes a selection of access methods that might be considered as alternatives to eye control to provide a context for the use of Eye Control Technology within the wider fields of Access Technology and Disability Without such background information eye control technology could be applied to meet a user s needs inappropriately when alternative access technology might be better Eye Control Technology is just one of many ways in which even the most disabled users can access the computer It is very
83. control Highly experienced users 15 30 hours of use medium experience 6 8 hours of use and low experience users 1 2 hours of use of the eye modality were chosen and their results examined Table 6 3 User Experience and device efficiency Efficiency by subject experience Device Low Medium High L M H Hand 83 3 Head Dwell 51 6 54 5 63 92 Eye Dwell 19 7 31 8 61 1 Head Click 55 0 66 9 73 0 Eye Click 21 2 48 4 73 5 Head Dwell Zoom 49 4f 59 7 62 01 Eye Dwell Zoom 44 1 57 1 66 24 Head Click Zoom 55 0 70 2 75 3 Eye Click Zoom 48 6f 741 78 9 Table 6 3 User experience and efficiency This variation in specified users for both the original eye control modality and the eye modality with the modified particular environment showed that by changing the users specified by experience or training now made the eye control modality more efficient than the head modality Table 6 2 right hand column in bold with the eye control modality approaching the hand modality efficiency In summary usability analysis based on objective efficiency and subjective satisfaction showed that eye control can be as effective as head based control provided the specified users and particular environments of eye control are well defined and optimised It can show the individual elements and characteristics of eye control that limit its performance and show how usability criteria can be used to define specific
84. disabilities to accomplish daily living tasks assist them in communication education work or recreational activities In essence it can help them to achieve greater independence and enhance their quality of life Individuals with disabilities are often unable to achieve certain daily living tasks independently so they must rely on others for assistance with eating mobility reading and writing This situation can lead to feelings of frustration and unhappiness for the individual Fortunately technology offers a range of opportunities to support disabled people in achieving maintain independence in certain key areas using assistive devices Social Communication and Writing using Assistive Technology Often the biggest problem for a disabled is the communication to express himself to have relationship with relatives friends colleagues school friends with the world around Communication is crucial to establishing an effective relationship but without self respect it s difficult to express concern for the other person With some good assistive devices the user can establish relationships with the other persons he can write letters he can ask questions he can report a need he can meet friends etc so the person can play an active part in these situations There is a wide range of people with communication difficulties for example those with cerebral palsy MND Motor Neurone Disease Head Injury and ALS Amyotrophic Lateral Sc
85. e included e Portability Portable and connected to a laptop so that the user was not tied to a PC indoors e Control of positioning Operation of an indoor static chair for changes in position Changing the position of a profiling bed e All aspects of daily living From the professionals suggestions relating to their additional requirements of eye control technology that they felt were not being provided at the moment included e Leisure Playing PC games e Arithmetic for children e Alarms and assistance devices Respondents were invited to make any further comments they wished relating to issues that had not been covered in the questionnaire One respondent emphasised the need for information which is one of the areas that are indeed already being addressed under COGAIN e How do we find out about the suitability of different products From some of the professionals there were several comments relating to the practical issues involved in using eye control technology with confidence with all users for all of the applications they wish to use and wherever they wished to use them These included e The need to overcome potential technical problems including conflicts between Windows and the eye control system software involuntary movement problems caused by natural and certain artificial lighting robustness snagging cables difficulties in the setting up and calibration process the need for gr
86. e a deliberate movement and not accidental such as a tremor A key issue is that the input device must be activated with a minimum of effort in relation to the result achieved without causing any kind of discomfort Computer input devices for people with disabilities are classified by different features According to Shein et al 1992 there are three main groups e Switches e Keyboards e Pointing devices In turn there are 3 ways in which these input devices can be operated e Mechanical interaction e Optical interaction e Electrical interaction 20 10 2005 25 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Mechanical interaction requires the user to make a single deliberate movement to activate a switch e g by applying direct pressure or blinking or by breath With optical interaction interaction between the user and input device occurs by means of light waves in visible or infrared form With electrical interaction devices use acquired electrical signals directly from the user s skin or tissues Also it is pointed out Shein et al 1992 that proper seating and positioning that provides stabilization and support for the person with a physical disability They emphasize that it is the foundation for promoting effective interaction with a computer Devices with direct mechanical interaction Firstly there are switches that require direct mechanical interaction A wide range of switches is available
87. e a good starting point to look into the experiences gained from headmouse gaming How effective is gaze technology for users compared with other access methods e g in terms of speed comfort fatigue etc While it is unlikely that a gaze controlled AAC system will ever achieve communication rates comparable to unimpeded speech 150 wpm cf Table 8 1 as even a highly trained touch typist using QWERTY is unable to keep up with ordinary conversation the long term goal is certainly to achieve an input rate that is comparable to the QWERTY keyboard for a text composition task as this is usually sufficient to partake in on line conversations chatting 20 10 2005 60 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Method Speed of communication Speech 150 250 wom Fould 1980 Traditional typewriting 65 wpm Matias et al 1993 Stylus tapping on screen keyboard 9 30 wpm MacKenzie and Soukoreff 2002 Cursive handwriting 30 wom Wiklund etal 1987 Morse code 25 30 wpm Pierpont 1997 Palm top with Graffiti 20 wpm Zhai etal 2000 Multi Tap on 12 key phone pads 8 wpm J ames and Reischel 2001 12 key phone pads with T9 disambiguation 20 wpm J ames and Reischel 2001 Various combinations of AAC systems 2 8 wpm Beukelman and Mirenda 1992 Gaze typing with full on screen keyboard 7 wpm Saepen etal 1996 Majaranta and R ih 2002 Gaze typing with r
88. e are various commercial mouse alternatives Some systems use infrared emitters that are attached to the user s glasses headband or cap Other systems place the transmitter over the monitor and use an infrared reflector that is attached to the user s forehead or glasses see Fig 5 4 The user s head movements control the mouse cursor on the screen Mouse clicks are generated with a physical switch or a software interface Evans et al 2000 recently described a head mounted infrared emitting control system that is a relative pointing device and acts like a joystick rather than a mouse Chen et al 1999 developed a system that contains an infrared transmitter mounted onto the user s eyeglasses a set of infrared receiving modules that substitute the keys of a computer keyboard and a tongue touch panel to activate the infrared beam From the commercial sector the Headmouse uses a infrared to turn head movement into cursor control Mouse functions can be controlled by dwelling over a key for a set period of time dwell select Alternatively an external switch or switches can be set up to emulate the mouse button s such as a Sip Puff Switch It must be acknowledged that helmets electrodes goggles and mouthsticks can be uncomfortable to wear or use Commercial head mounted devices can often not be adjusted to fit a child s head Figure 5 5 Origin Instruments However some users in particular Headmouse Extreme young childre
89. e control system or have had only a little experience with one was gathered through questionnaires and informal interviews e To gain an insight into the issues involved in calibration and utilisation of eye control systems with users who are not currently using eye control a number of user trials were carried out with end users not currently using eye control Background Literature COGAIN WP3 partners have written the background literature sections collaboratively in Chapters 3 8 inclusive Refer to Appendices Refer to Appendices These user trials involved trying to achieve as successful a calibration as possible with a selection of users with complex difficulties and subsequently using the calibration to gain an indication of how effectively they might be able to use a selection of eye writing applications 20 10 2005 9 106 Communication by Gaze Interaction Communication by Gaze Interaction COGAIN IST 2003 511598 2 3 Summary An essential purpose of this document is to ensure that COGAIN partners are provided with information relating to e The wide range of user requirements that exist e The wide range of difficulties that need to be overcome in order to meet these requirements e The wide range of potential benefits of this technology in comparison with other access technology e The wide range of potential beneficiaries who are not yet able to access this technology With this information it is i
90. e gradually adjusted over time The Metrovision VISIOBOARD also allows recalibration on any part of the screen When the user fixates on any place of the screen for predefined period of time longer that the normal duration dwell time needed for a single or double click a calibration point appears on that spot If eye gaze is used to control a mouse not even 0 5 1 degree accuracy is enough for selecting the small elements in the standard graphical environment The accuracy problems can be at least partly overcome by using special methods for selecting the tiny objects on the screen For example The Quick Glance VISIOBOARD and ERICA use zooming screen magnifiers and fish eye views to select tiny icons and menus on the screen Lankford 2000 Rasmusson et al 1999 a mee Fe AYO goose DAs S a Ba leo 2 Figure 3 7 ERICA uses zooming Figure 3 8 VisionKey mounted on eyeglasses http www eyeresponse com zooming html htto www evecan ca The eye tracking devices can be table mounted remote or head mounted worn on the head If the device is used for interactive applications heavy head mounted trackers that must be tightly attached to the user s head are not convenient There are however lightweight head mounted eye tracking devices that can be used for a prolonged period of time without too much inconvenience For example the VisionKey by H K EyeCan is mounted on a pair of standard frames If the eye tracking system is u
91. e joystick fastened to a mouthpiece that can be keyguard guide to guide operated by the users tongue Salem and Zhai 1997 The joystick provides the users fingers to the cursor control while two switches a bite switch and a manual switch allow the letter keys user to use the left right click buttons Devices with optical interaction Optical systems rely on measurements of reflected or emitted light These systems inevitably have two components light sources and optical sensors The light sources might be passive objects that reflect ambient light or active devices that emit internally generated light Examples of active light sources include light emitting diodes LEDs lasers or simple light bulbs Optical sensors can be either discrete devices photodiodes phototransistors and other or image sensors CCD CMOS Lenses and apertures can be used to project images onto the sensor indicating the angle to the source The intensity of light reaching the sensor can be used to estimate the distance to the source Filters can be added to selectively admit or reject certain wavelengths of light For example a sensor system might use infrared light sources in conjunction with filters that only admit infrared light effectively providing a light channel separate from the ambient visible light Examples include an eye blink detector or the Headmouse from Origin Instruments The technology to detect a deliberate blinking action is based o
92. e monitor However the eye tracking system should be able to be operated from a semi reclined position if necessary Medication side effects that affect eye tracking Many commonly prescribed medications have potential side effects that can make it difficult to operate an eye tracking system Anticonvulsants seizure drugs can cause nystagmus blurred vision diplopia dizziness drowsiness headache and confusion Some antidepressants can cause blurred vision and mydriasis abnormally dilated pupil Some drugs commonly used to decrease muscle spasms e g Baclofen can cause dizziness drowsiness headache disorientation blurred vision and mydriasis Mydriasis can be severe enough to block eye tracking If the retinal reflection is extremely bright and the corneal reflection is sitting on top of a big bright pupil the corneal reflection may be indistinguishable and therefore unreadable by the computer In addition to these physical prerequisites which can make tracking of users eyes impossible there is the second problem of how to interpret the gaze data of an eye tracking system Jacob 1995 pointedly termed this difficulty the Midas Touch Problem The System has to differentiate attentive saccades with an intended goal of communication from the low level eye movements that are just random or provoked by external stimulation In order to answer this question one needs a model of eye movement control in relation to information processi
93. eed and simplicity The perceived potential benefits of powered mobility using eye control At present as far as we are aware powered mobility using eye control is not an option that is commercially available However the views of respondents were extremely informative Eye control users Of the 3 eye control users one did not comment one feels it would be beneficial the other very beneficial Their reasons included the following e Freedom from always asking others for help e Tam not moving around a lot in my chair Potential eye control users Of the 8 users of other forms of access technology 2 thought it would be very beneficial 1 not very beneficial 3 not benefical 1 was unsure and 1 did not comment Reasons for considering it to be beneficial included e increased independence The reservations related to safety issues e Problems with looking at the computer and direction at the same time problems due to a visual impairment problems with outdoor use Professionals Of the 5 professionals 4 thought that the facility to control a wheelchair using eye control would be not beneficial and the other 1 said they did not know Their reasons for it not being beneficial included e Safety issues as the eyes need to be used all the time for various purposes or errors could be made if the user slid down the chair or was under stress The potential and perceived potential benefits of environmental
94. eeds at the centre of COGAIN Figure 2 1 Adapted from Diagram of Usability in a wider context p 15 ECSC EC EAEC Brussels Luxembourg 1996 The aims of the COGAIN Project fit very well with the European Commission s TIDE USERfit ideology see Figure 2 1 Essentially this promotes an approach that is e User centred e System Oriented e Iterative 20 10 2005 7 106 Communication by Gaze Interaction COGAIN IST 2003 511598 A user centred approach is one which argues that it is the end user s requirements rather than technological capabilities which drives product development The logical starting point then is to take the time to understand the user population in some detail and understand what they need from products before going too far down the route of deciding on specific solutions Too often design is driven by technical feasibility which can lead to a poor match to user s needs A user centred approach on the other hand is keen to ensure that the proposed solutions e Have real value for end users e Are matched to user capabilities e Are fit for the purpose for which they were designed For these reasons COGAIN is ensuring that potential users of Eye Control Technology should be involved as frequently and extensively as possible This is why the ACE Centre as specialists in the field of Access Technology have been assigned with the responsibility of leading the process of facilitating the representation of
95. ely slow painful harmful or even impossible for them There are many thousands of people who fall into this category They range from some of those with conditions such as athetoid cerebral palsy who might have strong but uncontrolled body and head movements to certain people with head injuries who might find any form of movement including eye movement difficult to either initiate or control Both prior to and during the COGAIN Project The ACE Centre has already been collaborating with Tobii Technology in order to make their eye control system accommodate the accessing difficulties of even more users with disabilities even more effectively All eye control systems are different though of course and some systems will be more effective for certain users than others As a result in order to meet the user requirements of as many people with disabilities as possible one of COGAIN s aims should be to e Trial as many specialist eye control systems as possible This will provide an opportunity to e Feed back to eye control system developers how effective their technology is in meeting the needs of the full range of existing and potential users and e Make observations and suggestions relating to any potential modifications to their systems and or software that might make it more accessible and or more effective for more users 9 2 Issues relating to users eye control software requirements Though it is important that developers o
96. em he used it very quickly and efficiently and was soon able to use it far more quickly and comfortably than his current switch access method He was extremely pleased and impressed by this new quick and efficient method of computer access However despite this there is no way in which eye control can replace switch access in relation to his overall assistive technology requirements at present For example e Stewart can use his switch to control his wheelchair Eye control systems cannot be used for wheelchair control at present e Whilst the Tobii system can be set up for environmental control Stewart would have to go to whichever room the Tobii was in in order to control his environment His Cameleon on the other hand is attached to his wheelchair and with this his environmental control software can be used 20 10 2005 55 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction anywhere in the house It is acknowledged that some eye control systems can be attached to a wheelchair and used for environmental control However at present there are no commercially available eye control systems that enable a user to move around the house independently in order to control the whole of their environment e Using his switch and the Cameleon attached to his wheelchair Stewart can communicate wherever he happens to be whether in the house at the shops at the pub etc At the moment whi
97. er e g due to Motor Neurone Disease Head Injury ME Cerebral Palsy If you d rather not give a name to your disability please just describe how it affects your ability to control the computer Bib What age group do you belong to Please choose Under 5 years 5 11 years old 11 18 years Over 18 Way over 18 B2a B2b Are you able to control assistive technology in any way e g access the computer operate a wheelchair control your environment If Yes please answer B3 B4 and B5 If No please Go to B6a How are technology E g Sitting in wheelchair lying down side lying etc you positioned when you use B3 If you are able to control assistive technology which device s do you use e g keyboard and mouse joystick headmouse switch es trackerball keyboard and guard etc B4 What do you use your access device s for i e Computer access Speech output communication aid Powered wheelchair Environmental Control etc 20 10 2005 92 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction B5a On the computer is there any special software that you use Yes No If Yes please answer B5b and B5c B5b If you answered Yes to B5a Which special software do you use e g Softype The Grid Speaking Dynamically Pro Gazetalk etc B5c If you answered Yes to B5a What do you u
98. ery promising observation indicating that continuous typing on wearable micro displays may be feasible and can be as efficient as typing on a standard size monitor The advantage of gaze tracking on head mounted displays is the freedom of orientation and position Heynen and Snuggs 1999 report that the portability of the VisionKey head mounted gaze tracking system is highly valued Mounting of the tracking camera and display on the head itself eliminates the need to compensate for head movements Also the short distance between the eye and the display may provide a noticeable reflection in the cornea of the screen image which can be used for system self calibrating purposes or to eliminate vibration noise As discussed earlier however the disadvantage is that these systems may look awkward at least until it becomes common practice for users to wear such a display in front of a pair of glasses 8 2 Information from stakeholders Even though in terms of long term users the number of questionnaire responses and observations was small they were nonetheless very informative They provided an indication of the range of activities that are already being achieved through certain eye control systems and those users would like to achieve The range of activities carried out by those users from whom information was gathered included e Emailing e Internet Access e Social Communication e Writing i e not for emailing or social communication e g for
99. estricted keyboard 6 wpm Hansen et al 2004 Gaze writing with Dasher 25 wpm Ward and MacKay 2002 Table 8 1 Typical range of human communication words per minute wpm equals 5 characters per minute Present day eye gaze communication is efficient at a level that compares to text input methods on mobile devices PDA and mobile phones and one particular system Dasher shows promise for reaching the level of Morse code and cursive handwriting c f Table 8 1 Experimental investigations of gaze based single selections have often found them to be faster than mouse selections e g Sibert and Jacob 2000 Once the target is located the pointer is already there However when gaze based selections are used for more involved tasks such as typing or switch selections this superiority has not been manifest The speed of gaze selections has often been measured to be very similar to that of hand mouse selections e g Calhoun 1986 Miyoshi and Murata 2001 but exhibiting a higher error rate e g Ohno 1998 Hansen et al 2003 The productivity of previous gaze typing systems using on screen keyboards has been relatively low compared with other input modalities For example Instance Spinner and Howarth 1996 reported subjects to produce as little as one word per minute wpm 5 characters including space when entering their name and address mainly because they spent much time correcting entry errors e g typing the same
100. f Eye Control hardware systems are supported in allowing as many potential users to use their technology as effectively as possible it is nonetheless crucial that the assistive software that is used with their systems meets as many users requirements as possible Features of the wide range of assistive software already used via a range of access methods include the following It is anticipated that this will be carried out in partnership with WP5 Eye tracker development It is anticipated that this will be carried out in partnership with WP5 Eye tracker development It is anticipated that this will be carried out in partnership with WPS Eye tracker development 20 10 2005 69 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction A wide choice of on screen software interfaces Clicker 4 DsH cAB UPEESSIc e TOW This is a framework program called Clicker which can utilise symbols pictures and text working with Penfriend which is a predictive program and is very po Fie EM Inset Format Gad Opens OFB ABIL amp amp I like fish and chips I don t like hamburgers EYA el EN E A EN EA EA ree Sl LE EE E ee ESTEE L ECCES T N EA EN SS T Ein eadem aw Figure 9 2 A well established and well designed framework program like Clicker can provide a wide range of choice to users in relation to their preferences regarding text symbol or p
101. f Michael s eyes have a significant amount of involuntary side to side movement nystagmus to become more severe when he is tired A Tobii system was tried with Michael With this particular system if there are any specific areas that require re calibration they can be selected and recalibrated individually Figure 4 7 With the Tobii system any specific areas that Figure 4 8 It was very helpful to be able to switch the require re calibration can be selected and recalibrated Tobii from the eye control mode to eye tracking analysis individually note the red square illustrating the area of to be able to have a real time analysis of Michael s eye the screen that needs to be recalibrated movements The grid used was experimental with no special facilities such as prediction etc The first time it was tried in the morning the Tobii system was able to produce a calibration that was accurate enough for Michael to be able to select a letter he required for eye typing with two hits e g selecting a cell with the letters A F and then selecting the actual letter he wanted Unfortunately when it was tried in the afternoon when Michael s involuntary eye movement was greater finding a calibration that would enable him to reliably access the same size of cells proved very difficult The reason for this can be clearly seen in the screenshots below 20 10 2005 22 106 Communication by Gaze Interaction COGAIN IST 2003 511598 GRec
102. f access at least for some of the applications they wish to use Given the existing situation where many eye control systems are unable to accommodate certain users with certain physical and or visual difficulties the starting point for COGAIN on behalf of these currently excluded users is to make every effort to make it accessible to them It is recommended that a good starting point would be to e Measure how effectively the eye control technology available can meet the needs of the full range of users who might benefit from it 20 10 2005 68 106 Communication by Gaze Interaction COGAIN IST 2003 511598 There are currently two broad categories of users those who can use eye control effectively and those who cannot From the questionnaires and observations made they can be described as follows e Those who have good or mainstream eye control and who at least in principle should be able to use existing Eye Control Technology effectively e Those whose eye or head control is sufficiently different to the mainstream that currently available Eye Control Technology is unable to provide an effective calibration One of COGAIN s greatest challenges and if successful greatest achievements will be to help to make Eye Control Technology available and accessible to as many of those who cannot currently use it effectively as possible This is particularly important because the alternative methods of access to technology might be extrem
103. f motor disability approaches neck level only a range of single switch devices some unusual and limited bandwidth devices such as brain activity and muscle EMG speech and head and eye movement could be usable with sufficient bandwidth to give interaction for these users Bates 2002 Of these available modalities it is vitally important to accurately assess which is most usable for the person relying on these limited modalities for their everyday communication needs It is not sufficient to simply observe that the user is able to generate meaningful input or communication to a system with a given choice of modality it is quite possible that this choice of modality is not the best suited to the user The user may struggle with the modality or experience considerable workload and undergo considerable exertion to accomplish tasks It is also possible that the user can only use this given modality as this is all that is available Spinal Injuries Association Company Limited Newpoint House 76 St James Lane London Where bandwidth may be defined as the amount of information communicated to the interface per unit time by the modality For example a switch generates low bandwidth binary information a desktop mouse higher bandwidth x y positional information 20 10 2005 36 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction to them in this case the user must make do with the
104. features of different software that can or could be used for eye control The comparison would be based on features such as those described above such as choice of output methods range of access methods range of multi modal access etc Tt is anticipated that this will be carried out in partnership with WP4 Tool development 20 10 2005 6 106 Communication by Gaze Interaction COGAIN IST 2003 511598 2 Introduction what is the purpose of this User Requirements document 2 1 Putting the user at the centre All technology operates within a context and in designing products it is important to look at that context in addition to the technology itself From this perspective all technology is seen as being part of a wider system which must be designed correctly if that technology is to function appropriately USERfit Introduction p 19 ECSC EC EAEC Brussels Luxembourg 1996 Over the 5 years duration of the COGAIN Project partners will be involved in developing technology that will a accommodate the specific individual needs and difficulties of users with disabilities and b meet as many of their overall requirements of Access Technology as possible To help to put the next 5 years work into a context therefore this document will focus on providing information that will assist COGAIN partners in putting the real needs of end users at the centre of their research and development work Putting user n
105. fety issues when it comes to either environmental control or powered mobility As described earlier facilities currently exist to enable users of both eye control systems and other forms of access technology used by respondents to control their environment Of the 3 eye control users one did not comment Another who already uses eye control for their environment feels it is beneficial The other who does not have eye control over their environment feels it would be very beneficial Their reasons included the following e Freedom e Im doing things on my own Of the 8 users of other forms of access technology 4 thought it would be very beneficial 2 not beneficial one was able to use standard technology already and the other did not comment Reasons for considering it to be beneficial included e Increased independence privacy e He has more control over his eyes than his hands so it is likely to be easier quicker and more accurate It would increase the options available to him Their reservations related to potential difficulties with reliability and reduced portability e t would be a lot easier but reliability would be a big concern My independence is very important and I simply would not trust it My environmental control system is extremely portable I cannot see an eyegaze system ever being as portable Of the 5 professionals 4 thought that the facility to control the environment using eye control is
106. ficial quite beneficial not beneficial A15b Please give the reason s why you think that environmental control would be either very beneficial quite beneficial not beneficial Al6a Whether or not you are able to communicate socially e g through speech output with eye control technology how beneficial do you think this is would be to you i e please answer either very beneficial quite beneficial not beneficial 20 10 2005 90 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction Al6b Please give the reason s why you think that social communication is would be either very beneficial quite beneficial not beneficial Al7 Is there anything else that you would like to use eye control for If so what and why A18 Is there anything else you d like to add about eye control technology that hasn t been covered in the questions so far Thank you 20 10 2005 91 106 Interaction Communication by Gaze Interaction COGAIN IST 2003 511598 Section B I have a disability and do not use an eye control system or have had only a little experience with one Name Email Address Telephone Number Organisation Company if applicable Question Please write answer in this column Bla Why do you or the person you are writing for need to use a special way of controlling the comput
107. for independent mobility Many people in wheelchairs are able to control the wheelchair using a joystick However some people are unable to control the wheelchair with a joystick For these a number of alternative devices can be used For a switch user for example switches can be positioned within the headrest to enable the switch user to have full independent controlled mobility Using a special interface the switch user can independently change from controlling a wheelchair mounted PC to controlling their wheelchair and back again using the same switch switches See Case study Stewart in 7 2 below Leisure activities using Assistive Technology For a person with a physical disability that might spend their time in a wheelchair or confined to a bed it is important to have the opportunity use technology for leisure activities Technology can enable the user to play games whether by themselves or with others and also to express themselves creatively e g with Design or Music programmes A range of PC games are available commercially that can be used by people with disabilities without being modified For example if a person has good pointer control whether with a headmouse cameramouse etc Many pointer controlled board games for example can be played without modification in this way In addition specialised versions of games are also available for a wide range of access methods including switches such as QualiCHESS 20 10 2005 52
108. g The disadvantage is that it takes time for the user and communication partner to master it especially without the paper in front of 20 10 2005 57 106 Communication by Gaze Interaction COGAIN IST 2003 511598 them It does require mental concentration from either partner or eye contact between them From our observations typing speed varies In some cases when the user expresses daily needs in a well known context it may be several words per minute If irregular words and or original thoughts are to be expressed it may be just a few words per minute Computer based gaze communication High end eye trackers are available on the market at high prices They have some obvious advantages in that they may provide the user with pixel precision in pointing by gaze under optimal conditions For instance Tobii Technologies offers a Plug and Play eye tracking system It can be installed on a computer within a few minutes without any requirements or complex hardware besides the tracker itself which is integrated into a 17 TFT display Tracking is performed automatically so there is no need for a dedicated operator The tolerance to head movements is high 20x15x15 cm horizontal vertical depth at 50 cm distance from the screen The user can leave the system and when s he comes back tracking is resumed instantly The cost of the system is approximately 20 000 as of February 2005 On screen soft keyboards e g Point for Windo
109. g to such questions as e How well does the existing range of specialist Eye Control software e g GazeTalk and UKO ID compare with the wide range of currently available framework and specialist software used for access and communication e g The Grid Intercomm SAW currently used for pointer and switch access by many people with disabilities already e What improvements modifications need to be made to the current range of Eye Control software to meet as many needs of as many existing and potential users as possible To following recommendation is suggested in order to find answers to these questions e Measure how effectively current software applications are meeting the full range of eye control requirements of users with disabilities The User Trials that are an integral part of the COGAIN Project are critical therefore to find out this sort of information and should involve the following e Make a comparison of the various types of software that are currently or potentially available for eye control This will include for example a comparison of the features of both the kinds of assistive software currently used with a range of access methods SAW the Grid Clicker with specially written eye control software It is anticipated that this will be carried out in partnership with WP4 Tool development 20 10 2005 74 106 Communication by Gaze Interaction COGAIN IST 2003 511598 For example the WWAAC Internet
110. gs of the IEA 2000 HFES 2000 Congress pp 917 920 Kahn D A Heynen J and Snuggs G L 1999 Eye Controlled Computing The VisionKey Experience Proceedings of the Fourteenth International Conference on Technology and Persons with Disabilities CSUN 99 Los Angeles CA ten Kate J H Donker Duyvis J and LePoole J B 1983 84 Optical Communication Aids in Rehabilitation Medical Progress Through Technology 10 2 pp 109 123 ten Kate J H Frietman E E Stoel F J and Williems W 1980 Eye controlled Communication Aids Medical Progress Through Technology 8 1 pp 1 21 ten Kate J H Frietman E E E Willems W Ter Haar Romeny B M and Tenkink E 1979 Eye Switch Controlled Communication Aids Proceedings of the 12th International Conference on Medical amp Biological Engineering Jerusalem Israel Kreutz K 2003 Infrared Gaze Treacking in Matlab Barchelor student thesis The University of Queensland Available from innovexpo itee uq edu au confirmed 2005 02 22 Lankford C 2000 Effective Bye Gaze Input Into Windows Proceedings of the Eye Tracking Research amp Applications Symposium 2000 ETRA 00 Palm Beach Gardens FL ACM Press pp 23 27 LC Eyegaze 2001 The Eyegaze Development System A Tool for Eyetracking Applications LC Technologies Inc August 2001 MacKay D J C Ball C J and Donegan M 2004 Efficient communication with one or two buttons Proceedings of Maximum Entropy and
111. gy makes it possible to recreate human voices at a very convincing level compared to normal speech In future type to talk systems we imagine that some users might record their own pronunciations of their most frequently used words e g before the ALS progresses to the stage of voice impairments or before a laryngectomy patient gets the throat removed The communication system should then remind and assist the user in recording his or her most important words A digitised personalized recording of frequently used words may be used in combination with a pronunciation of unknown words by conventional synthetic speech Future development of speech technology may present us with a synthetic voice that can be attuned to a personal voice simply by listening to old samples of it taken from e g private video recordings Near range audio communication with communication partners By observing users of gaze controlled type to talk systems in their homes we have identified a need for the synthetic voice output to be transmitted across rooms in a house For instance the communication partner may be in the kitchen while the user sits in front of his computer in the living room At this distance voice output can be difficult to hear even when played loudly Also the communication partner would like to reply without going back to the living room Therefore we suggest including an option in the next generation of gaze communication systems that would transmit the
112. h as the amount of development time and the budget available e Some users will have more limited requirements of this technology than others so it is also reasonable to acknowledge that not all users will need to have a top level eye control system with top level assistive software Whichever decisions COGAIN partners make in meeting user requirements it is important that users are closely involved as part of a dynamic iterative process It is indeed the responsibility of the ACE Centre as leaders of WP3 to ensure that the evolving and emerging needs and requirements of users are taken into account on an ongoing basis For this reason this User Requirements Document should be regarded as a working document and one that evolves as the COGAIN Project progresses Eye Control Technology is a developing field so many of those 20 10 2005 8 106 Communication by Gaze Interaction COGAIN IST 2003 511598 whose opinions have been sought in relation to this document will not yet have had the opportunity to base their opinions on direct experience As a result some of their expectations and requirements might not actually be achievable under COGAIN As the project progresses and those involved become more familiar with what is and is not possible through Eye Control Technology their expectations of it will of course be shaped by events It is important therefore that this document captures these evolving requirements For this reason it w
113. he amount of infrared exposure there are clear guidelines on the level of infra red output for commercially available products e Itis recommended that only equipment which meets the legal safety guidelines on infrared output is used in user trials Tt is anticipated that this will be carried out in partnership with WP4 Tool development 20 10 2005 75 106 COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 interaction Positioning and mounting issues e When members of the COGAIN partnership carry out user trials involving equipment that the user is not already using it is recommended that they involve a specialist in the positioning and mounting of the equipment This might be a Physiotherapist Occupational Therapist Clinical Scientist or similar specialist This is a highly complex area where the experience of specialists in positioning and mounting is critical to comfort and safety e It is important to ensure that the responsibility for setting up the equipment and its subsequent use is fully covered against any unforeseen adverse effects that might result from the intervention Of course where a user already has an eye control system in place and the intervention only involves trialling software then the issues described do not apply Nonetheless COGAIN partners must make sure that they have appropriate public liability insurance cover in place 20 10 2005 76 106 Communication by Gaze In
114. he first problem is one of the eye tracking hardware and the second a problem of the application software using the data from the eye tracking hardware Influences on the hardware are closely associated with the physical abilities of the users It is worth noting that that these physical abilities may also have an impact on certain on eye tracking in healthy subjects but some of those listed in the next section are more prevalent in users with the aforementioned diseases Since literature on the impact of physical abilities on eye tracking is sparse most of the information given below came from practitioners with extensive experience The following physical conditions may make successful eye tracking difficult Difficulties with eye control To have full control over commercially available eye control as opposed to eye blink systems users must be able to look up down left and right They must be able to direct their gaze on all areas of a computer screen At a typical distance of a user from the screen this means that a user must be able to move their eyes approximately 25 of visual angle horizontally and 19 of visual angle vertically In addition they must be able to focus on a spot for at least 500ms in order for most eye typing systems to work properly There are certain common eye movement problems that make a satisfactory calibration difficult to achieve Text adapted from http www eyegaze com 2Products Disability Medicaldata html
115. he whole house or any other environment For example if the users want to control the TV activate an alarm turn the light on and off open and close the curtains answer or make phone calls unlock or lock a door or set the temperature it is important to be able to do this independently if possible Some of these environmental control systems are stand alone devices that are not PC based These include systems like Steeper or Possum With these users with disabilities can control their environment wherever they are in the house using their preferred access method e g switch access As well as stand alone devices there are many PC based systems that enable users to control their environment Several of these in fact are an integral part of the users communication software package such as EZ Keys QualiLife or The Grid It is important to note that in principle such environmental software could be used effectively using an eye control system 20 10 2005 Figure 7 7 Possum switch operated environmental controller 51 106 Communication by Gaze Interaction COGAIN IST 2003 511598 orta ena k D Emergency Figure 7 8 Examples from The Grid s environmental control grids which could potentially be adapted for eye control Powered Mobility using Assistive Technology A range of assistive technology can be used to enable people with mobility impairments to gain opportunities
116. here any special software that you use with eye control Yes No If Yes please answer A5b and ASc A5b If you answered Yes to A5a Which special software do you use e g Softype The Grid Speaking Dynamically Pro Gazetalk etc A5c If you answered Yes to A5a What do you use the special software for e g I use Softype as an on screen keyboard or I use the Grid for writing emailing and social communication A6a Is there anything else that you d like to do with eye control but can t at the moment If Yes please answer A6b A6c and A6d below A6b If you answered Yes to A6a What sorts of things would you like to do on the computer using eye control that you can t do at the moment e g Environmental control i e control television radio lights etc A6c If you answered Yes to A6a Why can t you do these things at the moment using eye control e g I don t have the right software A6d If you answered Yes to A6a Why would it be useful to be able to do each of these things 20 10 2005 88 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction A7a Do you write with your eye control system sometimes called eye typing or gaze typing If Yes please answer A7b and A7c below If No please explain why not A7b If you answered Yes to A7
117. hich the user is operating The definition then goes on to state the success and satisfaction of the user when using a given modality in the defined scenario Here it should be noted that measuring and understanding effectiveness and efficiency are more critical issues from the point of view of the motor impaired user than the able bodied user Motor impaired users may need to trade off apparently efficient ways of working which are physically demanding against less efficient methods which are less demanding but enable longer periods of use Users may often have a finite amount of energy that has to be measured out over the tasks they wish to achieve For example using a communication aid should be sustainable throughout a day even if communication is slow rather than only be usable for a short period of rapid communication However there are balances here where modalities may be chosen for their high effectiveness for short periods such as playing a game or driving a wheelchair will fallback modalities available for more sustained usage such as communication Thus measuring the satisfaction of a modality is a vital component in understanding the cost and sustainability of modality choices in terms of the available energy quota and the nature of tasks undertaken Tools currently in use for measuring the usability of eye control Reviewing previous work on assessing eye based pointing devices finds that tools for measuring the usa
118. ial quite beneficial not beneficial C12a If he she they were able to use the computer for emailing using eye control how beneficial do you think this would be i e please answer either very beneficial quite beneficial not beneficial C12b Please give the reason s why you think that emailing using eye control would be either very beneficial quite beneficial not beneficial C13a If he she they were able to use the computer for Internet access using eye control how beneficial do you think this would be i e please answer either very beneficial quite beneficial not beneficial 20 10 2005 98 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction C13b Please give the reason s why you think that Internet access using eye control would be either very beneficial quite beneficial not beneficial Cl4a If one was able to use the computer to control a wheelchair using eye control how beneficial do you think this would be i e please answer either very beneficial quite beneficial not beneficial C14b Please give the reason s why you think that wheelchair control using eye control would be either very beneficial quite beneficial not beneficial C15a If he she they were able to use the computer to control his her their environment eg TV Radio Telephone etc with eye control how beneficial do yo
119. icker Less effort Etc B10 If you think it would not be as good why do you think this e g difficult to use out of doors etc Blla If you were able to use the computer for writing using eye control sometimes called eye typing how beneficial do you think_this would be_for you i e please answer either very beneficial quite beneficial not beneficial Blib Please give the reason s why you think that writing with your eyes sometimes called eye typing would be either very beneficial quite beneficial not beneficial B12a If you were able to use the computer for emailing using eye control how beneficial do you think this would be for you i e please answer either very beneficial quite beneficial not beneficial B12b Please give the reason s why you think that emailing using eye control would be either very beneficial quite beneficial not beneficial B13a B13b If you were able to use the computer for Internet access using eye control how beneficial do you think this would be for you i e please answer either very beneficial quite beneficial not beneficial Please give the reason s why you think that Internet access using eye control would be either very beneficial quite beneficial not beneficial Bl4a If you were able to use the computer to control a wheelchair using eye control how beneficial do you think this would be to you i e please an
120. ictures As wide a variety of on screen software interfaces need to be offered as possible to enable the range of choice and the likelihood of matching the system to the user s needs and abilities Whichever software is used in order to enable users to interact with their eye control system their interface software needs to be as flexible as possible In relation to a grid based system for example the grids need to be resizable for example However a grid based system is not the only method of interfacing with the computer For example two eye writing systems that can perform the same task but could not be more different from a user s perspective are Qwikwrter and Dasher LI negi w c DOB4 FL a m mH o 09 8 EES Nee DO A OSE BAT Qwikwriter NUS is a different O On screen _ Keyboard ogz gt x eh ICE CED tor Er aa a at dosovwJo is qua gres em mc nm om fo TH 343200 sn Figure 9 3 Whilst both QwikWriter and Dasher can be used for eye writing they could not be more different from a user s perspective Whilst both applications are similar in that the user can eye write without the need to dwell click blink or press a switch Qwikwriter on the one hand offers the user a virtually static interface whereas Dasher is of course extremely dynamic A range of input methods Users require eye control software that can accommodate a wide range of input methods in additi
121. ill be periodically reviewed and revised 2 2 Data collection methods and background information Because of the relatively small numbers of existing users of eye control systems and their wide geographical spread it was anticipated that within the timescale of this document the numbers from whom information could be gathered and of whom observations could be made would be small Similarly the number of non users with whom it would be possible to trial eye control systems with would also be small For this reason the approach to data collection involved a qualitative rather than quantitative approach to with a view to gaining an insight into the range of issues involved for consideration by COGAIN partners rather than attempting to gather information of any quantitative significance at this stage The data collected therefore is intended to augment and enrich the information provided by the available literature Information from those supporting people with disabilities e The views of parents and professionals e g teacher carer health professional assistive technology specialist who support a people who have a disability were gathered through questionnaires Information from end users e Information from those people who have a disability and who are or have been regular users of eye control systems was gathered through observation and questionnaires e Information from those people who have a disability and who do not use an ey
122. in terms of words per minute wpm were offered Obviously not too much should be read into these figures because they are only rough estimates and the numbers of users are very small e Eye control users range of speed 6 20 wpm e Other access methods 1 19 wpm 50 100 wpm with voice recognition Opinions of eye control users on how they felt eye control compared with other access methods Of the 3 eye control users two said it was the only option they had The other said it was easier than using switches One user emphasised the increase in speed that had been achieved through Eye control e ts the only option I have used or could use Plus I am faster with my eye than I ever was when my fingers used to work Opinions of potential eye control users on how they felt eye control would compare with their existing access method in general Where an opinion was expressed 6 out of 8 they felt that eye control would be better or much better Reasons given included e His eye pointing is very good so I would hope that it would be easier and quicker e Less effort greater choice more control e Her head control is good but becomes unreliable when she is trying to press a switch e From the limited testing I have done potential for increasing my access speed especially when my body gets tired However there were certain concerns expressed These included 20 10 2005 101 106 Communication by Gaze Interaction
123. is is the collaborative work of The Inference Group at Cambridge University and The ACE Centre Oxford with the DECO Dasher Eye Control Optimisation Project Donegan and MacKay 2004 Over a period of two years the two organisations have been working in consultation with users to produce an enhanced version of Dasher that is better suited to eye control Enhanced features include hands free starting and stopping colour coding and a range of speech output facilities Speech facilities planned to be fully implemented by Summer 2005 20 10 2005 73 106 COGAIN Com jon Communication by Gaze Interaction COGAIN IST 2003 511598 Stop and start without pressing switch or mouse button i SSUDA Saren Starting and stopping is Starting and stopping is Figure 9 7 Screen from a Powerpoint presentation illustrating the way in which Dasher can be started stopped and started again at any time usina eve control only Whilst the development of an eye control version of Dasher through the DECO project has involved collaboration between the developers from The Inference Group Cambridge Assistive Technology specialists from The ACE Centre and users this type of collaboration seems to be the exception rather than the rule In general the involvement of users with disabilities has not always been extensively employed in eye control software research and development work There is a shortage of information therefore relatin
124. is prepared to trade off greater accuracy for less comfort 20 10 2005 46 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction 6 3 Summary The extent to which an access method and or application is efficient effective and satisfying to use is a complex and dynamic issue Users preferences for which access method they prefer to use is dependent on a range of factors including the level of accuracy required for a specific application and their physical comfort when using it For this reason it must be remembered that users also need to be able to make a choice depending on the application they wish to use For example e Sarah prefers to use the mouthstick for web design because whist it s not her most as efficient as eye control for this particular application her most important requirement is the effectiveness of her eye control in terms of accuracy On the other hand for activities when pinpoint accuracy is not required and effectiveness and satisfaction ie comfort are more important use eye control might be preferred For example e For Internet access and Games such as Solitaire Sarah prefers to use eye control e Ahmar prefers to use eye writing rather than the headmouse to produce text when the targets are large enough for him to access easily He can relax his whole body and does not need to stiffen in order to control his head movement something he has t
125. isease Head Injury ME Cerebral Palsy If you d rather not give a name to your disability please just describe how it affects your ability to control the computer Please write answer in this column Alb What age group do you belong to Please choose Under 5 years 5 11 years old 11 18 years Over 18 Way over 18 A2a Does your disability affect your eye movement or head control in any way If Yes please answer A2b below if No please go on to A3 A2b A2c How are you positioned when you use technology E g Sitting in wheelchair lying down side lying etc If you answered Yes to A2a How does this affect your use of the eye control system A3a Which eye control system do you use e g Quickglance II Metrovision Visioboard LC Technologies Eyegaze etc A3b How long have you been using eye control Years months A4a What do you use eye control for e g Emailing Internet Word processing Social Communication Games Leisure etc 20 10 2005 87 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction A4b How often do you use your eye control system e g every day daily weekly a few times a month A4c A4d A5a How long do you use your eye control system for in one session e g 2 hours Where do you use your eye control system e g at school work home therapy sessions etc Is t
126. itioned when he she they use technology E g Sitting in wheelchair lying down side lying etc 20 10 2005 96 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction C4 What do es he she they use his her their access device s for if anything e g Emailing Internet Word processing Social Communication etc C5a Is there any special software that he she they use s Yes No If Yes please answer C5b and C5c C5b If you answered Yes to A5a What kinds of special software do es he she they use e g Softype The Grid Speaking Dynamically Pro Gazetalk etc C5c If you answered Yes to A5a What do es he she they use the special software for e g They use Softype as an on screen keyboard or T use the Grid for writing emailing and social communication C6a Is there anything else that you would like him her them to do using access technology but can t at the moment If Yes please answer C6b C6c and C d below C b If you answered Yes to A6a What sorts of things would you like him her them to do on the computer that he she they can t do at the moment e g Environmental control ie control television radio lights etc C c If you answered Yes to C6a Why can t he she they do these things at the moment e g They don t have the right software C d If you answered Yes to C6a Why would it
127. l response of human eyes Proceedings of the Symposium on Eye Tracking Research amp Applications ETRA 2002 New Orleans LA ACM Press New York pp 133 138 Ohno T 1998 Features of Eye Gaze Interface for Selection Tasks The Third Asia Pacific Computer Human Interaction APCHI 98 IEEE Computer Society pp 176 182 Origin Instruments Corporation homepage Available at http www orin com Pierpont W G 1997 The Art amp Skill of Radio Telegraphy NOHFF 3rd Revised Edition 2002 Available at http www qsl net nObor nOhff htm 20 10 2005 82 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Rasmusson D Chappell R and Trego M 1999 Quick Glance Eye tracking Access to the Windows95 Operating Environment Proceedings of the Fourteenth International Conference on Technology and Persons with Disabilities CSUN 99 Los Angeles CA Radwin R G Vanderheiden G C and Lin M L 1990 A method for evaluating head controlled computer input devices using Fitts law Human Factors Aug 32 4 423 38 Salem C and Zhai S 1997 An Isometric Tongue Pointing Device Proceedings of CHI 97 Available at http www acm org sigchi chi97 proceedings tech note cs htm Salvucci D D 1999 Inferring intent in eye based interfaces tracing eye movements with process models Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit pp 254 261 May 15 20 1999 Pittsburgh
128. lerosis The way in which a particular form of disability can affect their ability to communicate varies considerably People with ALS for example can become unable to control muscles voluntary This can impact on their writing skills and oral communication The result is that they might find it impossible to communicate either through the written or 20 10 2005 48 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction spoken word When communication difficulties are this severe a communication device whether high tech or low tech can be an essential tool The E tran frame can be used with both symbols and text However to use an E tran frame with letters obviously requires the user to be literate A wide range of high tech devices and software exist to enable the literate user with a communication difficulty to express themselves These can be accessed by a range of methods see Chapter 5 such as switch es and pointing device e g joystick Headmouse CameraMouse and eye control The Lightwriter is a text based device that is light enough to be carried around by ambulant users Figure 7 1 The E tran frame is a Devices that enable literate users to communicate socially frequently commonly used low tech eye involve a PC based system with speech output For example the pointing device Cameleon is a device that is PC based and with a program such as EZ keys it can be
129. lling a wheelchair However they decided not to utilize gaze direction but to use tracking of face direction instead because gaze cannot be reliably controlled by intention in a dynamic environment where for instance other people walk around Canzler and Kraiss 2004 report on problems with highly variable lighting conditions e g sunlight and neon lights shining directly into the camera pointing upwards at the user s face Vibrations from the rolling wheelchair also complicated tracking by their experience It seems like there may be a couple of reasons why there are no reports on successful use of what might otherwise seem like an obvious control for a wheelchair First the weight of gaze trackers and their sensitivity to sunlight may have prevented mobile and outdoor use Secondly while it can be annoying to do an unintended selection on a PC it may cause a serious accident if it happens while driving Finally a lack of standard input protocols for wheelchairs may also have prevented gaze tracking suppliers from supporting this control mode As an alternative to video based tracking of gaze EOG tracking may overcome some of the problems associated with analysis of poor images see e g Barea et al 2002 However the danger of unintended eye movements causing dangerous driving still persists unless some kind of activation deactivation clutch is included in the control Fitting the electrodes for EOG recording on the user s head is not
130. logy in a Predictive Restricted On screen Keyboard with Ambiguous Layout for Severely Disabled People EACL 2003 Workshop on Language Modeling for Text Entry Methods April 13 2003 Budapest Hungary Hansen J P Johansen A S Hansen D W Itoh K and Mashino S 2003b Command Without a Click Dwell Time Typing by Mouse and Gaze Selections Rauterberg M et al Eds Human Computer Interaction INTERACT 03 IOS Press pp 121 128 Hansen J P Torning K Johansen A S Itoh K and Aoki H 2004 Gaze typing compared with input by head and hand Proceedings of the Eye Tracking Research amp Applications Conference ETRA 2004 ACM Press Hansen J P Hansen D W Johansen A S 2001 Bringing Gaze based Interaction Back to Basics Proceedings of Universal Access in Human Computer Interaction UAHCI 2001 New Orleans Louisiana Heynen J and Snuggs G 1999 Eye Controlled Computing The VisionKey Experience Paper presented at CSUN 99 Available at http www eyecan ca pages vkexperience htm confirmed 2005 02 19 Hornof A J and Cavender A 2005 to appear EyeDraw Enabling children with severe motor impairments to draw with their eyes Proceedings of ACM CHI 2005 Conference on Human Factors in Computing Systems New York ACM Hornof A J Cavender A and Hoselton R 2004 EyeDraw A system for drawing pictures with the eyes Extended Abstracts of ACM CHI 2004 Conference on Human Factors in Computi
131. lst the Tobii can be set up for use with social communication software it cannot be attached to the wheelchair and cannot be used outside in daylight 7 3 Summary It is clear that at this moment in time eye control technology can be used to access and control a range of applications that provide enormous enjoyment and independence to many users of this technology One application that is not commercially available yet however is eye control for powered mobility In addition whilst certain developers such as LC Technologies have included environmental control as a focus for their eye control applications this is an area that has not been widely addressed in research and development work In general the primary focus for much of the research and development that has taken place over the last two decades at least has been eye writing or eye typing However as this chapter has described there is a wide range of applications that users with disabilities need to use and are already able to use successfully with other kinds of access technology These include not only environmental control and powered mobility but also social communication and a wide range of games leisure activities Furthermore it is not only is it important for COGAIN to focus on increasing the range of applications accessible through eye control but also to focus on the quality of access to them too in terms us usability issues such as comfort safety efficacy etc Ques
132. ly This makes the use of the system slow because several selections are needed for one 20 10 2005 12 106 C COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 command By using predictive algorithms the system can speed up the selection process Frey et al 1990 Hansen et al 2003b HELLO Figure 3 4 Two level buttons EagleE yes Staggered Speech application Whilst it is acknowledged that the use of larger targets can be slower because it might necessitate two hits instead of one it must be remembered that larger targets can make the difference between a user being able to use an eye tracking device effectively and not using it at all Figure 3 5 Paul left picture was able to use a grid with smaller cells whereas larger cells were more appropriate for Michael right Making the cells in a grid larger can make the difference between accessing a target with two or three hits and not being able to access it at all see Figure 3 5 When compared with scanning even two or three hits to achieve a required target might be far less tiring quicker and more efficient for some users than scanning Refer to Ahmar s case study in Chapter 6 2 However if everything else fails the eyes might also be used as simple one or two way switches and the focus can be shifted from one item to another by using a method called scanning ten Kate et al L ILIL IL OUL 1979 For example the H K EyeCan
133. mage Understanding to appear April 2005 and books addressing the problems are forthcoming e g Physics of the automatic target recognition Springer Verlag in press It is fair therefore to believe that basic research within the field will mature in the years to come and new tracking methods will be invented Third area of improvement Accelerate gaze selections and reduce user errors Jakob 1991 identified The Midas Touch usability problem for gaze based interactivity namely that selections can happen unintentionally simply because the user is studying the interface He also noticed that it could be difficult for some people to stare at will in order to do a dwell time selection Naturally the eyes are moved whenever a piece of information has been noticed and a decision to act has been taken But if this is done before the end of the dwell time the selection is cancelled These two problems may explain why gaze selection often falls short on usability in more demanding tasks There are several ways to accelerate gaze selection and or to reduce time consuming error correction Using word or character predictions to minimize search time for target locations e g Ward and MacKay 2002 Reducing or eliminating the dwell time for each selection e g Salvucci 1999 Ohno 1998 Using task models to interpret inaccurate input e g Salvucci and Anderson 2000 Designing keys especially for gaze operation e g Majaranta et al 2
134. move their hands and arms Finally gaze may be the only function left that can be controlled at will So a comprehensive system should support input by either a keyboard mouse joystick or tracking sensors cameras Hand pointing may be an option to consider for gaze controlled systems in addition to keyboard mouse head and gaze input When the communication partner needs to operate the system we imagine that s he could do so simply by moving a pointing finger in front of the camera that is normally used for head or gaze tracking and thereby control the mouse pointer position The keyboard is often put aside when it is not needed anyway we have observed Even if the keyboard is ready at hand reaching it from a position aside or behind the user seems less comfortable than simply pointing with a finger in the field of the camera 20 10 2005 64 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction Combining voice input with gaze selections could also be a preference for some disabled users even when it is slurred For instance a simple rrrhmmm sound could command that the system should for example return to the main menu or initiate a re calibration Personalized voice output AAC has been pioneering the field of synthetic speech Most of the old synthetic voices available were very robotic and monotonous yet understandable e g to blind people New speech synthesis technolo
135. mpare in terms of the desirable features described above 20 10 2005 72 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Typical Eye writing program The Grid a widely available framework assistive technology program Resizable cells and grids Yes Yes A range of access methods Limited Wide Range A choice of symbols or text Text only Text pictures and symbols output A wide choice of text output One font one size Range of font sizes and styles styles and colours A range of speech facilities Limited speech output Full auditory support letter by letter word by word sentence etc Choice of languages Limited Extensive Table 9 1 Matrix showing a selection of features of a typical eye writing program with a widely available framework assistive technology program The Grid These facilities are associated with written and social communication only Of course there is a wide range of additional facilities available within programmes like The Grid that users also require such as the ability to change independently from one application to the next environmental control etc Because so many of these desirable features are already available in existing software it could be argued that if we truly wish to meet the full range of user requirements then it might be easier to adapt an existing piece of software to eye control than to create a new one An example of th
136. n dislike to be touched on their face and might object to any devices being attached to their heads Non mechanical switches such as the Infrared Jelly Bean switch from AbleNet Inc can be operated by moving any part of the body through an infrared beam Figure 5 6 S martNav Infrared There is a range of software available that enables a camera to be used device converting head movement for tracking used to track body movements such as head movement The into pointer control via reflective dot CameraMouse is software that enables for hands free control of a worn on the forehead computer using a video camera to track body movements head for example and convert those movements into cursor movements on a 20 10 2005 28 106 Communication by Gaze Interaction COGAIN IST 2003 511598 computer screen An on screen toolbar can allow the user to emulate all of the mouse controls The CameraMouse works with all standard software and requires no wires dots infrared beams or anything else to be worn on the head It tracks a body feature for example the tip of the nose with a video camera and uses the detected motion of the feature to directly control the mouse pointer on a computer The CameraMouse system currently involves two computers that are linked together a vision computer and a user computer The vision computer executes the visual tracking algorithm and sends the position of the tracked feature to the user c
137. n Sip Puff Switch Fig 5 2 is a head mounted accessory used to activate a two position switch by a simple sip or puff It consists of a head frame with attached mouth tube and a switch box connected to the head frame by a second plastic tube Sips and puffs are converted to switch closures inside the Switch Box In some cases it is not necessary to develop a new input device to enable a user to interact with the computer One alternative is to provide the user with a special device that acts as an interface between them and a mainstream input device For example for users who find it difficult to control their hands but who can move their heads effectively a special helmet can be worn Attached to the helmet is a wand with which they can access an ordinary keyboard Alternatively instead of attaching an enabling device to the user a device can be attached to the mainstream device itself to make it more accessible For example a keyguard can be attached to an ordinary keyboard This can enable those who have a tremor or other difficulties when trying to access a keyboard to use the holes in the keyguard to accurately guide their fingers to the target keys Figure 5 2 Head mounted Sip puff System from Origin Instruments Joysticks manipulated by hand feet chin tongue are used to control the cursor on screen For example the Tonguepoint system IBM Trackpoint III Trademark Figure 5 3 Using a is comprised of a pressure sensitiv
138. n the different amount of reflection from an opened and closed eye The detection bracket houses both the IR emitting diode and corresponding IR photodetector phototransistor Fig 5 3 and can be clamped onto the frame of a normal pair of glasses Shaw et al 1990 The emitters and detectors one on each side of the glasses are focused at the same spot on the sclera at the lateral corner of the eye Since the eyelid is both more absorbent and less reflective than the sclera the IR beam will be less strongly detected when the lid is closed Thus a clear threshold can be established between the open and closed states of the eyelid 20 10 2005 27 106 Communication by Gaze Interaction COGAIN IST 2003 511598 By setting up a dwell time that exceeds the users normal blink time the system can be set to be activated only when the user makes a deliberate wink In this way the eye blink becomes a single switch and can perform the same function as a mechanical switch pressed with the hand or any other part of the body Making different dwell times perform different functions can extend the functionality of the eye blink switch For example deliberately closing the eye for 0 5 seconds would produce a different result on the computer to a blink of 1 0 second a b J AU Figure 5 4 Optical blink detector a basic concept b detectors on bracket Shaw et al 1990 For those who have good head control ther
139. nclude for example those with involuntary head movement e g those who have athetoid cerebral palsy or involuntary eye movement e g due to nystagmus This chapter sets out to explore the wide range of difficulties and abilities of users with disabilities who wish to use this technology but who at present are unable to do so effectively It will consider the reasons for this and indicate the technical difficulties that need to be overcome in order to include this currently excluded group 4 1 What the literature says Eye tracking for disabled people who have various physical difficulties resulting from for example Amyotrophic Lateral Sclerosis ALS head Injury cerebral palsy Multiple Sclerosis Muscular Dystrophy Spinal Muscular Atrophy Werdnig Hoffman syndrome Rett syndrome Spinal Cord Injuries resulting in Quadriplegia and locked in syndrome may have difficulties using eye control technologies with respect to two possible sources of influence The first one is tracking the eyes themselves in terms of reliability and validity i e in order to further process eye tracking data First of all this data must reflect a the true gaze position and b the true temporal characteristics of the eye movement in question The ability to reliably track users eyes is a prerequisite for the second problem the eye tracking software has to interpret the data and respond to the user s intention in an appropriate way From another point of view t
140. ng Systems New York ACM pp 1251 125 Hornof A and Sato L 2004 EyeMusic Making Music with the Eyes Proceedings of the 2004 Conference on New Interfaces for Musical Expression NIME04 Hamamatsu Japan June 3 5 pp 185 188 Hutchinson T E White K P Martin W N Reichert K C and Frey L A 1989 Human Computer Interaction Using Eye Gaze Input IEEE Transactions on Systems Man and Cybernetics 19 6 pp 1527 1534 International Alliance of ALS MND Amyotrophic Lateral Sclerosis and Motor Neurone Disease Associations http www alsmndalliance org ISO 1998 ISO TC 159 SC4 WG3 N147 Ergonomic requirements for office work with visual display terminals VDTs Part 9 Requirements for non keyboard input devices International Organisation for Standardisation May 25 1998 Istance H O I and Howarth P 1993 Performance Modelling and Investigation of User Performance using an Eyetracker as an Input Device Polish English Meeting on Information Systems Technical University of Bialystock Poland Istance H O L Spinner C and Howarth P A 1996a Eye based control of standard GUI software Proceedings of People and Computers XI HCI 96 Springer Istance H O L Spinner C and Howarth P A 1996b Providing Motor Impaired users with Access to Standard Graphical User Interfaces GUI Software via Eye Based Interaction Proceedings of the Ist European Conference on Disability Virtual Reality and Associated Techn
141. ng tasks First of all a very basic function of eye movements is search If a potentially important object or feature has been discovered it should be perceptually described Such identification is only hardly possible without eye movements see also Findlay and Gilchrist 2003 After identification the information can be semantically categorized and some of the information may also receive additional self refential interpretation Only at this point the information may be considered to be ready for a possible communication The crucial parameter to distinguish different levels of processing and thus different states of intention is fixation duration In many experiments e g Velichkovsky Pomplun and Rieser 1996 a threshold value in the order of 450 500ms has proved as a practical solution to the Midas Touch Problem In terms of eye tracking based communication systems the eye tracking data should hence be temporarily filtered to avoid unintended selections 4 2 Information from stakeholders Under the COGAIN project it is anticipated that an effective methodology for evaluating the usability of a range of systems and software will be developed in relation to the wide range of user s abilities and requirements However to gain a small insight into how well eye control technology could cope with certain kinds of physical and visual difficulties a small number of brief user trials was carried out The aim was simply to find out how good a calib
142. nt format that can be used to compare results across both modalities and usage scenarios Looking back to the definitions of usability with respect to eye based pointing Figures 6 1 and 6 2 such a usability assessment scheme should be based on these definitions of objective efficiency and subjective satisfaction Usability measurement and the usability of eye control An attractive element in the usage of the notion of usability as defined previously Figures 6 1 and 6 2 is the derivation of metrics for efficiency effectiveness and satisfaction to enable these quantities to be measured for a given pointing device used by specified users for specified goals in particular environments This can provide a basis for comparing pointing device usability to be able to recommend which device or combination of device options to use for a given situation It also enables modifications or improvements proposed for devices to be assessed and quantified in terms of their impact on usability An example of the effectiveness of this approach can be shown for the evaluation of the usability of eye based control Typically eye control is regarded as inaccurate and difficult However by performing a detailed evaluation of eye control with both objective and subjective metrics of usability it can be shown that eye control can be effective for some users One study has been found that used this approach to show that eye based pointing can
143. ntended that COGAIN partners will be able to share a common understanding of the range of user centred considerations necessary to promote well informed pragmatic decisions about the hardware and software they develop 20 10 2005 10 106 Communication by Gaze Interaction COGAIN IST 2003 511598 3 Who is currently able to use eye control technology 3 1 What the literature says Current eye tracking technologies can be divided into the following categories electro oculography EOG scleral contact lens search coil video oculography VOG or photo oculography POG and video based combined pupil corneal reflection techniques Duchowski 2003 Not all of them are practical for interactive applications For example systems that use contact lenses are not practical or convenient for interactive applications even though they can be very accurate for psychological or physiological studies for example Certain EOG systems are impractical for everyday use because they require electrodes to be placed around the eye to measure the skin s electric potential differences However there are EOG systems aimed for augmentative and alternative communication e g the EagleEyes system Gips et al 1993 Figure 3 1 The most popular and suitable systems for use in interactive systems are video based Video based combined pupil corneal reflection techniques provide the Point Of Regard POR measurement the system can calculate the direction of g
144. ntitative analysis Rather it was intended that the response would give an insight into the range of views held by those involved and potentially involved In this respect the questionnaires have been an unqualified success Despite the small numbers the range and richness of the views expressed by people with extensive experience whether as end users or those supporting them provides an invaluable insight into the relevant issues and the tasks that consequently need to be undertaken under the COGAIN Project The ACE Centre is extremely grateful for the time taken the depth of thought and the honesty of all respondents some of which responded in great detail using their assistive devices Range of respondents There were 16 respondents 5 professionals 11 people with disabilities or others writing on their behalf Technology Used Of the 11 people with disabilities represented 3 are existing users of eye control systems 5 are switch users 1 uses a keyboard trackerball and 1 uses a headmouse dwell select 1 uses no technology at present Under 5 with spastic quadriplegia registered blind Ages 2 were under 5 years old 2 were aged between 5 11 years 7 were adults Range of disabilities The range of disabilities included cerebral palsy various forms ALS Traumatic Brain Injury TBI Professionals Collectively the professionals have experience working with a wide range of end users including Stroke Head Injury Motor Ne
145. ntrally mounted joystick When The ACE Centre first assessed him for access technology 9 years ago he was able to use an ordinary keyboard and mouse However since then his physical abilities have changed The speed at which Paul has been able to move his fingers and arms has slowed down his range of movement has decreased and the amount of pressure he is able to apply has steadily reduced For this reason it has been necessary for The ACE Centre to review Paul s progress and recommend any changes necessary to the access i Figure 5 12 Paul four years ago using technology Paul uses on an ongoing basis his finger to control a mini joystick that he can now no longer use due to deterioration in his physical condition 20 10 2005 33 106 q COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 In terms of access devices Paul initially used an ordinary mouse and keyboard but then when his range of movement and strength decreased he needed to use a mini joystick with an on screen keyboard and predictor Now because he can no longer apply enough pressure to the mini joystick he uses a SmartNav headmouse because he still has a reasonable range of head movement DIDI RE maem DOR 4 a c mm a x ET This is the Penfriend on screen keyboard and Dragge 3033333 Bil vie r ty iei e He combines the headmouse with a dwell click utility Dragger 32 that enables him to carry out all of the functions
146. o do in order to accurately control a headmouse to produce text Whilst the concept of choice is important at the same time it must be noted that some users with disabilities do not have a choice Clearly from some of the questionnaire returns for some people with complex accessing difficulties the choice has been made for them Some users with ALS and MND have been provided with eye blink systems without ever having had the chance to try an eye control system For these users then it is eye blink or nothing In COGAIN Annex 1 2004 Description of Work it has been described how partners have already shown that current eye writing rates can be increased from 1 5 words per minute which is common in many present day AAC systems to 10 25 words per minute Though it is encouraging that at least that the users of eye blink systems have some method of access to the computer it is a responsibility of COGAIN partners to make both users and those supporting them aware of the significant potential advantages of eye control over many other currently used access methods A clear distinction needs to be made between the concepts of usable and usability Many of those assessing and supporting those with complex disabilities are satisfied when they reach a point when they find an access method that enables their client to use their technology However it is clear from the views of users like Ahmar and Sarah that it is necessary to take into acco
147. ologies ECDVRAT 96 Maidenhead UK 20 10 2005 80 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Jacob R J K 1991 The use of eye movements in human computer interaction techniques what you look at is what you get ACM Transactions on Information Systems TOIS 9 pp 152 169 Jacob R J K 1993 Eye gaze computer interfaces What you look at is what you get IEEE Computer 26 7 July 1993 pp 65 67 Jacob R J K 1995 Eye tracking in advanced interface design In Barfield W and Furness T Eds Advanced interface design and virtual environments New York Oxford University Press Jacob R J K and Karn K S 2003 Eye tracking in human computer interaction and usability research Ready to deliver the promises Section commentary In Hy n J Radach R and Deubel H eds The Mind s Eyes Cognitive and Applied Aspects of Eye Movements Oxford Elsevier Science Available at http www cs tufts edu jacob papers ecem pdf James C L and Reischel K M 2001 Text input for mobile devices Comparing model predictions to actual performance Proceedings of the ACM Conference on Human Factors in Computing Systems CHI 2001 New York ACM 2001 pp 365 371 Jeppesen B B Andersen J Gr nnegaard A Hauge M Kirkegaard J and Seelen B 2004 Speaking with the eyes DVD in English available from www thi fyn dk Jordan P W 2000 Inclusive Design An Holistic Approach Proceedin
148. omputer The user computer interprets the received signals and runs any application software the user wishes to use CameraMouse will work with most USB Cameras that utilize the CCD image sensor The Logitech QuickCam Pro 4000 3000 Orbit and Intel Pro Video Figure 5 7 A CameraMouse system PC USB cameras have been lab tested and work well A single computer version of the system ha also been developed Electrical interaction Another approach is the Brain Computer Interface BCI that uses electroencephalographic EEG waves originating in the brain Users interact with physical devices through nothing more than the voluntary control of their own mental activity Between the simple EEG and the extremely invasive direct recording of neurons a researcher might reasonably consider using another established brain imaging technique such as magnetoencephalography functional magnetic resonance imaging and emission tomography Nevertheless all such techniques require sophisticated devices that can be operated only in specially designed medical facilities One of the major limitations of BCI systems is the high potential for electromyographic EMG contamination Any muscle movement on the head or neck can produce noise contamination from the corresponding EMG signal From an application standpoint this can result in difficulties for the user especially if they have a movement related disorder such as cerebral palsy The EMG EE
149. on to eye control This is in order to either a enable users to change from one method to another if and when their condition changes b spread the physical load if required or c enable multi modal access One reason for 20 10 2005 70 106 C COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 accommodating multi modal access for example is that whilst many users will have no choice but to use dwell select or blink as their mouse button equivalent others would wish to use combinations of access methods such as eye control voice eye control switch etc A recent example of potential modifications to a pointer access program so that it can also be used for switch access is provided by MacKay Ball and Donegan 2004 A wide choice of output methods Much of the software that is written for eye gaze systems is restricted to eye writing using text only However it is clear from the range of needs and abilities of users with disabilities that they require much more from their eye control applications They require for example A choice of symbols or text output Many people who would benefit from eye control communicate using symbols as opposed to text For some this is necessary because they do not have a high level of literacy For others this is through choice as they regard it as a quicker and more efficient form of communication e g using Minspeak V Vanguard Demo X File Options Help Speed i
150. only a nasal side patch to the other eye the user will retain peripheral vision on that side Visual acuity issues Several common vision problems may affect a user s ability to see visual elements clearly on an eye tracking system monitor These include the following A Inadequate Visual acuity The user must be able to see visual elements on the screen with adequate acuity If prior to an injury or the onset of an illness a user wore glasses he may need corrective lenses to operate an eye tracking system If a user is over 40 years old and has not had his vision checked recently he might need reading glasses in order to see the screen clearly In most cases eye tracking works well with glasses The calibration procedure should accommodate for the refractive properties of most lenses Hard line bifocals can be a problem if the lens boundary splits the image of the pupil making it difficult for a system s image processing software to determine the pupil centre accurately Graded bifocals however typically do not interfere with eye tracking If users wear contact lenses that cover all or most of the cornea eye tracking generally works well In this case corneal reflection is obtained from the contact lens surface rather than the cornea itself Small hard contacts can interfere if the lens moves around considerably on the cornea and causes the corneal reflection to move across the discontinuity between the contact lens and the cornea B Diplopia
151. only ever wishes to perform these limited tasks but what if the user wishes to do new or different tasks will the usability of the modality be the same as before In addition the factors that are assessed and quantified vary due to the task undertaken rather than using a common methodology Examining previous work conducted on head and eye based pointing found a range of different test scenarios A brief with only a small number of tasks but wide ranging assessment of eye based interaction with text entry text editing application and menu manipulation and limited internet browsing was found Istance et al 1996b In this work usability metrics were the text entry rate in number of characters per minute together with task times and task error rates Another attempt at a range of assessment scenarios for eye based pointing involved typing on a full screen keyboard typing on an environmental control with full screen keys and playing a simple game with metrics of simple success or failure of the tasks Chapman 1991 Metrics of task time and task error rate were used for eye based selection and manipulation of ship representations on a military interface Jacob 1993 Sibert and Jakob 2000 for drop down menus Byrne et al 1999 manipulation of a simplified icon and menu based interface Salvucci and Anderson 2000 Text editor and keyboard from Majaranta et al 2004 20 10 2005 39 106 Communication by Gaze Interaction
152. personal pleasure e Instant Messaging e Playing Games 20 10 2005 66 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction Those requirements which users are not currently achieving through eye control but would like to included e Powered mobility e Environmental control e Play more video games e Move independently from one application to another e g from Speaking Dynamically Pro to Writing software This is not to say that these activities cannot be achieved with certain appropriately set up eye control systems Nor is it suggested that the list of requirements is in any way comprehensive However it provides an important indication of some of the things that certain existing users would like to be able to achieve through eye control but cannot 8 3 Summary In addition to eye writing eye control can already be used for a variety of applications environmental control powered mobility and leisure applications such as games music and drawing In terms of its effectiveness for writing and communication eye control is already able to produce results of up to 25wpm One of COGAIN s aims is that eye control should work more effectively for more people As a result it is anticipated that more eye control units can potentially be sold In turn it is anticipated that the price per unit will come down As well as reducing its price it is planned that amongst anticipated impro
153. plays an important role for some gaze estimation methods Efforts are made to improve eye tracking under various light conditions Sun light and glasses can seriously disturb the reflective properties of IR light Methods using IR can therefore be less reliable in these situations and some researchers have suggested new approaches to these problems e g Zhu et al 2002 Other approaches avoid the use of active illumination but rely solely on natural light e g Hansen and Pece 2005 This is for two reasons Standard affordable components of off the shelf camera equipment may then be used and problems with sunlight and glasses may partly be overcome Natural light sources however make the problem of detection much harder as fewer assumptions on the image data can be made In summary several new approaches are currently being taken within computer vision research projects that eye tracking may benefit from The challenging case of eye tracking is attaining the interest of several research groups Eye tracking has become a recurring session theme in various computer vision conferences e g Computer Vision and Pattern Recognition CVPR International Conference on Computer Vision ICCV International Conference on Automatic Face and Gesture Recognition FG Special journal issues 20 10 2005 63 106 q COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 on this topic have been planned e g Computer Vision and I
154. processing and web browsing tasks Selection of targets on screen was made by two common methods used for eye control by continued pointing at the same object dwell or by using a command click from separate device such as a switch controlled by another modality e g sip puff The overall comparisons of device efficiency and satisfaction are shown in the figures below Figures 6 4 and 6 4 The results for both of the eye based modalities selection by dwell and selection by click were significantly worse in terms of efficiency and satisfaction than for the head based modalities Figures 6 4 and 6 4 Both the head and the eye pointing modalities were worse than the hand held mouse modality Task Efficiency by Device All Devices 100 90 4 a 80 m o x 704 gt 2 60 o E o 2 50 i i i x 40 o S i Notes 30 4 H Dn NN Data sets with 20 4 the same letter are not 19 i significantly 0 different p gt 0 05 Hand Head Eye Head Eye Dwell Click Device Figure 6 4 Device efficiency 20 10 2005 41 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Questionnaire Results All Devices High a i i 9 i e 6 ro B o o 5 4 o E o am b o o 5 47 d c j d 9 2 3 T tp
155. ration if any could be achieved with a small number of users who had visual difficulties difficulties with head control or both In all there were 10 users involved in these introductory user trials They included children and adults whose range of difficulties included head injury stroke cerebral palsy and athetoid cerebral palsy With several of them along with their physical difficulties they had an associated visual difficulty e g divergent squint nystagmus User Trials Our very limited at this stage user trials suggested that to a degree certain eye control technology is already able to cope with certain users who have certain difficulties with head and or eye control In turn even if a calibration has been successfully carried out eye control might only be effectively be used up to a certain level of accuracy Two examples follow 20 10 2005 19 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Case study Claire Claire has athetoid cerebral palsy which means it is very difficult for her to control her movements She is very bright literate and well motivated She uses a special joystick to access the computer Using the joystick and a range of specialised on screen grids designed within SAW she is able to use the joystick effectively and accurately to control the computer Nonetheless the method is very time consuming and involves a great deal of physical effort for her because with her par
156. reen keyboard used by Julius with the secu eje DAAE Cr aeea Jenn 8 e 7 8 s e 099 RISUS UH eliriitivivitieisic i MIB BE eio onim 9 B AMM es m m fn e ini mi 8088 mc2os0Tw eet ee ee fe UMLe3 re Figure 5 11 Even though Skeleton Keys has small targets Julius is able to control it quickly and effectively using the CameraMouse Key factors that determine his proficiency on the system are the ability to press both buttons on the mouse quickly and selectively and what he describes as the powerful functionality of Skeleton Keys with which he can carry out all of the functions he would be able to on an ordinary keyboard By comparison at present an eye control system would have very little to offer him e For Julius computer access needs an eye control system no matter how good would be unlikely to be more quick or accurate With his CameraMouse he is able to quickly and accurately select targets as small as the Minimise and Close Window buttons with ease as well as all of the keys on his small on screen QWERTY keyboard e Even a mid price range eye control system would be far more expensive than his CameraMouse system e The CameraMouse can be used out of doors as well as indoors more reliably than an eye control system Case Study Paul Paul is a young man who has cerebral palsy He uses a powered wheelchair for mobility that he controls albeit with difficulty using a ce
157. res of the wide range of assistive software already being successfully used via a range of access methods in addition to eye control include the following resizable cells and grids a range of input methods a wide choice of output methods a choice of symbols or text output a wide choice of text styles and colours a range of speech facilities a choice of languages etc As a result it is recommended that the following issues be investigated with the involvement of the users themselves e Of the wide range of specialist non eye control software that is already successfully being used by many people with disabilities for access and communication find out which can be adapted effectively for eye control e g The Grid SAW This will enable COGAIN partners to e Make a comparison of how effectively both the existing range of software specifically designed for eye control and the adapted specialist software compare in terms of their efficacy with eye control systems As aresult on behalf of the Users WP3 will be able to e Recommend modifications that could be made to the current range of software that can or could be used for eye control so that it meets as many of the needs of as many existing and potential users as possible As the above information is acquired to enable users to make an informed choice of which software to use for eye control it is recommended that e A matrix should be set up on the COGAIN website relating to
158. ronic eye type was developed at Delft University of Technology ten Kate et al 1980 ten Kate et al 1983 84 A mirror prism communicator consisted of a letter board containing prisms and angled mirrors that reflected the user s eye positions in a straight line from the area s he was looking at Two versions were developed one with a circular arrangement of letters and one with a column arrangement A freely available version of a gaze typing board is in regular use among ALS patients It consists of a matrix arrangement of letters written on a piece of paper Figure 8 1 First the communication partner asks the user which row the letter is located in First second third and when the target row is mentioned the user looks upwards Then the communication partner asks if it is for example m n 0o p q r and again the user looks upward when the right letter gets mentioned The communication partner can make a guess in order to complete a word or sentence once the user has begun spelling If this guess is correct the user can look upward to confirm If it is wrong a downward look tells the communication partner to go on Some Figure 8 1 ALS patient Birger Bergmann prefer frowning instead of looking up and down Jeppesen et al Jeppesen communicates by looking upwards 2004 when his wife reads the letter he has in mind The advantage of this system is that it may be used everywhere and it costs nothin
159. s effort greater choice more control e Her head control is good but becomes unreliable when she is trying to press a switch 20 10 2005 30 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 Interaction e From the limited testing I have done potential for increasing my access speed especially when my body gets tired However there were certain reservations expressed These included e Potential difficulties in ensuring the correct positioning of the eye control device e Outdoor use They were also asked specifically about the potential benefits of writing using eye control Of the 8 potential eye control users who expressed an opinion 2 thought it would be beneficial 4 thought it would be very beneficial and one thought it would be not very beneficial he already accesses the computer effectively with mouse and keyboard Their reasons were related to enabling greater choice ease of use independence and speed Comments included e It would allow her to have complete flexibility over what she wants to say e It is likely to be easier quicker and more accurate Similarly positive comments were made about using eye control for all standard PC applications such as Internet access email leisure software etc Against these potential benefits however there were reservations expressed by the some of the professionals in relation to reliability and or sa
160. s full eye contact The non speaking person is first asked to look at the block of letters that contains the letter first letter of the word they wish to communicate They then look at the coloured spot that corresponds with the colour of the letter The conversation partner may be able to guess the word or the non speaking person can carry on to the next letter In this way complex conversations can be held 20 10 2005 21 106 C COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 Figure 4 6 Close up of Michael s eyes Despite his nystagmus the Tobii system managed to provide a functional calibration on a specific occasion The challenge now is to enable him to achieve this on every occasion Before his stroke Michael was very active and enjoyed a wide range of leisure pursuits Despite the stroke he remains a very intelligent man with an excellent sense of humour He would like to be able to access the computer quickly and efficiently in order to communicate socially and assist with his wife s business However at present his only form of access to technology is via switches This he finds very slow and frustrating and would very much like to use eye control as a quicker and easier method if at all possible Because of the stroke Michael has a certain amount of difficulty with head control In addition he has nystagmus which means that he cannot fix his gaze in the same way that most other people can Both o
161. s what I want B NT Memory Used Caps Lock x mmm syne mmm Total Speech LS Pahoa al RAINBOW DRESS UP TOY MAN PLAYCARS jjJiGO FISH 1 TEE CHOOSE ACTIVITY DELETE LAST SELECTION CLEAR DISPLAY Figure 9 5 A Minspeak application enabling the user to combine symbols on a fixed overlay to be able to produce thousands of words and phrases Indeed many Minspeak users are able to produce text quicker than users with a traditional text plus predictor application For this reason users need to have a choice of whether they use symbols or text in their eye control grids whether for writing and or social communication 20 10 2005 71 106 Communication by Gaze Communication by Gaze Interaction COGAIN IST 2003 511598 interaction A wide choice of text styles and colours e Even if a user is literate they may find it preferable or even essential to have their text output presented in a specific way For this reason there needs to be a full range of text output including a choice of fonts choice of font sizes and choice of foreground and background colours L0 Bere Aou Figure 9 6 With Write OutLoud as the word processor you can choose whichever foreground and background colour you wish along with an extensive choice of auditory support e Whether literate or not some users may find it preferable or even essential to have s
162. se the special software for e g I use Softype as an on screen keyboard or T use the Grid for writing emailing and social communication Boa Is there anything else that you d like to do using access technology but can t at the moment If Yes please answer B6b B6c and B6d below B6b If you answered Yes to B6a What sorts of things would you like to do on the computer that you can t do at the moment e g Environmental control i e control television radio lights etc B6c If you answered Yes to B6a Why can t you do these things you d like to at the moment e g I don t have the right software B6d If you answered Yes to B6a Why would it be useful to be able to do each of these things you d like to B7a Do you write word process with your access method Qf Yes please answer B7b and B7c below If No please explain why not B7b If you answered Yes to B7a Which software do you use B7c If you answered Yes to B7a About how many words per minute can you type with your access method 20 10 2005 93 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction B8 B9 If you were able to use eye control instead of your current access method how do you think it would compare Not as good better much better If you think it would be better much better than your current access method why do you think this i e Is it qu
163. sed for social communication the user is likely to want to use it outdoors on the move as well as indoors While systems like the LC Eyegaze can be mounted to a wheelchair and move around with the user they are not designed for outdoor use With the Quick Glance II system as with other control systems the user s eye must be kept within in the camera s field of view This is about 6 by 6 cm for models 2B and 2S With the 2SH model however the area of Allowable Head Movement is about 10 x 10 cm for model This model therefore would be the one to try for someone with involuntary head movement because they would stand a better chance of keeping Figure 3 9 Quick Glance II maintaining the camera s view of their eye movement 20 10 2005 14 106 Communication by Gaze Interaction COGAIN IST 2003 511598 As with all eye control systems however the only way to evaluate their efficacy is to carry out trials with a range of users to find out how they work in practice For this reason an important aim of COGAIN on behalf of end users is to collaborate with them in order to provide information to assist them in making well informed choices This information will relate to the wide range of eye control systems available in terms of usability issues the software that is already being used for eye control and the software that could potentially be used for eye control In this way we can provide support to end users and those who
164. support them in answering such questions as this from the questionnaire of a parent of a child who she would like to consider eye control for How do we find out about the suitability of different products It is anticipated that Work package 3 in collaboration with COGAIN partners will help to provide some answers 3 2 Information from stakeholders One of the aims of COGAIN Work Package 3 User Requirements is to trial software applications developed under COGAIN To do this there is a need to find out where the existing users are Establishing direct contact with those who are already using Eye Control technology in any significant numbers is not a quick and easy process but one that will evolve over time It will necessitate for example raising awareness and gaining the trust and support of a range of manufacturers suppliers charities etc Publicity for COGAIN too through magazine and journal articles etc is an essential part of the awareness raising process and this process is already well underway throughout the COGAIN Project It is anticipated that the impact of these efforts in terms of establishing direct contact with a community of existing eye control users will be felt increasingly as the project progresses In the meantime though the numbers of existing long term eye control users we have established direct contact with is small what we have learnt from them is both rich and informative It provides a revealing insight
165. swer either very beneficial quite beneficial not beneficial 20 10 2005 94 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Communication by Gaze Interaction B14b Please give the reason s why you think that wheelchair control using eye control would be either very beneficial quite beneficial not beneficial B15a If you were able to use the computer to control your environment e g TV Radio Telephone etc with eye control how beneficial do you think this would be to you i e please answer either very beneficial quite beneficial not beneficial B15b Please give the reason s why you think that environmental control using eye control would be either very beneficial quite beneficial not beneficial B16a B16b B17 If you were able to use the computer to communicate socially e g through speech output using eye control technology how beneficial do you think this would be to you i e please answer either very beneficial quite beneficial not beneficial Please give the reason s why you think that social communication would be either very beneficial quite beneficial not beneficial Is there anything else that you would like to use eye control for If so what and why B18 Is there anything else you d like to add about eye control technology that hasn t been covered in the questions so far Thank you 20 10 2005
166. teraction Communication by Gaze Interaction COGAIN IST 2003 511598 10 Glossary It is very important to ensure that COGAIN partners have a clear and shared understanding of the terminology we use both a in relation to the Eye Control Technology itself and b those for whom it is intended For this reason a Glossary of Terms will be provided that will reduce the likelihood of any misunderstandings It is intended that the Glossary will help to a assist clarity of thought and purpose among partners from a technical perspective and b ensure that users are involved in a dignified and appropriate way Usability Headmouse Cameramouse Stakeholder User characteristics User centred design Requirements capture User Analysis Activity Analysis Eye writing eye typing Integrated system Assistive software 20 10 2005 Usability in design is about producing products and systems that perform their function effectively efficiently safely and comfortably use diagram page 15 An infrared device that tracks head movement to control the computer e g via an on screen pointer A camera that tracks the movement of the head or a feature on the face to control the computer e g via an on screen pointer Those parties who will be influenced in a development can be said to have a stake in its use and can be termed stakeholders Apart from the end users of course there are a range of secondary users including
167. ticular condition there is a great deal of involuntary movement whenever she tries to carry out a manual task Just reaching out in order to grasp the joystick handle in the first place is in itself very difficult with her hand and arm sometimes overshooting the target It is not just hand Figure 4 1 Claire illustration of wheelchair joystick movement that has this effect Even if Claire just tries to speak this also triggers off a range of involuntary movements and this too can be tiring for her On the other hand when there are no physical demands on Claire she has learnt to sit reasonably still with comparatively little involuntary movement When a calibration was tried with Claire using a Tobii eye control system the results were encouraging The system did manage to cope with a certain amount of involuntary head movement whether forwards backwards or sideways When she tried eye writing using dwell select despite the cognitive load of concentrating on eye typing using an unfamiliar grid layout she still managed to remain comparatively still Nonetheless the targets had to remain reasonably large to maintain accuracy using the type of grid shown in Figure 4 3 Figure 4 2 Claire sitting in a relaxed position even Figure 4 3 Claire using a letter grid with larger cells though she is concentrating on eye typing to help with accuracy 20 10 2005 20 106 q COGAIN Communication by Gaze Interac
168. tion COGAIN IST 2003 511598 hi will hannah have a part Figure 4 4 Claire using two hits to select a y to write the word party This grid was simply experimental with no special facilities such as prediction etc However Claire s successful access to the cells she wished to select gave the following encouraging indications e The Tobii system was able to achieve a calibration that was accurate enough for this user who had a certain amount of involuntary head movement to accurately select a letter she required for eye typing with two hits e g selecting a cell with the letters a f and then selecting the actual letter she wanted e Even without specialist or customised software there were already strong indications that eye control might have the potential to be a quicker b less tiring and because of the sharp reduction in involuntary head movement more comfortable for Claire Case study Michael Michael is in his early 40 s and has a wife and three boys He had a severe stroke about 2 years ago and now after rehabilitation is back home He cannot speak but communicates by looking at letters on an E tran frame Figure 4 5 Michael using the E tran frame to communicate with his wife Wendy The E tran frame is a deceptively simple system of communication that utilises eye pointing The transparent frame is held between the non speaking person and their conversation partner The central window enable
169. tion Springer Verlag Byrne M D Anderson J A Douglass S and Matessa M 1999 Eye tracking the visual search of click down menus Proceedings of CHI 99 Conference on Human Factors in Computing Systems NewYork ACM pp 402 409 Calhoun Gloris L 1986 Use of eye control to select switches Proceedings of the Human Factors Society 30th Annual Meeting pp 154 158 CameraMouse Inc homepage Available at http www cameramouse com Canzler U and Kraiss K F 2004 Person Adaptive Facial Feature Analysis for an Advanced Wheelchair User Interface In Paul Drews Eds Conference on Mechatronics amp Robotics 2004 Volume Part III September 13 15 Aachen Sascha Eysoldt Verlag ISBN 3 938153 50 X pp 871 876 Chapman J E 1991 The Use of Eye Operated Computer System in Locked In Syndrome Proceedings of the Sixth Annual International Conference on Technology and Persons with Disabilities CSUN 91 Los Angeles CA 20 10 2005 78 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Charlier J Buquet C Dubus F Hugeux J P and Degroc B 1997 VISIOBOARD A new gaze command system for handicapped subjects Medical and Biological Engineering and Computing 1997 35 416 supplement D90 0S1 03 Chen Y L Tang F T Chang W H Wong M K Shih Y Y and Kuo T S 1999 The new design of an infrared controlled human computer interface for the disabled IEEE Trans Rehab Eng vol 7 Dec 199
170. tionnaire responses from existing users of eye control systems provided brief but telling examples indicating the need to pay attention to such details In response to asking whether the eye control users had any additional requirements they might have of eye control technology over and above they are achieving already one said that they would like to move from their social communication application into their eye control application for writing etc themselves Clearly the facility to move from application to another so easily achieved by Stewart with his switch access as illustrated in the case study independently was sadly lacking for the eye control user Another said they would like to Play more video games At present while existing systems will enable users to access a certain number of leisure applications this is another area that is seriously under exploited and one that is ripe for COGAIN to focus upon Finally the comment from the eye control user who said If anything I would like to make it more portable underpinned the importance of trying to make this technology as portable and effective in as many situations and settings as possible Along with plans to increase in the number of eye control applications that COGAIN plans to focus on the ability to control these applications in the widest possible range of settings goes hand in hand However good an environmental control system a user has there are only limited benefits
171. truments Headmouse Extreme to access the computer Figure 6 10 Ahmar wearing a reflective dot on his forehead with which he controls the Headmouse Extreme on the top of the computer 20 10 2005 45 106 g COGAIN Communication by Gaze Interaction COGAIN IST 2003 511598 He uses two foot switches to carry out the same tasks as the left and right mouse button With this combination of headmouse and switches therefore he is able to accurately achieve whatever he wishes to in terms of mouse control However he cannot access a real computer keyboard so he uses an on screen keyboard SofType for this purpose with prediction Please select a window to type to Figure 6 11 The Soff ype on screen keyboard with prediction that Ahmar uses to write using a headmouse With SofType he is able to directly access all of the letters of the on screen keyboard directly and accurately with one hit For this reason it was not anticipated that eye control would have anything to offer Ahmar However even though he had tried eye control as an access method for a relatively brief period of time Ahmar was extremely clear and decisive about his views on this access method compared to the headmouse Ahmar felt that when accuracy was required e g for computer games he would still prefer to use the headmouse However for word processing he would have no hesitation in switching to eye control if available This
172. ty issues as the eyes need to be used all the time for various purposes or errors could be made if the user slid down the chair or was under stress The questionnaire responses then remind us that there are a range of issues to be considered in relation to choosing an access device including reliability safety ease of use and comfort When users could access other more established devices for mobility and environmental control there was a preference for these non eye control methods such as switch access because they were regarded as more safe and reliable alternatives Two case studies follow which provide examples of the way in which users select different access methods depending on which activity they wish to carry out Case Study J ulius Julius is a young man who was seriously injured as a result of an accident in N rburg in Germany in June 2002 He had a spinal injury at C2 3 and some brain damage to the lower brain brain stem full extent unclear He had a massive open book fracture to the pelvis requiring metal plates and an abdominal cage to stabilise For the first two months of his recovery he was either unconscious or semiconscious After three months he returned to the UK He then spent some time in the Spinal Injuries section Aylesbury before going home in September 2002 Julius now has a full time job working from home using the computer and intellectually Julius is as able as he always has been
173. u think this would be i e please answer either very beneficial quite beneficial not beneficial C15b Please give the reason s why you think that environmental control using eye control would be either very beneficial quite beneficial not beneficial C16a If he she they were able to use the computer to communicate socially e g through speech output using eye control technology how beneficial do you think this would be i e please answer either very beneficial quite beneficial not beneficial C16b Please give the reason s why you think that social communication would be either very beneficial quite beneficial not beneficial C17 Is there anything else that you would like to use eye control for If so what and why C18 Is there anything you d like to add about eye control technology that hasn t been covered in the questions so far Thank you 20 10 2005 99 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Appendix B Eye Control Questionnaire Summary Introduction This is a summary of the views of a small number of professionals eye control users and potential eye control users which were gathered in order to provide information for a document that is being written relating to User Requirements for COGAIN a European Network of Excellence Project concerned with eye control technology It was never intended the responses would be used for qua
174. unately web browsing can be quite a challenge if people do not have good fine motor control Most links and buttons on web pages are small in size and if they are to be accessed through step scanning it may take a long time to hit the target Hitting the tiny scrolling buttons can be particularly annoying without some kind of zooming tool and entering data in text fields is not a trivial task either Cascades of pop up windows may invade the browser if they are not properly blocked We have not been able to find any descriptions of a web browser designed for gaze control in the literature However using a two monitor set up with a normal browser window in one of the monitors and the control functions back forward etc in the other monitor seems like a durable design task Hornof and Sato 2004 have developed a system EyeMusic for direct control of electronic music compositions using a gaze tracker and established composition software One of their main motivations was to make it possible for people with physical disabilities who interact by gaze to enjoy new opportunities for musical expression During composition the primary melody is derived from the horizontal dimension and counterpoint is derived from the vertical dimension The resulting music produces a mysterious and lyrical ambiance p 188 they believe Music production is often used in therapy to communicate emotional non verbalized feelings We imagine that some people
175. unt a range of factors relating to a wide range of usability issues 20 10 2005 47 106 Communication by Gaze Interaction COGAIN IST 2003 511598 7 What can people with disabilities currently achieve with access technology overall For many years now research and development relating to Eye Control Technology has placed great emphasis on eye writing or eye typing Whilst very important for many people with disabilities it is only one of a wide range of applications that are important for them to use For this reason this chapter provides examples of user requirements that are already being met by non eye control applications but not commonly found as yet in Eye Control applications For example as well as eye typing many of these are also designed to meet users needs for social communication environmental control and powered mobility It is intended that this information will help to a inform COGAIN partners decisions about the features and facilities to include in their own Research and Development and b provide an opportunity to consider existing software which might potentially be adapted for use with eye control It is hoped that this will help to ensure that partners do not waste time re inventing the wheel and give them the opportunity to apply their energies to capitalising on the hardware and software tools that are already available 7 1 What the literature says Access technology can enable people with
176. urone Disease Spinal Injury Multiple Sclerosis paralysis ME Cerebral Palsy SMA and Wernig Hoffman Please note that those people with disabilities who are not long term eye control users are referred to as Potential eye control users Range of software applications used Eye control users The eye control users use their systems for Word processing social communication e g Speaking Dynamically Pro emailing Internet Instant messaging games Potential eye control users The potential eye control users use their systems for Word processing 1 Dasher user social communication e g EZ Keys emailing Internet Instant messaging games activating toys powered mobility environmental control 20 10 2005 100 106 Communication by Gaze Interaction COGAIN IST 2003 511598 Additional applications that potential eye control users would like to do through the computer e Emails Internet communicate present methods are incredibly slow e Indicate choice some degree of independence e Would like to make it easier for him to access all the available technology access is exhausting for him to do very much Reasons for wanting to use additional applications both eye control and non eye control users e The reasons related primarily to the need for independence but also included privacy and to reduce the shortfall between potential and performance Current estimated typing speeds Various speeds
177. used for a range of communication and daily living activities For people with communication difficulties who move around in a wheelchair communication aids need to be positioned on a tray or attached to a special mounting arm so that the user can communicate wherever they happen to be The process of communicating using techniques other than speech is called Alternative and Augmentative Communication AAC AAC involves a strategies set to enhance a person s ability to communicate For users who cannot communicate using written language or prefer to use an alternative a symbol based system might be more appropriate Many symbol users might use a grid based communication system both in low tech form eg a communication book with symbols or high tech on a speech output communication aid The screenshot below shows the word tomato being selected and spoken out using an alternative access device e g switch or pointing device Figure 7 2 The Lightwriter a text based speech output device requiring the user to access the keys directly e g using fingers Ey SOPRA INFORNARE ay 9 farina funghi mi aiuti un 4 ASPETTARE mmpastare prosciutto pronto yaana H rH fa Figure 7 3 The Cameleon PC based communication aid fh 7 4 Example of symbol based communication for both text based and symbol based communication software COMUNICA is communication software with voice output that uses PCS symbols CO
178. user s views This is a central element of the COGAIN Project In order to support COGAIN partners in making decisions about which user requirements to meet and how to meet them it is important to put eye control into the context of disability and access technology as a whole This document therefore sets out to consider e Who is currently able to use Eye Control Technology Chapter 3 e Which new users need to be able to use Eye Control Technology who can t already Chapter 4 e What are the potential alternatives to Eye Control Chapter 5 e Usability issues what are the potential benefits of Eye Control compared to other forms of access Chapter 6 e What sorts of activities are people with disabilities currently using access technology as whole not just eye control for Chapter 7 e Ifthey can use eye control what sorts of applications are people with disabilities currently using eye control for and how effectively Chapter 8 With this information it is intended that COGAIN partners will have the background knowledge required to make pragmatic decisions about which types of applications and which accessing issues to take into account in their research and development work under COGAIN It is acknowledged that this can only be done within the context of a range of practical reasonable considerations for example e The quality and effectiveness of eye control hardware and software is of course dependent on variables suc
179. vements in eye control technology will be greater reliability and accuracy Multi modal input is also essential to enable more users to access the technology more efficiently In addition multi modal input will enable the same user to continue to use the same software and applications whilst at the same time changing their access method s depending on changes in their physical capabilities whether during a single day e g if they become tired or over a period of time e g with a progressive condition With multi modal input for example a user can change from switch access to eye control and still keep the same social communication package with all of their personalised vocabulary whichever access method they use Practical considerations must also be taken into account such as portability of the systems some weigh up to 10 Kilos as well as their appearance How do users feel for instance about the appearance of the device Do they feel comfortable with it or are they too embarrassed to use it In response to the questionnaires the users themselves expressed a need to access as many applications as possible as effectively as possible Finally it will be an important goal of COGAIN to enable as many of these aims to be realised as possible whilst at the same time taking into account such practical considerations as safety and reliability 20 10 2005 67 106 COGAIN Com jon Communication by Gaze Interaction COGAIN IST 200
180. with ALS for example would appreciate having this means expression available to cope with the strong emotional changes the progress of their decease may cause Hornof Hornof et al 2004 Hornof and Cavender 2005 has also been involved in development of EyeDraw that enables children with severe motor impairments to draw pictures with their eyes Several able bodied children have used the system and the very first time they tried the system some of them were able to produce simple shapes and configurations that could be associated to the real object Hornof and Cavender 2005 report on an evaluation of EyeDraw with disabled users They identified two major problems the lack of an eye image on the screen and EyeDraw not being in the gaze tracking system s main eye controlled 20 10 2005 59 106 Communication by Gaze Interaction COGAIN IST 2003 511598 menu The last problem would require Hornof and Cavender to modify the gaze communication software but they are reluctant to risk introducing bugs Nonetheless it is clearly important to make EyeDraw and other software for this population accessible within their current eye controlled environment they state Besides providing the personal enjoyment of drawing systems like this may also be useful for sketching during communication Some computer games may work well with gaze control LC Technology producer of a gaze tracking system state on their homepage Identify the threat
181. ws Wivik and others are well suited in combination with these high precision trackers and they have been used for decades within the Augmentative and Alternative Communication AAC community Character sets can be arranged in a traditional QWERTY order in alphabetic order or according to frequency of use Acceleration features include predictions of the most likely next characters and words or access to a dictionary of words related to a specific context such as dinnertime or shopping Environmental control Besides type to talk gaze interaction has been used for environmental control of lighting appliances television etc Some of the more expensive systems have sufficient precision to allow control of a mouse pointer in a Windows environment The screen resolution may have to be at a rather low level e g 600 x 400 pixels and larger than normal icon size may have to be applied but in general it works A zooming principle that works with normal size icons and at a higher resolution e g 1024 x 768 has also been invented by one of the manufacturers Lankford 2000 There are only a few reports on gaze control of wheelchairs e g Roberts et al 1999 Matsumo et al 2001 achieved a precision of 3 degrees with their wheelchair mounted tracking system They were inspired by the fact that a person often looks in the direction of a next move when walking or driving so this may be a natural and intuitive way of contro
182. ymbols or pictures displayed instead of text They will in turn require a choice of size colour background etc for their on screen symbol displays A range of speech facilities e If using eye control for writing or generating symbols the user needs to have the option of auditory feedback With letter by letter and or word by word auditory feedback for example for many users this option would reduce or even make it unnecessary to check what is being written Without the need to check what has been written visually this would a make the production of text quicker and b overcome any difficulties related to the so called Midas Touch e If using eye control for social communication a range of speech facilities would be essential For example users need to have the facility to choose options relating to for example the speed of spoken output sex and age of the synthesised voice depending on their personal preference A choice of languages Clearly there is a need for the eye control software to support as many users globally as possible Therefore there is a need to provide an option to use as many languages as possible Dasher for example is available for use with over 80 languages Summary of software requirements If we look at a typical eye writing program in relation to a typical example of a widely available PC based assistive technology application such as The Grid it is interesting to consider how they co

D3.1 User requirements report with observations of

Contents

Download Pdf Manuals

Related Search

Related Contents