User Modelling & Simulation Tool - Cambridge Engineering Design
Contents
1. o o SEVENTH FRAMEWORK FROGEAMME GUIDE Gentle User Interfaces for Elderly People Project funding scheme STREP Work programme challenge FP7 ICT 2009 7 2 Grant agreement no 248893 Date of Annex I 4 November 2009 Project start date 1 February 2010 Project end date 31 January 2013 User Modelling amp Simulation Tool Document type Deliverable Distribution level Confidential tasks Deliverable due dat O Z OOOO Actual submission date o lt Authors and affiliations P Biswas CAM P Langdon CAM A Torquet TC L Almeida CCG J Cohello FFCUL G Heinrich VSX P Hamisu Fraunhofer Responsible editor P Biswas CAM reviewed by G Heinrich VSX M F Gonzalez ING and L Frid ING Keywords User Model Human Computer Interaction Cognitive Modelling Interface optimization Abstract This document describes the simulator used in the project and its application in developing interfaces Avatar and adaptation algorithm GUIDE application developers used the simulator either to confirm their design or make necessary corrections to make it usable by elderly and mild to moderate visual and motor impaired users The report also discusses the implication and limitations of simulation and user modeling Virtual User Simulator Executive Summary Elderly and disabled people can be hugely benefited through the advancement of modern electronic device2. cccceeeeeeeee cence eeeeseeeeesenseneeeeeneeneseesensansonsoneonenes 19 4 1 User IMILANZATION AD DICAUION sorsia 19 4 1 1 Visual impairment simulation 25220 5ccccdecccctesnetesensdaccecanscdaaedbasacamenacadoacacaneenenadaganncensieancaanbensdeoasoonaseeners 19 4 1 2 Motor impairment simulation 0 eecccecce cece a eeec eee eeeeae cece eeeeeeese esse eeseeeeseeeseeeeeeeesaeeseeetaueeseeeseeeeaeeeaes 20 4 2 Media ACCESS Application wicsicscovssiccscscdangeesatasdadestacervsveridvaasss ceerseanduassaepeevacivousuadendoees 22 4 2 1 Visual impairment simulation 2 0 0 ccc cece cc eeeceeec asec ce eeceeee ee eeeeeese sees esse eeeeeeeae ease eeseeesaeeeseeeseeeaeeeaeeegs 22 4 2 2 Motor impairment simulation cccccecccceecesceceeeceeeceuceceeeceueceueecuecsueceueesuessueeeeesauessueesueeseeesageeaes 24 4 3 Tele Learning Application cccccccccccecsecseceeceeceeceeeeeeeeeeceeseeseeseeseseeseeeeeseeeseeseeeeees 29 4 4 Video Conferencing amp Home Automation Applications ccccccccsecseeeeeeeeeeeeeeeeeees 27 3 Avatar desio e a e EEE E EE Eai 29 5 1 Visual impairment simulation based on close up representation ccceceeeeeee ees 29 5 2 Visual impairment simulation based on medium SNO ccceceeeeeeeceeeeteeeeeeeeeeeeees 30 6 Adaptation algorithm CeSIQN ccccccceseeseeeeceeeeeeneeeeeeeeeeseesensensensensenseeeseesensensensoneoness 32 r A ON e E E E A E E 34 O VAO A E E E
3. M e aa pearen Perceptual Cognitive gt Mobility gt _ ber aul ae ae ra Ae Sco ie fl Y ee fl lt Disease gt F Loy n EJA we oe oe r j ki a i pie a f 1 aan i f Z ooa an en a ee Disease ETA C Name Severity a no r i n aii j oe ef GED f f eee ete 8 Tremor creas Gama Gaver f _ 4 Tremor r ai e e p a s gt 10 CEED STM Short Term Memory IQ Intelligent Quotient EIQ Emotional Intelligent Quotient Figure 3 2 User ontology The visual perception model Biswas and Robinson 2009 simulates the phenomenon of visual perception like focusing and shifting attention We have investigated eye gaze patterns using a Tobii X120 eye tracker of people with and without visual impairment The model uses a back propagation neural network to predict eye gaze fixation points and can also simulate the effects of different visual impairments like Macular Degeneration Retinopathy and so on using image processing algorithms Figure 3 3 shows the actual and predicted eye movement paths green line for actual black line for predicted and points of eye colour blindness Diabetic 10 Virtual User Simulator gaze fixations overlapping green circles during a visual search task The figure shows the prediction for a protanope a type of colour blindness participant and so the right hand figure is different from the left hand one as the effect of protanopia w
4. Now you can repeat steps 4 to 14 on this particular user Instructions to run the Visual Impairment Simulator 3 3 2 1 Basic operations 1 2 ploy OL 10 11 12 Double click on the file Vl exe The first tab allows you to control severity of different diseases for three different devices Select a disease and change the level of severity using the slider Try to be within the quarter of the full range otherwise it may take long to run the simulator if the machine is slow Select the second Tab labeled Visual Parameters In this tab you can change the level of visual parameters of the user Select the third tab labeled as Image Parameters In this tab you can see the effect of visual impairment in terms of image processing algorithms and run different image processing algorithms to manually simulate the effect of impairment Now Click on the Demonstration button at the bottom right corner Select an image file in the folder GUIDEScreenshots You can also store any screenshot of your application as a 24bit Bitmap bmp file at 1280 x 800 resolution and simulate effect of impairment on it Wait until the image appears in the screen Double Click at any point on the image The simulation will show you the perception of the visually impaired user 3 3 2 2 Advanced operations 1 2 3 4 Click on load button on bottom left Select an xml file in the folder Profiles It will load profile
5. While most developers have a layman s understanding of the potential impact that a certain design has on a disability simulation allows quantifying and concretising this For the example of the TL application this became clear in two design adaptations that will be introduced in the next version developed 4 4 Video Conferencing amp Home Automation Applications We have undertaken similar tests as in the previous sections for Video Conferencing and Home Automation applications We investigate effects of mild to moderate visual and motor impairments It can be seen from figure 4 9 below that the fonts are sufficiently large to accommodate slight visual acuity loss for aging or disease like Myopia or distorted vision due to disease like macular degeneration The colour combination is selected as white in blue background so that it can remain legible for dichromatic colour blindness The buttons are sufficiently large to accommodate random cursor movement during homing on a target using a pointing device 27 Virtual User Simulator ial eel yi mete Simulation of mild visual acuity loss Simulation of severe visual acuity loss p UNATRSII Y Ol 1 he O ser i jb TE UNSTVERSITY OF CAMBRIDGE CAMBRIDGE simulating early stage of dry macular Simulating early stage of wet macular deg eneration and cursor trace deg eneration and cursor trace Home Automation Home Automation
6. and the user would still be able to figure out what was written on the welcome message However this user would need some help with complementary output modalities figure 4 1e e User with moderate visual impairments affected by color blindness and moderate diabetic retinopathy Image is easy to understand for the user with the discoloration 19 Virtual User Simulator caused by the impairments not being sufficient to prevent the user from using it without any complementary output modality figure 4 1f GUIDE has detected a hew useri A pew eT will be created a User Initialization application as perceived by people having no visual impairment GUIDE has detected a new veer A new profile well be eraatad c User Initialization application as perceived by people having moderate visual impairment aa h iin to o ee ee eo y ee mp Hil e User Initialization application as perceived by people having moderate wet Macular Degeneration GUIDE has detocted a new weer Anew profile will be crab b User Initialization application as perceived by people having mild visual impairment lt GUIDE has detoctad a new t a A new profile well be created d User Initialization application as perceived by people having mild Macular Degeneration GUE has detected a rane ee amp new jolie wii pe created f User Initialization application as perceived by people having colour blindness
7. 4 3c d the font size has been increased and in the third series figure 4 3e f the Arial font has been replaced by Tiresias font for the middle text a 1st version of the scanning operation with a Mild b 1st version of the scanning operation with a Visual Impaired Moderate MI profile Moderate Visual Impaired Moderate MI profile Sr aniiiing Opadat iim je c 2nd version of the scanning operation with a profile d 2nd version of the scanning operation with a profile Mild Visual Impaired Moderate MI profile Moderate Visual Impaired Moderate MI profile 22 Virtual User Simulator Ea hey aa mi a Vir tem Bae junii Pied d ce ee L he E hi hoes re Fm bee ee Fo Gume e 3rd version of the scanning operation with a profile f 3rd version of the scanning operation with a profile Mild Visual Impaired Moderate MI profile Moderate Visual Impaired Moderate MI profile Figure 4 3 Affect of visual impairment of Media Access application and corresponding design modification Some more tests were performed with the Visual Impairment Simulator to verify if the contrast intended on the background and buttons of some parts of the Media Access application was still present with users having three different kind of dichromatic colour blindness Figure 4 4 i BDG On Fi i Hrd Haddin ji jail Ti E is sij IBITI Ta g 7 BBC Tiwee 7 fit Three a GUIDE TV Info with
8. 43 15 40 41 Virtual User Simulator 1 Introduction The World Health Organisation WHO states that the number of people aged 60 and over will be 1 2 billion by 2025 and 2 billion by 2050 WHO website 2009 The very old age 80 is the fastest growing population group in the developed world Many of these elderly people have disabilities which make it difficult for them to use computers The definition of the term Disability differs across countries and cultures but the World Bank estimates a rate of 10 12 of population worldwide having a condition that inhibits their use of standard computer systems World Bank website 2009 However existing design practices often isolate elderly or disabled users by considering them as users with special needs and do not consider their problems during the design phase Later they try to solve the problem by providing a few accessibility features Considering any part of the society as special can never solve the accessibility problems of interactive systems Unfortunately existing accessibility guidelines are also not adequate to analyze the effects of impairment on interaction with devices So designers should consider the range of abilities of users from the early design process so that any application they develop can either adapt to users with a wide range of abilities or specify the minimum capability of users it requires In this document we have presented a simulation system that helps
9. 6 2 1 Procedure We selected four screenshots and a representative task for the validation study To generalize the study none of the screenshots in the validation study have been used in the simulation study To test the escape strategy we selected the task in such a way that the cursor has to cross neighbouring button in reaching the target Figure 6 6 shows the interfaces used in the study The task involves pressing one button at each screen marked 1 2 3 4 in the figure and the dotted line shows the ideal pointing path If the user clicks on a wrong target the screen does not change and the event is registered as a missed click We investigated two conditions pointing with and without gravity well 36 Virtual User Simulator Mens eel Store a Number Delete a Number Resize GUIDY Go Back Go Back G0 Back e Settings IOE m Browser 3 Virtual User Simulator Figure 6 6 Interfaces and task used in the study All users were initially trained about using the system and they have no difficulty in reading the button caption 6 2 2 Material We have conducted the study on a HP Laptop with 12 5 screen size We have tested the system with three different devices e Mouse and e Trackball involving people with mobility impairment 38 Virtual User Simulator e Wiimote 2011 involving able bodied users to impose situational impairment figure 6 7 The Wiimote is connected with l
10. Mobility Impaired Moderate VI profile Moderate VI profile geming Opani gaming Opani fall ina TTT em il ia ean PTT iced tonne ami tome ami Wah Phe ar pore pred aj Kai nearne Wai in IDE Pp ee c 2nd version of the scanning operation with a Mild gq 2nd version of the scanning operation with a Mobility Impaired Moderate VI profile Severe Mobility Impaired Moderate VI profile 24 Virtual User Simulator Branning Oper ne bn Soanrirey Diper pike inn ie HT T ema Thee mi bk mi ae oa 7 pm dhas abt hbr Ciim bem brel mie boae ee E ia bes e 3rd version of the scanning operation with a Mild f 3rd version of the scanning operation with a Severe Mobility Impaired Moderate VI profile Mobility Impaired Moderate VI profile Figure 4 5 Affect of motor impairment on Media Access application 4 3 Tele Learning Application The TeleLearning TL application is built around two central tasks Users select a lecture from a set of subscriptions and watch the combined video and slide content of the lecture Exemplarily two UI designs of the currently developed TL application are studied as shown in figure 4 6 The content is from a congress and for each lecture the title the presenter and some short description are given left The user can select the video by scrolling up and down in the list Selection is done by pressing on the example slide or the description text right of it The selected l
11. a user having normal colour GUIDE TV Info with a user having colour vision blindness Deuteranopia Cerasi Progen 2 BS Tee eee es ee BES Three 2 Be Tea Horags Matas tjian ee Mim ir 3 BBC Three c GUIDE TV Info with a user having colour d GUIDE TV Info with a user having colour blindness Protanopia blindness Tritanopia Figure 4 4 Affect of colour blindness of Media Access application 23 Virtual User Simulator 4 2 2 Motor impairment simulation some tests were performed with the Mobility Impairment Simulator to verify if the location on size of buttons being displayed as arrows or real buttons are easily reachable even by a user suffering from mobility impairments The tests were done for three different profiles Mild Moderate and Severe Mobility Impairment but only the two first were stored as the Moderate profile was not representative enough The first series figure 4 5a b of screenshots is dedicated to a specific display of clickable elements retrieved at several places in the Media Access application the arrows here displayed in the GUIDE TV Info part The two left series of screenshots show some adjustment made on the Scanning operation after the first check regarding the spacing between two buttons figure 4 5 c f 2 hi Two A l hac Two E 1E Hasan Hap Tmj ee 1 A r een Ha mmj oer 1G Pe a GUIDE TV Info with a Mild Mobility Impaired b GUIDE TV Info with a Severe
12. and install the simulator and also describes its applications in optimizing interfaces designing Avatar and run time adaptation algorithm The task reported in this document will be continued through task 13 8 User simulation Components till month 24 and a more elaborate report D3 2 will be submitted upon completion Virtual User Simulator Prototype Systems Best Alternative New Systems User Testing Interaction Patterns Existing Systems Figure 1 1 Use of the simulator 1 2 Work plan task and partner contribution Table 1 1 shows contributions by the different GUIDE partners according to the responsibilities in each task covered in this deliverable Table 1 1 Work plan Partners Effort T6 4 VSX e Optimized interface layout of Tele Learning Application T6 1 e Optimized interface layout of Video Conferencing and Home PM13 Automation Application PM18 T3 8 CAM e Implemented User Models in virtual user simulator ee T7 e Simulated and validated an adaptation algorithm FFCUL e Optimized interface layout of User Initialization Application 4 PM18 e Optimized interface layout of Media Access Application 3 6 CG e Optimized design of Avatar PM18 1 3 Related documents This document is related to the detailed user manual delivered with the simulator prototype during first review It also refers to the clusters in D7 1 and conforms to the requirements pointed in D2 1 The applica
13. and mild visual acuity loss Figure 4 1 Affect of visual impairment on user initialization application As it can be seen the choice for having a really simple UIA interface can help in the beginning of the interaction for a big range of users with visual impairments Audio output can also be provided from the beginning of the interaction helping users with severe visual impairments and not disrupting the interaction also for any other user 4 1 2 Motor impairment simulation The second use case concerns how users with different motor impairments can interact with the UIA interface in order to perform a simple step by step testing until the system understands his 20 Virtual User Simulator or her impairments and capabilities and performs adaptation The UIA should be very simple so that the biggest range of users independently or their impairments can use it and in this way to be possible for everyone to make an entrance in GUIDE i e everyone should be capable of using it so that GUIDE can learn the capabilities of every single user Male user 81 years old suffering mild motor impairments slightly affected by Polio disease User is able to interact and select any of the buttons without difficulty and almost without deviations from the targets figure 4 2b Female user 75 years old suffering moderate motor impairments slight moderately affected by Polio s and Cerebral palsy User is able to interact and select any of the but
14. and severity of use impairment of users 13 24 connected Shan vadine i eneg ti a Fo cronig aiei lt CREEN hype mysu amber welcome bocokr WHOTE gt diam pte Mie ate Wu Pacts ontas A bgel WHITE oare BLE heare DLD ie E whith eo Faighte LO fs SCREEN lt CREEM Hypu wua mbai Veia bocaor iHITE giem trpe abel text Wiel Df fontsze 20 beyo WHITE gcom BLUE Aol SL 6 0 ya dth L0 heht L gt diem type babbo berie CIECIA fontaines boodor WHITE gcokr BLUE Hcokr e GOL 2 y GiT idth iay ighi A6 y gt ctem tpz bettor text TE ionwasrez a0 bhgcobor WHITE fodor ELE hodo e eS pS dhA L haughty is atem Type battor terte DEPTE Tontane i bocclor WHITE focokr BLUE Heokr GLD ae gi yG idh i0 aighi P6 i gt atem pes bettor texst MUBDGA fontur F booolor WHITE coas BLUE Hoora GOLUT 9B ye dth LT height LA fa Abem Type berbbom baite WIARE ntaa 4H bgcokr WHITE Medor e ALARE hicdor GO LE 2D 5S p 6S idie height a4 gt atem bpas betton ta WIARE onire H bgcolor WHITE fgodor Ue hice SL ee Se edt LR heht fs dem pes betimi berie WIARE fontsite O bhgcokr WHITE cdo HARE hedo DD ce eA dihi aight ga i SCREEN Dial aeonhga Figure 3 7 Screenshot of an interface with sample AUI From a designers perspective he can record the interface screenshots and the AUls of an application and then directly feed them to the simulator If the designers want to simulate interaction for a task
15. contrast on the text background has decreased making it a little more difficult to read figure 4 1b However the UI serves its purposes being well perceivable by users with this level of visual impairment because the message avatar and ok button are all well readable e User with moderate visual impairments particularly affected by slight wet Macular Degeneration and moderate acuity loss and consequently some distortion The Avatar is even lighter than on the image before making it harder to perceive and the text message is a little bit blurred figure 4 1c However the UI still serves its purpose being well perceivable by users with this level of visual impairment still there is no problem reading the message the avatar or the button e User with severe visual impairments particularly affected by moderate dry Macular Degeneration moderate acuity loss and slight peripheral visual field loss and consequently some distortion and blur There is no big difference to the previous image just a slight aggravating of the blur and font image definition Text button and Avatar are still well perceivable to this user figure 4 1d e User with severe visual impairments affected by a more severe acuity loss and macular degeneration than the previous one Image is now more difficult to understand and there is some difficulty reading the message or the letters on the button Still an image of a face is perceivable but not with great precision
16. he can also record the cursor positions from the run time system GUIDE Core and feed it to the simulator The simulator provides probable visual and auditory perception and movement trace of a user The designer can modify his design based on that This prediction also in turn helps to develop the user model part of the GUIDE profile which will be explained in more detail in D4 1 and D5 1 16 Virtual User Simulator Applications Run Time System GUIDE Core Screenshots Cursor amp positions GUIDE AUlIs Profile Simulator Figure 3 8 Relating Simulator to GUIDE Framework 3 3 Software manual The simulator is designed as a tool to help designers in visualizing understanding and measuring effect of age and impairment on their designs At present it demonstrates the simulation of visual and mobility impairment only It can be downloaded from http www quide project eu index php mainltem Publications amp subltem Project Deliverables amp paqgeNumber 1 amp item 22 To run the software please follow the steps below The software runs on a Windows XP or Windows 7 operating system You may need to manually register one or more library files depending on your machine configuration Please use the command For a in lt Your Drive Name gt Simulator Dependencies ocx do regsvr32 s All the library files can be found in the Dependencies folder 3 3 1 Instructions to run the Mobility Impairment Simulator 3 3 1 1 Ba
17. of age and gender on hand strength and the system can show the effects of Cerebral Palsy or Parkinson s disease for different age group and gender The simulator works in the following three steps 1 While a task is undertaken by participants a monitor program records the interaction This monitor program records a A list of key presses and mouse clicks operations b A sequence of bitmap images of the interfaces low level snapshot c Locations of windows icons buttons and other controls in the screen high level snapshot 2 Initially the cognitive model analyzes the task and produces a list of atomic tasks detailed task specification 3 If an atomic task involves perception the perception model operates on the event list and the sequence of bitmap images Similarly if an atomic task involves movement the motor behaviour model operates on the event list and the high level snapshot In the remaining sections of the paper we demonstrate the use of the simulator through an icon searching and menu selection task In the first application the simulation accurately predicts performance of users with visual and mobility impairment In the second case the simulator is used to identify the accessibility problems of menus and thus redesign a menu selection interface 3 2 Integration of simulator to GUIDE framework This section explains how GUIDE developers can use the simulator in their application development process As a startin
18. output devices to be used by a particular user and sets parameters for an interface The User model simulates the interaction patterns of users for undertaking a task analysed by the task model under the configuration set by the interface model It uses the sequence of phases defined by Model Human Processor The perception model simulates the visual perception of interface objects It is based on the theories of visual attention The cognitive model determines an action to accomplish the current task It is more detailed than the GOMS model but not as complex as other cognitive architectures The motor behaviour model predicts the completion time and possible interaction patterns for performing that action It is based on statistical analysis of screen navigation paths of disabled users Virtual User Simulator i oy a Display Model Cognitive Model Application l Input Motor Model Model Behaviour ae Model Environment Model Device Model User Model Figure 3 1 Architecture of the Simulator The details about users are store in xml format in the user profile following the ontology shown in figure 3 2 below The ontology stores demographic detail of users like age and sex and divide the functional abilities in perception cognition and motor action The perception cognitive and motor behaviour models takes input from the respective functional abilities of users User gt Age net i ee ee
19. to develop inclusive systems by o Helping designers in understanding the problems faced by elderly people with different range of abilities knowledge and skill o Providing designers a tool to make interactive systems inclusive o Assisting designers in evaluating systems with respect to elderly people with a wide range of abilities o Modifying the design process of interactive system to o evaluate the scope of it with respect to the range of abilities of users o investigate the possibilities of adaptation of the interfaces for catering users with different ranges of abilities The simulator can predict the likely interaction patterns when undertaking a task using a variety of input devices and estimate the time to complete the task in the presence of different disabilities and for different levels of skill Figure 1 1 shows the intended use of the simulator We aim to help evaluate existing systems and different design alternatives with respect to many types of disability The evaluation process would be used to select a particular interface which can then be validated by a formal user trial The user trials also provide feedback to the models to increase their accuracy As each alternative design does not need to be evaluated by a user trial it will reduce the development time significantly 1 1 Purpose This report presents a prototype of the simulator and its applications in GUIDE development process It contains instructions to download
20. 0 affective state social interaction consciousness in rational and social band Newell 1990 and so on Simulations of each individual band have their own implications and limitations However the cognitive band is particularly important since models working in this band are technically feasible experimentally verifiable and practically usable Research in computational psychology and more recently in cognitive architectures supports this claim We have added a new dimension in cognitive modelling by including users with special needs 42 Virtual User Simulator TIME SCALE OF HUMAN ACTION a B Task Tak RATIORIAL Task Unit Task Operations COOMTTTVE Delberate Act Neural Circuit Neuron BIOLOGICAL Organele Figure 7 1 Timescale of human action adapted from Newell 1990 8 Conclusion This report presents the virtual simulator prototype and explains its role in application development process We discussed detail on the simulator and using it to optimize interfaces designing avatar and adaptation algorithm In general designers found the simulator useful to confirm or correct their designs to cater elderly and disabled users We shall further continue this work and report that in deliverable D3 2 at the end of month 24 References e Anderson J R and Lebiere C The Atomic Components of Thought Hillsdale NJ USA Lawrence Erlbaum Associates 1998 e Benyon D Murray D Applying User Modeling to Human Compu
21. 1990 and so on and has also been used to simulate human machine interaction to both explain and predict interaction behaviour A simplified view of these cognitive architectures is known as the GOMS model John and Kieras 1996 and still now is most widely used in human computer interaction However the GOMS Goal Operator Model Selection family of HCI models e g KLM CMN GOMS CPM GOMS is mainly suitable for modelling the optimal 8 Virtual User Simulator skilled behaviour of users while models developed using cognitive architectures consider the uncertainty of human behaviour in detail but have not been widely adopted for simulating HCI as their use demands a detailed knowledge of psychology 3 The simulator 3 1 Description To address the limitations of existing user modelling systems we have developed the simulator Biswas Robinson and Langdon 2011 as shown in figure 3 1 It consists of the following modules The Environment model contains a representation of an application and context of use It consists of e The Application model containing a representation of interface layout and application states e The Task model representing the current task undertaken by a user that will be simulated by breaking it up into a set of simple atomic tasks following the KLM model e The Context model representing the context of use like background noise illumination and so on The Device model decides the type of input and
22. 4 Avatar medium shoot display The simulation of users with mild visual impairments is depicted in Figure 5 5 The nonverbal communication transmitted by the Avatar body is neatly recognizable Figure 5 5 Display with mild visual impairments Both the user with moderate and the user with severe visual impairments are still able to perceive well the nonverbal communication transmitted by the Avatar s body Figure 5 6 As in the first test it is difficult to perceive the Avatar facial expression It is evident that in the case of severe impairments the black areas Screen areas not perceptible by visual impaired user could also hide the pointing gesture depending on its placement on screen The simulation of still images does not yet allow deciding whether this possible issue requires further adaptation measures as recognition here is probably further facilitated by the perception of the movement 31 Virtual User Simulator of the arm A close up of the arm and hand as further adaptation would be technically feasible but requires additional camera steering algorithms and user tests Figure 5 6 Display with moderate visual impairments left and display with severe visual impairments right In general even when users have moderate or severe visual impairments the silhouette of an image of the Avatar is easily perceptible this should allow them to easily recognize the moving Avatar as well This finding is also important becaus
23. A 36 6 2 1 POCE OO aa E E E ee ee ee 36 6 2 2 AILE A EE E E A E T A E T AE A EE E 38 6 2 3 EARN IO GM o recnici E EEEE EA EE EEEE 39 6 2 4 FS SU a E S 39 6 2 5 DE USSON e R E E ucumeedmecieiGencauatedaercteoncsseendl 41 7 implications and limitations of simulation amp user modelling c scceeeeeeeeeeeeees 42 5 CONCUSSION siruina EE E Erai 43 BS CO ON COS iara E 43 Virtual User Simulator List of Figures Figure 2 1 Simplistic view of a user model Figure 3 1 Architecture of the Simulator Figure 3 2 User ontology Figure 3 3 Eye movement trajectory for a user with colour blindness Figure 3 4 Sequence of events in an interaction Figure 3 5 Mouse movement trajectory for a user with cerebral palsy Figure 3 6 A few interfaces of a prototype of the toolbox Figure 3 7 Screenshot of an interface with sample AUI Figure 3 8 Relating Simulator to GUIDE Framework Figure 4 1 Affect of visual impairment on user initialization application Figure 4 2 Affect of motor impairment on user initialization application Figure 4 3 Affect of visual impairment of Media Access application and corresponding design modification Figure 4 4 Affect of colour blindness of Media Access application Figure 4 5 Affect of motor impairment on Media Access application Figure 4 6 Overview lecture list and lecture presentation in the TL application Figure 4 7 Overview lecture list seen through the eyes of person with medium strong visual
24. aptop using the Bluetooth interface The able bodied users never use Wiimote as a pointing device before though they were trained to do it before the study The Wiimote works like a gesture based joystick interface which responds to the tilt of the wrist Figure 6 7 Using gesture based interface Wiimote 6 2 3 Participants We have collected data from 7 motor impaired users and 6 able bodied people table 6 1 The disabled participants were recruited from a local centre which works on treatment and rehabilitation of disabled people and they volunteered for the study The able bodied users were recruited from our institution All participants were expert computer users The motor impaired users use either mouse or a variant of track ball as their normal interaction device However we used the same device for all to keep the results consistent across participants 6 2 4 Results We have measured the time gap between the first and fourth button presses Participant P1 could not use a mouse for his tremor in fingers while participant P7 did not have time to use the Track ball Table 6 2 lists these task completion times for each participant It has been found that the average task completion time is significantly lower with the gravity well than the without it p lt 0 01 Figure 6 8 shows that the average task completion time is lower for gravity well for each device individually as well The standard deviation is higher without gravity well
25. as simulated on the input image Ge G 8 GUO NEN A O E A E A E A 8 9 A B Actual Predicted Figure 3 3 Eye movement trajectory for a user with colour blindness The auditory perception model is under development It will simulate effect of both conductive outer ear problem and sensorineural inner ear problem hearing impairment The models will be developed using frequence smearing algorithm Nejime and Moore 1997 and will be calibrated through audiogram tests The cognitive model Biswas and Robinson 2008 breaks up a high level task specification into a set of atomic tasks to be performed on the application in question The operation of it is illustrated in figure 3 4 At any stage users have a fixed policy based on the current task in hand The policy produces an action which in turn is converted into a device operation e g clicking on a button selecting a menu item and so on After application of the operation the device moves to a new state Users have to map this state to one of the state in the user space Then they again decide a new action until the goal state is achieved 11 Virtual User Simulator Action Operation Old User ecco State peration Mapping New Device State State Mapping New User State User Space Mapping Device Space Figure 3 4 Sequence of events in an interaction Besides performance simulation the model also has the ability to learn new techniques
26. d clicks with gravity well P4 also made 4 missed clicks without gravity well while rest of the participants did not make any missed click under any condition Across the devices on average mouse took least time though the difference is not significant through a one way ANOVA under both conditions with or without gravity well 6 2 5 Discussion The gravity well algorithm significantly reduces pointing time for most of the users The algorithm worked for both ordinary pointing devices like mouse and track ball and gesture based interaction device viz the Wiimote However in our trials the pointer was often attracted to a wrong target and participants need to waste time to get rid of the wrong target The wasted time 41 Virtual User Simulator was smaller in comparison to the reduction of overall pointing time but we still have scope to reduce pointing time further using algorithms like probabilistic target identification Hurst Hudson and Mankoff 2010 The study also shows the utility of the simulation in design optimization Without the simulator the user study would investigate all possible sizes of gravity well which would take a lot of time The validation study proved that design optimization through the simulation holds true in practice and can reduce system development time considerably 7 Implications and limitations of simulation amp user modelling User trials are always expensive in terms of both time and cost A design
27. detail of users Figure 3 6 A few interfaces of a prototype of the toolbox At present it supports a few types of visual and mobility impairments For both visual and mobility impairment we have developed the user interfaces in three different levels o Inthe first level figure 3 6a the system simulates different diseases o Inthe next level figure 3 6b the system simulates the effect of change in different visual functions like Visual acuity Contrast sensitivity Visual field loss and so on hand strength metrics like Grip Strength Range of Motion of forearm wrist and so on and auditory parameters like audiogram loudness and so on o In the third level figure 3 6c the system allows different image processing and digital filtering algorithms to be run such as high low band pass filtering blurring etc on input images and to set demographic detail of users The simulator can show the effects of a particular disease on visual functions and hand strength metrics and in turn their effect on interaction For example it can demonstrate how the progress of dry macular degeneration increases the number and sizes of scotoma dark spots in eyes and converts a slight peripheral visual field loss into total central vision loss Similarly it can 14 Virtual User Simulator show the perception of an elderly colourblind user or in other words the combined effect of visual acuity loss and colour blindness We have modelled the effects
28. e the emotional expressiveness of the Avatar movement can enable the disambiguation of the Avatar s facial expression which is more difficult to recognize for the visually impaired Further studies will be realized as soon as other modules of the simulator are available for example the simulation of hearing impairments which will permit refinements on the actual implementation of Avatar synthetic speech TTS and extract conclusion on how to setup TTS according to the user profile 6 Adaptation algorithm design Users with motor impairment especially hyperkinetic often have difficulty in homing on target and the resulting random movements may generate a wrong target selection For example figure 6 1 shows cursor traces on three different buttons in an interface It can be seen that if the user aims to click at the middle of the button then in spite of the random movements there is no chance of miss clicking neighbouring buttons However if the user aims to click at the edge of the button as in the case of the rightmost button labelled Disconnect the user may end up clicking on a neighbouring button in this case the right arrow key marked with dotted red circle 32 Virtual User Simulator Coment Shanng Ficture selected for veewing UNIVERSITY OF CAMBRIDGE a i Figure 6 1 An example of missed clicking We can alleviate this problem by employing an adaptation principle called gravity well Hwang e
29. ecture is presented using the setting on the right In principle the user can simply sit back and get the presentation and additionally navigation in the lecture is possible touching one of the thumbnails to select the respective slide the one in the middle is the active one touching on the large slide or by using the control bar below the speaker video aaraa Astley pe der DGNA 2010 Dr ke a aiima saab ek liam Aegean arate lave erami ge ee ee ee ot i ar LASTE nee aa we Sed ee Traim Pt aen aparews ET NETEN Op mem Ie kH Pada a Er miren aliron rrari ern rane mre WS i ei ee al hai E Loree me aman Hee r ure oe a cp a lt i serimi La ELE mi iG Faa Fi EA GASDA ea a ee rerem urs E s PEE a i a Figure 4 6 Overview lecture list and lecture presentation in the TL application Although the preparations for the use of this application in an adaptive framework have been implemented from the outset the application is not far from what would be implemented for able bodied users The simulation on this prototype was performed for several central screens that are representative of the interaction necessary in the TL application A range of four profiles was tested with mild to medium strong visual and motor impairments combined Lecture selection Using the simulators on the list view results in two findings The partition of the screen for selection is unproblematic for mild to moderate motor impairme
30. ed clicks as the movement is random there and thus lacks control Figure 3 5 Mouse movement trajectory for a user with cerebral palsy Each of the perception cognitive and motor behaviour models were calibrated and validated separately involving people with and without visual and mobility impairment The perception model was validated through an eye gaze tracking study for a visual search task We compared the correlation between actual and predicted visual search time eye gaze and also investigated the error in prediction The actual and predicted visual search time correlated statistically significantly with less than 40 error rate for more than half of the trials Biswas and Robinson 2009a The cognitive model was used to simulate interaction for first time users and it can simulate the effect of learning as well Biswas and Robinson 2008 The motor behaviour model was validated through ISO 9241 pointing task The actual and predicted movement time correlated statistically significantly with less than 40 error rate for more than half of the trials Biswas and Robinson 2009b These models do not need detailed knowledge of psychology or programming to operate They have graphical user interfaces to provide input parameters and showing output of simulation Figure 3 6 shows a few interfaces of the simulator 13 Virtual User Simulator c Interfaces to run image processing algorithms and set demographic
31. emotion transmitted by the Avatar since the facial expression remains sufficiently clear 29 Virtual User Simulator Figure 5 2 Simulation of on screen avatar perception by user with low visual impairments In contrast both the user with moderate and the user with severe visual impairments see Figure 5 3 will hardly notice the expression of the avatar Possible solutions i e adaptations at the level of the avatar component may involve the use of the hearing sense and of an emotional voice of the avatar to convey the emotions but emotional TTS is rare and has its issues increasing the resolution of the close up e g instead of using a screen fraction use the entire screen or even more difficult to tune controlling the color intensity and contrast of avatar representation Figure 5 3 Display with moderate visual impairments left and display with severe visual impairments right 5 2 Visual impairment simulation based on medium shot Another visual simulation performed was based on the Avatar display in medium shot mode see Figure 5 4 In this test we intended to understand how the impaired user perceives the non verbal communication of the Avatar body In the original presentation of the Avatar a pointing gesture is clearly visible The intention was to evaluate if this expression is perceived by all users different degrees of impairment 30 Virtual User Simulator Scanning Operation Figure 5
32. evolves through an iteration of prototypes and if each prototype is to be evaluated by a user trial the whole design process will be slowed down Buxton 2010 has also noted that While we believe strongly in user testing and iterative design each iteration of a design is expensive The effective use of such models means that we get the most out of each iteration that we do implement Additionally it often turns difficult for developer to conduct trials with users with a wide range of abilities which in turn reduces the scalability of the corresponding applications across different users A good simulation with a principled theoretical foundation can be more useful than a user trial in such cases Exploratory use of modelling can also help designers to understand the problems and requirements of users which may not always easily be found through user trials or controlled experiments This work show that it is possible to develop engineering models to simulate human computer interaction of people with a wide range of abilities and that the prediction is useful in designing and evaluating interfaces According to Allen Newell s time scale of human action figure 7 1 Newell 1990 our model works in the cognitive band and predicts activity in millisecond to second range It can not model activities outside the cognitive band like micro saccadic eye gaze movements response characteristics of different brain regions in biological band Newell 199
33. for interactions Learning can occur either offline or online The offline learning takes place when the user of the model such as an interface designer adds new states or operations to the user space The model can also learn new states and operations itself During execution whenever the model cannot map the intended action of the user into an operation permissible by the device it tries to learn a new operation To do so it first asks for instructions from outside The interface designer is provided with the information about previous current and future states and he can choose an operation on behalf of the model If the model does not get any external instructions then it searches the state transition matrix of the device space and selects an operation according to the label matching principle Rieman and Young 1996 If the label matching principle cannot return a prospective operation it randomly selects an operation that can change the device state in a favourable way It then adds this new operation to the user Space and updates the state transition matrix of the user space accordingly In the same way the model can also learn a new device state Whenever it arrives in a device state unknown to the user space it adds this new state to the user space It then selects or learns an operation that can bring the device into a state desirable to the user If it cannot reach a desirable state it simply selects or learns an operation that can bring
34. g point the following table summarizes the input requirement of the simulator and how can it be found from the GUIDE framework Table 3 1 Relating simulator to the GUIDE framework Input to the Source in GUIDE Utility in Simulation Simulator Screenshots of This can be recorded using an The visual impairment interface automatic screen recording tool or simulator takes the screenshot manually during design of the interface to simulate effect of visual impairment Locations and The GUIDE interfaces The motor sizes of targets in screen application communicate with the GUIDE core by publishing an abstract user interface AUI definition about their layout The AUI is a description of the interface layout and controls in it stored in XML format A sample AUI along with the actual interface is shown below Fig 3 7 The AUI contains relevant detail about the targets impairment simulator uses the abstract user interface AUI to know the locations and sizes of buttons and use this to calculate movement time using different input modalities 15 Virtual User Simulator Sound stream The sound stream will be captured If the application has sound either from the input microphone or the output then the hearing fusion module impairment simulator works on the output sound stream Detail of user The GUIDE profile inside GUIDE core All simulation modules need contains detail of user and context of the type
35. iS Rr i Simulation of mild motor impairment on Home Simulation of moderate motor impairment on Home Automation GUI Automation GUI Figure 4 9 Affect of visual and motor impairment on Video Conferencing amp Home Automation applications 28 Virtual User Simulator 5 Avatar design In this section we present some examples of simulator utilization for Antropomorphic avatar design specification and implementation Considering the actual status of the simulator given a user with visual impairments it enables the simulation of the impaired perception of the avatar Using the simulator it is possible to find new visual requirements or parameterization points for the avatar component like for example the avatar size on screen the color intensity and contrast the facial and body expressions among many others and these have been the main goals performing these design studies 5 1 Visual impairment simulation based on close up representation The first example of visual test was done based on the Avatar display in close up mode see Figure 5 1 This test shows how the impaired user would perceive the emotion facial expression conveyed by the avatar Scanning Operation Stact Soon OTT Cancel Figure 5 1 Screen capture showing Avatar close up mode The simulation of a user with mild visual impairments is shown in Figure 5 2 The avatar is not perceived as in the original image however it is still possible to distinguish the
36. impairment myopia and macular degeneration Figure 4 8 Motor and visual impairment simulation for Tele Learning application Figure 4 9 Affect of motor impairment on Video Conferencing amp Home Automation application Figure 5 1 Screen capture showing Avatar close up mode Figure 5 2 Simulation of on screen avatar perception by user with low visual impairments Figure 5 3 Display with moderate visual impairments left and display with severe visual impairments right Figure 5 4 Avatar medium shoot display Figure 5 5 Display with mild visual impairments Figure 5 6 Display with moderate visual impairments left and display with severe visual impairments right Figure 6 1 An example of missed clicking 10 11 11 13 14 16 17 20 21 23 23 25 25 26 2 28 29 30 30 31 31 32 33 Virtual User Simulator Figure 6 2 An example of the gravity well Figure 6 3 Effects of gravity well Figure 6 4 Wrong target selection by gravity well Figure 6 5 Escape strategy based on pointer movement characteristics Figure 6 6 Interfaces and task used in the study Figure 6 7 Using gesture based interface Wiimote Figure 6 8 Results on gravity well Figure 7 1 Timescale of human action adapted from Newell 1990 List of Tables Table 1 1 Work plan Table 3 1 Relating simulator to the GUIDE framework Table 6 1 Participants Table 6 2 Results on gravity well 33 34 35 36 38 39 40
37. indicating more random pointer movements without gravity than with it 39 Virtual User Simulator Table 6 1 Participants Participa Age Sex Impairment nts Able bodied users Cerebral attack significant tremor in whole upper body part fingers always remain folded Dystonia cannot speak cannot move fingers wheelchair user Spinal cord injury in childhood due to falling from a window wheelchair user difficulty in hand movement Cerebral Palsy reduced manual dexterity also some tremor in hand wheel chair user Reduced manual dexterity in limbs due to neurological problem wheel chair user Cerebral Palsy reduced manual dexterity wheel chair user Cerebral Palsy wheelchair user Effect of Dynamic Adaptation 15000 10000 5000 40000 35000 E 30000 25000 e With Adaptation S 20000 m Without Adaptation Q roy E O O x 14 H All Mouse TrackBall Wiimote Input Devices Figure 6 8 Results on gravity well 40 Trackball P1 P2 P3 P4 P5 P6 Virtual User Simulator Table 6 2 Results on gravity well Task Completion Time in msec With Gravity well 38361 11326 14508 8752 19281 20171 Without Gravity well 73180 11794 14758 7800 29796 25397 Difference Mouse P2 P3 P7 P4 P5 P6 Wiimote C1 C2 C3 C4 C5 C6 Average Std Dev Ttest Participant P3 and P4 made 5 misse
38. irment Decrease the font size until you Coes tt feai confortable to raced this motor impairment Decrease the tont size until you find difficult ta read ind diieu io read Dess ii feel confortable lo reac this sentante ou nc o c Movement trace of a person having moderate d Movement trace of a person having severe motor cerebral palsy impairment like Parkinson s disease Figure 4 2 Affect of motor impairment on user initialization application 21 Virtual User Simulator As it can be seen the choice for having a really simple UIA interface can also help in the beginning of the interaction for a wide range of users with motor impairments The use of the simulator to test this type of situations helps in the verification of which Uls are eligible or a good Starting point for GUIDE and which are not and will need to be adapted and consequently modified 4 2 Media Access Application 4 2 1 Visual impairment simulation some tests were performed with the Visual Impairment Simulator to check the legibility of the text considering the size of the font but also the font itself the font family or name of the font The tests were done for three different profiles Mild Moderate and Severe Visual Impairment but only the two first were stored as the Severe profile was not representative enough figure 4 3a b The 3 series of screenshots show some adjustments made after the first check in the second series figure
39. nts as the area to 25 Virtual User Simulator select a particular item is about a quarter of the screen The main problem here is scrolling and in a design iteration explicit buttons will be introduced to provide an alternative to up and down gestures The other finding is about text size figure 4 7 While the event heading is legible under the conditions of the moderate visual impairment profile in question the important headings for the lectures are not or not conveniently Here adaptation to a larger text size is indicated NeurOHAD Jahrestagung der DGNH 2010 EE TE e es La m E Figure 4 7 Overview lecture list seen through the eyes of person with medium strong visual impairment myopia and macular degeneration Webcast watching and navigation For watching lectures and navigating in content again two findings could be made Using a person with medium strong motor impairment results in the slide icons that act as navigation buttons to be not easily accessible cf figure 4 8a while mild levels of tremor seem to give the user still adequate control The main adaptation to this is to provide large versions of control buttons for navigation or make the left and right part of the slide active for rewind and forward navigation Furthermore visual impairment as simulated in figure 4 8b results in finding that text size is often too small on the slides to be legible In the full screen mode where only the
40. of a sample user stored in the xml file Now you can repeat steps 4 to 12 on this particular user 18 Virtual User Simulator 4 Application interface optimization In this section we have discussed the use of the simulator in optimizing interfaces of GUIDE applications Application designers chose a set of interfaces in their application and simulated interaction of user profiles based on the clustered profiles discussed in D7 1 They used the simulation to either confirm or made corrections to the design to make it compatible to mild to moderate level of disability 4 1 User Initialization Application Regarding the User Initialization Application UIA some use cases were performed regarding different aspects from visual components of the UI to interaction aspects like selecting the ok button to proceed to the next screen 4 1 1 Visual impairment simulation The first use case concerns how users with different characteristics perceive the same screens of the UIA application figure 4 1a to understand at what extend the big buttons that constitute the Ul would serve the initial intention of making the content perceivable to anyone considering that is the first time the user interacts with the system we don t know his or her impairments at the beginning of the interaction so the UI should be perceivable by users with a wide range of abilities e User with mild visual impairments The Avatar is not as perceivable as before and the
41. on The task reported in this document will be continued through task T3 8 User simulation Components till month 24 and a more elaborate report D3 2 will be submitted upon completion Virtual User Simulator Table of Contents t WOVE OCU CEI O ID eiia I O E E E E EN 6 1 1 PUDO Canar A E noo seaneneeneereees 6 1 2 Work plan task and partner CONTIIDUTION ccccc ccc ecc eee eeceeeeee eee eteeceeseeaeeeeeeeseeteeaeees T 1 3 Related documents cccecccc ccc ecceeceeceeeeeeeceesneceesecseceesseseeceesecseeesseseeseeeecseeeeseeseeeeees T 1 4 DOCUMENT organization cece cece eee eee cece ees ece eee eee eee eeaeeseeseeseeseeeseeeeeseeaeeaeeseeseeeeees 8 2 Liter t r SUV CY ciccrimiscrerinin nininini eens con rnamouanswamnnenseenrs iia 8 2 EAVO SUI ON acisis incense na ONE EEEE E ERNER 9 3 1 DESCHID UO eaa E en eee te enn eee ee eee ee ee ec re 9 3 2 Integration of simulator to GUIDE framework cccc cece cecceeeeeeceeeeeeeeeeseeeeeeeeeeeeeaes 15 339 SONWare MANU A win ccrocianensusvusacacuedtindunniantstans caniaduibieiniiiwsieeiucesisoiaaelecudliantestsanawiasiwahecelesnawaioantawias 17 3 3 1 Instructions to run the Mobility Impairment Simulator ccccceccccceeeeeeceeeeeeaeeeeeseeeeeeseeeeeeseeeeesaaeees 17 3 3 2 Instructions to run the Visual Impairment SiMUlatOr ce ccccecceceececeeeecaeeeeceeceseeeesseeeeseeeeseeeessnees 18 4 Application interface OptiMIZAation
42. on to neighbouring targets However even with a well size of minimum target width the following example demonstrates that the gravity well always attract the pointer to the first target on the cursor trace marked with a green circle in figure 6 4 To eliminate this problem we investigated different an escape strategies lt er sy Settings Ser ge et nee Browser Figure 6 4 Wrong target selection by gravity well We explored escape strategies based on movement direction velocity and acceleration of pointer movement However it has been found that these parameters change quite a lot across participants devices and movement path and can attract the pointer towards wrong target the Settings button in figure 6 5 35 Virtual User Simulator lt er sly Browser Figure 6 5 Escape strategy based on pointer movement characteristics So finally we used the following escape strategy if a target is not selected after 1500 msec of the attraction the gravity well for that target will turn off and the pointer will be free to move towards another target The value 1500 msec is chosen to be higher than the measured motor reaction time tm 100 msec Keates et al 2000 of motor impaired users to avoid accidental missed clicks 6 2 Validation Previous work on gravity well did not test it on real life applications We investigated the effect of gravity well on the video conferencing application of the GUIDE project
43. s as those can help them to engage more fully with the world However existing design practices often isolate elderly or disabled users by considering them as users with special needs In this document we present a simulator that can reflect problems faced by elderly and disabled users while they use computer television and similar electronic devices The simulator embodies models for both the internal state of an application and also the perceptual cognitive and motor processes of its user It can help interface designers to understand visualize and measure the effect of impairment on interaction with an interface We also present manual to download and use the software GUIDE application developers used the simulator either to confirm their design or make necessary corrections to make it usable by elderly and mild to moderate visual and motor impaired users We also used the simulator in designing the Avatar and developing runtime adaptation algorithm The report concludes with the discussion on the implication and limitations of simulation and user modeling In short this report provides e Short literature survey on existing user modeling techniques e Description of the GUIDE virtual simulator and underlying models e Manual to use the system e Examples of applications of the simulator o To optimize interface layout o To design anthropomorphic Avatar o To parameterize adaptation algorithm e Implication and limitations of user modeling and simulati
44. sic operations 1 Double click on the file Ml exe 2 The first tab allows you to control severity of three different diseases for three different devices 3 Select a disease and change the level of severity using the slider 4 Select Mouse or Trackball as the input device The touchpad is not working on this version Select the second Tab labeled Hand Functions In this tab you can change the level of hand functions of the user Select the third tab labeled as Demography RL 17 10 11 12 Double click at any point on the image 14 Virtual User Simulator In this tab you can select the age sex and height of the user lf you change anything at the Demography tab do take a look at the Hand Function tab as the hand metrics will change according to the age sex and height of user Now Click on the Demonstration button at the bottom right corner Select an image file in the folder GUIDEScreenshots You can also store any screenshot of your application as a 24bit Bitmap bmp file at 1280 x 800 resolution and simulate effect of impairment on it Wait until the image appears in the screen The simulation will show you the cursor trace up to that point from the centre of the screen 3 3 1 2 Advanced operations 1 2 3 4 3 3 2 Click on load button on bottom left Select an xml file in the folder Profiles It will load profile of a sample user stored in the xml file
45. slide is shown this is alleviated because text size increases by approximately factor 1 5 However as there is only restricted control on the content presented additional means of multi modal presentation need to be considered starting from guidelines for authors and ending with the possibility to adapt text size in slides However the latter is difficult to achieve because slide content is often laid out relying on a certain visual composition that is usually broken if text dimensions change dramatically What may be done is to provide a means to touch up text and present visual captions for instance in connection with the tablet Regarding navigation for more severe visual impact a similar approach to the motor impact adaptation is envisioned Reusing large areas on the screen for a minimum set of interaction commands In addition the remote control will be a handy interaction device with familiar commands apart from the tablet as navigation device 26 Virtual User Simulator a TL webcast simulating interaction for b TL webcast in navigation mode visual navigation profile of an 81 year old with a polio appearance for medium strong visual impairment history among others profile as in Fig 4 7 Figure 4 8 Motor and visual impairment simulation for Tele Learning application In conclusion the simulation of the different impairment profiles indeed gives great insight into the perception and interaction from impaired users
46. t al 2002 The gravity well attracts the pointer in the middle of a button if it is in vicinity of the button So even if the user points towards the edge of a button the pointer will automatically move to the centre of the button The thick blue line in figure 6 2 shows the modified cursor traces after employing the gravity well and the dotted red circle highlights how the cursor has been attracted to the middle of the Disconnect button influenced by the gravity well a a Shared Items Figure 6 2 An example of the gravity well Hwang et al 2002 found that gravity well can reduce pointing time significantly though it works less well for multiple distractors We have conducted a study to investigate the optimum size of the gravity well and an escape strategy for distractors for a real life application Initially we have 33 Virtual User Simulator used the simulator to investigate different well sizes Later we validated the study with a user trial 6 1 Simulation We chose a representative application from the GUIDE project In the simulation study we selected a set of interfaces and simulated cursor traces for all possible targets in the screen with and without gravity well If we consider an interface as a rectangular grid of targets of uniform size then the optimum well size should be half of the dimension of a target However in real life the targets are not of uniform size So we investigated the effect of gravi
47. ter Interaction Design Artificial Intelligence Review Volume 7 Numbers 3 4 August 1993 pp 199 225 e Biswas P amp Robinson P 2008 Automatic Evaluation of Assistive Interfaces ACM International Conference on Intelligent User Interfaces IUI 2008 pp 247 256 e Biswas P amp Robinson P 2009a Predicting Pointing Time from Hand Strength USAB 2009 LNCS 5889 pp 428 447 e Biswas P amp Robinson P 2009b Modelling Perception using Image Processing Algorithms 23rd British Computer Society Conference on Human Computer Interaction HCI 09 e Biswas P Langdon P amp Robinson P 2011 Designing inclusive interfaces through user modelling and simulation International Journal of Human Computer Interaction Taylor amp _ Francis DOI 10 1080 10447318 2011 565718 43 Virtual User Simulator Buxton W Human Input to Computer Systems Theories Techniques and Technology Available at http www billbuxton com inputManuscript html Accessed on 27th October 2009 Duffy V G Handbook of Digital Human Modeling Research for Applied Ergonomics and Human Factors Engineering Boca Raton FL USA CRC Press 2008 Hwang F et al Cursor Characteristics And Haptic Interfaces For Motor Impaired Users Cambridge Workshop on Universal Access and Assistive Technology 2002 87 96 John B E and Kieras D The GOMS Family of User Interface Analysis Techniques Comparison And Contrast ACM Transactions on Computer Human Interac
48. the device into a state known to the user The model can also simulate the practice effect of users Initially the mapping between the user space and the device space remains uncertain It means that the probabilities for each pair of state action in the user space and state operation in the device space are less than 1 After each successful completion of a task the model increases the probabilities of those mappings that lead to the successful completion of the task and after sufficient practice the probability values of certain mappings reach one At this stage the user can map his space unambiguously to the device space and thus behave optimally The motor behaviour model Biswas and Robinson 2009 is developed by statistical analysis of cursor traces from motor impaired users We have evaluated hand strength using a Baseline 7 pc Hand Evaluation Kit of able bodied and motor impaired people and investigated how hand 12 Virtual User Simulator strength affects human computer interaction Based on the analysis we have developed a regression model to predict pointing time Figure 3 5 shows an example of the output from the model The thin purple line shows a sample trajectory of mouse movement of a motor impaired user It can be seen that the trajectory contains random movements near the source and the target The thick red and black lines encircle the contour of these random movements The area under the contour has a high probability of miss
49. the last three decades that are claimed to be user models Many of them modelled users for certain applications most notably for online recommendation and e learning systems These models in general have two parts a user profile and an inference machine figure 2 1 The user profile section stores detail about user relevant for a particular application and inference machine use this information to personalize the system A plethora of examples of such models can be found at the User Modelling and User Adapted Interaction journal and proceedings of User Modelling Adaptation and Personalization conference However most of these models are closely tied with an application limiting their scalability to different projects i User Profle i gt Output Figure 2 1 Simplistic view of a user model On a different dimension ergonomics and computer animation follow a different view of user model Duffy 2008 Instead of modelling human behaviour in detail they aim to simulate human anatomy or face which can be used to predict posture facial expression and so on Finally there is a bunch of models which merges psychology and artificial intelligence to model human behaviour in detail In theory they are capable of modelling any behaviour of users while interacting with environment or a system This type of models is termed as cognitive architecture e g SOAR Newell 1990 ACT R PM Anderson and Lebiere 1998 EPIC Kieras and Meyer
50. tion 3 1996 320 351 Keates S Clarkson J and Robinson P Investigating the Applicability of User Models For Motion Impaired Users ACM SIGACCESS Conference on Computers and Accessibility ASSETS 2000 129 136 Kieras D and Meyer D E An Overview of The EPIC Architecture For Cognition And Performance With Application to Human Computer Interaction Human Computer Interaction 12 1990 391 438 Nejime Y and Moore B C J Simulation of the effect of threshold elevation and loudness recruitment combined with reduced frequency selectivity on the intelligibility of speech in noise J Acoust Soc Am 1997 102 603 615 Newell A Unified Theories of Cognition Cambridge MA USA Harvard University Press 1990 Rieman J and Young R M A dual space model of iteratively deepening exploratory learning International Journal of Human Computer Studies 44 1996 743 775 Tobii Eye Tracker Available at http www imotionsglobal com TobiitX120 Eye Tracker 344 aspx Accessed on 12th December 2008 WHO website Available at http www who int ageing en Accessed on 18th September 2009 World Bank Website Available at http web worldbank org Accessed on 18th September 2009 44
51. tions mentioned in this report were discussed in detail in D2 3 Virtual User Simulator 1 4 Document organization The document is organized as follows In the next section we present a brief survey of existing user modeling techniques followed by the description of the simulator in Section 3 The next three sections present applications of the simulator for optimizing interfaces for different GUIDE applications designing Avatar and an adaptation algorithm respectively Finally we point out the implication and limitation of the work in section 7 followed by conclusion in section 8 2 Literature survey A model can be defined as a simplified representation of a system or phenomenon with any hypotheses required to describe the system or explain the phenomenon often mathematically The concept of modelling is widely used in different disciplines of science and engineering ranging from models of neurons or different brain regions in neurology to construction model in architecture or model of universe in theoretical physics Modelling human or human systems is widely used in different branches of physiology psychology and ergonomics A few of these models are termed as user models when their purpose is to design better consumer products By definition a user model is a representation of the knowledge and preferences of users that the system believes the user posses Benyon and Murray 1993 There was a plethora of systems developed during
52. tons without difficulty and almost without deviations from the targets However with more difficulty than the previous user particularly when selecting the OK button The large size of this button proves its usability on this use case because even with a lot of tremor the user never goes out of the button before selecting it figure 4 2C Male user 77 years old suffering severe motor impairments moderately affected by Polio s Parkinson s and cerebral palsy User is able to interact and select any of the buttons however he has difficulties targeting the and the ok button and also makes a lot of deviations before reaching any of the three buttons but specially before reaching the closest button to the center Again the large size of the buttons proves its usability and efficiency for users with motor impairments and even with a lot of tremors and uncertainty the user can reach and select the different buttons without ever selecting other than the targeted one and almost without going out of any button before selecting it just minor problems with the button figure 4 2d Decrease the font size until you Decrease the font size until you find difficult to read find difficult to read Lice Il heal oontorishle to need this Lies Ht Pel orforable to raeg this at i na m Sel f sanba SOnkence sr BEE a Movement trace of a person having no motor b Movement trace of an elderly person having mild senbence mpa
53. ty well with sizes of e Average target width e Minimum Target width We simulated cursor traces for different levels of severity of diseases for these two well sizes We have also considered the exact location of clicks inside a button especially pressing at the middle of the button and pressing at the edge of the button Figure 6 3 shows the effect of gravity well on these four conditions Well Size x location of Click a Well size Avg Target Size and Clicking b Well size Avg Target Size and Clicking at Middle at Edge c Well size Min Target Size and Clicking d Well size Min Target Size and Clicking at Middle at Edge Figure 6 3 Effects of gravity well In all these figures the thin red line shows the simulated cursor trace without gravity well and the thick blue line shows the cursor trace with the gravity well activated In all these case we have 34 Virtual User Simulator confirmed Hwang s result that gravity well indeed reduces the pointing time However it has also been found that while the pool size is equal to the average target width the pointer may be attracted to the neighbouring button if the user chooses to press at the edge of a button For example in figure 6 3b the gravity well attraction misses the Disconnect and right arrow key button marked with green circles in the figure So in the validation study we kept the size of the well equal to the minimum target size to reduce unwanted attracti
Download Pdf Manuals
Related Search