Home

Multimodal User Interface Design: Increasing Accessibility to

1. Experimental Tasks Pre test Questionnaire Perfom Task I Post task TK IG Questionnaire Task 1 6 Post task Short Interview Post test Questionnaire Post test Short Interview l Quality Characteristic and Metric 1 Accuracy of Information Perception a Correctness of answers to identify geometric shapes b Achieved completeness of working with virtual objects c Level of realism in virtual reality 2 Efficiency a The time to identify geometric shapes b The time to point out the location of virtual objects c Level of easiness to identify shapes of objects d Level of easiness to manipulate a device 3 Effectiveness a Number of tasks completed 4 Satisfaction a Level of confidence to manipulate a device b Level of user s emotion after the test 2 sighted with blind hold and 3 blind participants Majority is in school age Verbal training about Virtual Reality Haptic device Haptic interaction and Manipulation of machine a Verbal representation of pre test questionnaire b Verbal representation of 10 different task scenarios c Verbal representation of post task questionnaire d Verbal representation of post test questionnaire Input Output force oa E Haptic prototype application Figure 6 1 Overview of experiment with haptic mode 62 6 3 2 Problems Statement and Test Objectives The goal of the usability test is to obtain data addressing to user i
2. 2 Consider different representations to enhance different properties To test this point we added magnetic effects on objects Magnetic effects on the sphere increased the level of perception Four participants accomplished the task to understand the magnetic 105 sphere in less time than the other two spheres And three out of these four participants accomplished this task within 30 seconds Three participants rated the magnetic sphere as the easiest sphere to understand and only one person rated this sphere as the most difficult one Appendix F On the other hand the magnetic cube was not the best object to understand The reason is described below in 3 Another test to see this point was representing the object in the bounded space In our experiments we presented the object in free space and bounded space Two users had a hard time to navigate the virtual space and were not able to touch the object They moved the stylus beyond the screen space Bounded space restricts the area that participants move the stylus If they touch the wall of bounded area they feel force and cannot move any farther If they move the stylus beyond the screen space the vibration force was sent to them These features made it easy for participants to navigate in the virtual space The bounded space served as reference point as well Adachi et al used friction on the object to enhance the perception They showed that an additional friction ef
3. Object tested Sphere in the free space Users Response Ring Something round Cube in the free space Sphere in the bounded space Circular plate Knife cover Half circle Ellipse Cube in the bounded space House upside down Sphere with magnetic effect Egg Cube with magnetic effect Rectangular box 8 1 2 Pointing Positions on Objects All participants performed tasks of pointing the specific position on objects within 30 seconds except in 3 instances out of 180 After the completion of task 2 all participants accomplished this task within 2 seconds except in 1 instance out of 120 All locations were pointed out within essentially the same amount of time reflect with the answer from after task questionnaire Il Table 8 2 Average time to place the stylus on the object sec Results from this task Objects tested Location on the object Top Bottom Left Right Front Back Sphere in free space 8 5 13 0 17 0 9 5 9 5 17 0 Cube in free space 3 2 9 4 9 4 7 4 7 8 9 8 Sphere in bounded space 1 5 1 5 1 25 1 25 1 25 1 25 Cube in bounded space 1 5 1 5 1 25 1 25 1 25 1 25 Sphere with magnetic effect 1 0 1 0 1 0 1 0 1 0 1 0 Cube with magnetic effect 1 4 1 4 1 2 1 2 1 4 1 4 T1 8 2 Result from After Task Questionnaire Task 1 6 Data from the after task questionnaire is used as a metric see Appendix C Table 8 3 shows the su
4. 3 7 Haptic Interface for Blind Users Potential use of haptic technologies for blind users has been studied by several researchers Sj str m Sj str m and Rassmus Grohn 1999 at Certec in Sweden has done intensive investigation of potential application areas by utilizing the PHANToM haptic device for blind users Sj str m and Grohn R K Sj str m and Rassmus Grohn 1999 Sj str m 2001 Magnusson Rassmus Grohn et al 2002 Sj str m Danielsson et al 2002 Sj str m Danielsson et al 2002 He has produced many various applications including graph representation of mathematical information He investigated how blind people orient and navigate the virtual haptic environment explored possible solutions to access the graphical user interface through haptical interaction and 28 studied how different types of graphical information such as maps floor plans and pictures could be accessed through haptic interaction Through testing prototype application with blind users he presented the guidelines for haptic interaction design Sj str m 2002 The purpose of his guidelines is 1 to develop usable efficient learnable and user acceptable design of products 2 to evaluate the existing designs and 3 to assist users to provide the feedbacks from their experiences In the guidelines he classified the haptic interaction design into five aspects 1 virtual objects 2 navigation and overview in the virtual environment 3
5. the level of realism in virtual reality Efficiency Efficiency relates to the level of effort to accomplish the task To measure the efficiency the following data is used the time to complete the task the time to place the stylus at certain positions of virtual object the data collected from questionnaire to measure how easy it was to identify objects and to manipulate the device Effectiveness Effectiveness relates to the accuracy and completeness of the achieved goal The number of completed tasks is used to validate effectiveness Satisfaction Satisfaction relates to the comfort and overall acceptability of the task This point is measured by answers in questionnaire and user observations Since none of the participants had ever experienced 70 haptic interaction measuring satisfaction by Liked scaling may not be reliable so that subjective data is used instead Table 6 3 Summary of quality characteristics and metrics Quality Characteristic a Correctness of answer to identify geometric shapes b Achieved completeness of working with virtual objects c Level of realism in virtual reality Accuracy of Information Perception Ata Q4 The time to identify geometric shapes The time to point out the location of virtual objects Efficiency Level of ease to identify shapes of objects Ata Q1 Level of ease to manipulate a device Ata Q2 Level of user s confidence to manipulate a device Ata Q3 Ate Q4
6. Multimodal User Interface Design Increasing Accessibility to Mathematics For Blind students Yayoi Shimomura October 2005 University of Iceland Reykjavik Iceland Abstract The project investigates the potential benefits of multimodal user interfaces for blind students A multimodal user interface uses the combination of two or more different human senses to interact with computers The work focused on finding the needs of the interaction model for blind students by using different human modalities so that they perceive and control information in efficient and effective ways Sweller s cognitive load theory Sweller 1988 is applied in this study to examine efficient and effective learning through computer interaction In the first part of this work we studied how visually impaired computer users interact with computers with currently used solutions screen reader and Braille display In the second part the work explored the potential use of haptic technology to visualize the computer information for mathematical learning as an experimental data type which visually impaired people have difficulty to access The work was carried out in a real context with users who are blind The result indicated that the audio user interface screen reader was not always the best interaction solution though this is the most commonly used among the blind computer users The effectiveness of tactile user interface Braille display to accomplis
7. a Number of tasks completed feet ae ot Satisfaction a Characteristic of user s emotion after the test Note Ata Q After task questionnaire Ate Q After test questionnaire see Appendix C In order to validate points in the Sjostrom guideline see Table 6 1 the following data is used questions 1 and 2 in After task questionnaire 2 for task 1 6 and questions 1 2 and 5 in After test questionnaire see Appendix C 71 7 Results with Audio and Tactile Mode In this chapter results from experiments with screen reader and Braille display are presented Details of results with screen reader are in Appendix D and with Braille display in Appendix E The findings were collected from the after task interview and the observations The data of findings were categorized into four types of problems 1 Usability and Accessibility related problem 2 Software fault related problem 3 Participant s cognitive mechanism related problem and 4 Learning method related problem The problem category 4 was applied only for tasks solving mathematics In experiment with Braille display data was collected from two participants One of the participants was not able to participate in the experiment due to the technical problem with Braille display He had this problem often and we were not able to schedule him for testing Table 7 1 shows the example of data collected from experiments with screen reader and Braille display
8. Avoid objects with small and scattered surfaces In order to provide information using haptic sense in optimized way it is necessary to test different effects on same objects and to understand the level of perception The advantage of virtual haptic force feedback over the real object is that effects can be controlled by application and more information can be sent to the users to improve the interaction to perceive the geometric properties By applying Sjostro6m s guidelines to a prototype we examine if different effects improve the haptic interaction for blind users Objective 2 To explore the use of haptic cue for mathematics learning Hypothesis 3 Haptics have a potential for allowing students to conceptualize abstract mathematics information Brewser Yu Brewster et al 2000 Yu and Brewster 2003 applied haptic interaction for graphs Sj str m and his colleagues Magnusson Rassmus Grohn et al 2002 also studied to present graphs haptically Both parties presented graphs in two dimensional In this study we would like to seek other potential area to use haptic cue to present teach mathematical information We assume that results from hypothesis 1 and 2 show the range of strength in haptic interaction By 10 applying theses results into this hypothesis the study examines whether haptic interaction can be used to understand more mathematics than just graphs We predict that the haptic technology w
9. In this study we focused on studying single mode prior to design the multimodal user interfaces with high usability and accessibility Suggested future works for audio tactile and haptic interfaces described earlier need to provide the base for designing efficient and effective multimodal user interfaces The following objective and hypothesis are suggested to examine in the future work Objective To explore the effect of multimodality on learning This objective is to examine whether multimodality increases the blind student s performance and subjective satisfaction Several multimodal studies indicate that adding extra sensory cue to interact with computer provides better performance to conceptualize the information It is suggested to examine if the same result from these studies apply for the intended users blind students in specific application domain learning mathematics 113 Hypothesis By combining more than one mode of information representations users are able to access information more easily than with a single mode Brewster et al represented multimodal graph Yu and Brewster 2003 They stated that combining haptic and audio cue to understand graphs improved the information perception for blind people It is assume that their study may apply for intended application domain learning geometry in this study 114
10. eeeeeeeeeeeeeeeeeeeeeeteneeeeeeees 5 2 1 Overall Research PUrpOSe ssssssessssssssessssssseeeseesesseeeeees 5 2 2 Research Objectives ccccccccceeeeeeeeeeceeceeeeeeeeeeeeeeaeeeeeeeeeeeeeeas 6 2 3 Research Hypothesis 8 3 Literature Review ccccccceecccceeeeeeeeeeeeeeeeeaaeeaaeeeeeeeeesesaaannneeeeeeees 13 3 1 Computer Interaction and Blind Users esesseeeeeeeeeeeeeees 13 3 2 Mathematical Learning and Blind Students eee 14 359 Cognitive Load THEO raei er e nent aati 15 3 4 Reading vs Listening to Information and Memory 008 18 3 5 Haptic INGSIFA CE 5 cid ioc ositascnoninon ee out nn sted ceocessesestecseas cues sneseatse 21 3 7 Haptic Interface for Blind Users cccceeeeeeeeeeeeeeeeeeeeeeeeeees 28 3 8 Screen Reader and Braille Display ceeececeeeeeeeeeeeeeeeeeees 30 3 9 FRCCOSS IDI Ye a vac p aa a dadaa deheiea te a schon 32 4 Haptic Prototype imei aianei a a a ri a 38 AeA VINUOGUCTION sta a a a a E ee OA ds 38 4 2 Objects reside in the free SPaCe cccccceeeeeeeeeeeteeeeeeeeeeeeeees 39 4 3 Objects reside in the bounded Space seseeeeeeeeeeeeeeees 39 4 4 Objects with magnetic force effect reside in the free space 40 4 5 The computation of magnetic force effects for a sphere 41 4 6 The computation of magnetic force effects for a cube 43 4 7 Cylinder Cone Torus and Rotated
11. 3 If HIP is inside the inner sphere the direction vector v v is toward outside And if HIP is between inner sphere and outer sphere the direction vector v v is toward the center of sphere Using Hooke s laws the anchor of the spring is the center of the sphere and the length of the spring is r1 If the magnitude of vector from HIP to the 41 center is larger than r1 the effect is as if the spring is pulled and the spring tries to pull back toward the center If the magnitude of this vector is smaller than r1 the effect is as if the spring is pushed and the spring tries to stretch toward outside 4 Force k r1 v _ where k is a constant integer Boundary sphere Graphically presented sphere Magnetic effects Figure 4 5 Concept of magnetic sphere 42 4 6 The computation of magnetic force effects for a cube Again the Hooke s spring law is applied The graphically presented object is surrounded by the cube shaped boundary The side of boundary cube is slightly larger than the graphically presented cube The force is applied as soon as the virtual stylus enters the boundary cube The simulated magnetic attraction is toward one of the axes The steps of computation are as follows 1 Detect haptic interface position HIP the position of the virtual stylus in the coordinate system 2 Compute the closest plane of the cube from the position HIP 3 Compute the normal vector n to the ax
12. With screen reader 36 problems or observations were collected in 3 sessions 24 out of 36 problems were related to usability and accessiblity issues In 5 task instances out of 20 totals participants 72 were not able to finish their tasks In seven out of the 20 instances participants were not satisfied with their performance i e the amount of time it took them to finish the task With Braille display 23 problems or observations were collected in 2 sessions 14 out of 23 problems were related to usability and accessiblity issues All tasks were performed successfully out of 10 instances in total In 2 out of the 10 instances participants were not satisfied with their performance Table 7 2 shows the distribution of problems observed during two tests Table 7 1 Sample Data Participant P1 P2 P3 Time to complete this task min 45 4 7 Satisfaction with the amount of time spent y N y Suitability of screen reader for this task Y y Y Nipp Observation Usability and Blank lines indentations and capital letters are usually used in the AE EA document to increase the readability But some of these design y guidelines for example blank lines destructed blind participants Software fault Listens to a problem many times until understanding The screen Cognitive reader is not a hindrance for him but the tools that he has available mechanism i e Word in this case Listening and solving the problem at
13. a learning tool for the blind students the weaknesses of current haptic interaction with PHANTOM need to be addressed Future work needs to consider how we can improve the accuracy of information perception and how we can present the small details of virtual object haptically Following additional works are suggested in the future study 1 Use the unique characteristic of haptic interaction bidirectional information flow Salisbuty Conti et al 2004 The prototype in this study did not apply this characteristic for the interaction If this characteristic is used in haptic interaction more information flow occurs between a user and a machine 2 Apply various haptic effects on the virtual object In this study we have applied only magnetic effects and created boundary as reference points Our study showed that different haptic effects affect the level of perception In the future work it is suggested to add different effects on virtual object and compare Such effects are vibrations with different level of intensity friction and viscous effects 3 Combine another sensory channel with haptic sensory to improve the accuracy Brewster et al showed that providing the 112 audio information during the haptic interaction improve the level of information perception PHANToM uses the one point haptic interaction It is also suggested to investigate the advantage of one point haptic interaction Multimodal user interfaces
14. and records any noticeable issues 6 3 5 Task Scenario Tasks to be tested are selected to evaluate the hypotheses 1 2 and 3 Since the target users do not rely on visualization it is important to 67 examine how accurately participants conceptualize haptically presented information and how well they can manipulate virtual objects Because the virtual objects given to the participants are familiar and simple they must be able to recognize them exactly for what they are and how they are Sjostrom developed guidelines for non visual haptic interface design Sj str m 2002 Selected tasks validate the following points from this guideline while at the same time evaluating the hypotheses in this study Table 6 1 Sjostr m s Guidelines to be validated Selected points from Sjostrom s Guidelines to be validated Use rounded corners rather than sharp ones Consider different representations to enhance different properties Tasks to be tested There are 10 tasks to be tested by test participants Tasks are grouped together into two task 1 6 and task 7 10 In tasks 1 6 familiar shapes such as sphere and cube are repeatedly touched In tasks 7 10 less familiar shapes such as cone cylinder torus rotated positioned cube are perceptualized 68 Table 6 2 Tasks to be tested Task Familiaritys Object Surrounded Environment Validate or Effect e MED Gainer __i pordedenm yia vewe Torus Inbounded s
15. computer users are able to access information which was impossible to 13 access before the current trend of computer user interface and information presentation still hinders full access for these users 3 2 Mathematical Learning and Blind Students The mathematics discipline is the base for science and engineering study but it is one of the greatest challenges for blind students Many blind students have much less chance to develop mathematical skills because of mathematical concepts and notations are represented graphically Cahill and Boormans 1994 Osterhaus a secondary mathematics teacher at the Texas School for the Blind and Visually Impaired http www tsbvi edu index htm described that one of the most difficult challenges has been teaching concepts involving three dimensional objects Three dimensional problems are found in all levels of mathematics They are often difficult for students with vision to understand especially when trying to create 3 D objects in 2 D drawing Such a drawing even when tactually raised makes little sense without a sighted perspective Osterhaus Another problem she described was the visualization of a three dimensional graph to solve the equations in three variables Most sighted students understand the concepts of equations by graph Osterhaus There are several studies done in both educational and technical areas to solve these problems Karshmer et al tackled the problem of r
16. context of virtual objects 4 modalities and 5 learning Brewster and Yu studied the haptic interface for blind people in a much narrower area They focused on the graph representations of mathematical and business data They combined force feedback synthesized speech and non speech audio to improve the access to graphs and tables for blind users Yu Brewster et al 2000 Brewster and Yu 2002 Brewster and Yu 2003 Brewster and Yu 2003 Yu and Brewster 2003 They also compared two different force feedback devices PHANToM from SensAble Technology Inc and WingMan from Logitech Yu and Brewster 2003 They investigated the practical use of haptic interaction and developed a web based haptic application to create bar and line graphs with a WingMan mouse Brewster and Yu 2002 There is a big difference in price between 29 PHANToM and WingMan PHANToM costs almost 100 times more Their solution in the work Brewster and Yu 2002 with popular web based application and low cost device is a rare example of providing a real life application They also investigated the role of haptics in multimodal interfaces synthesized speech non speech sound and haptics The result showed that the role of haptics depends on the information representation Moreover the nature of the application and the dimensions of the information being presented determine the amount of haptic input and the way it is used Yu and Brewster 2003
17. different types of text style like title and body or color It reads as typing by character or and by word and has selection of 12 languages besides Icelandic as an output language The software does not automatically detect the language difference but this feature is very useful for students when they explore the Internet and learn foreign languages Without this feature for example the screen reader reads out English web contents with Icelandic phonetic rules or vice versa This software also allows users to select a different type of voice and edit speed and pitch http www dolphinuk co uk Dolphine Supernova outputs screen readed information onto Braille display The Braille display output on screen text on a row of 40 to 80 characters formed by 31 groups of 6 pins Dolphin Oceanic Ltd producer of Dolphine SuperNova developed its own synthesizer and uses formant synthesis technology There are two major technologies in synthesized speech concatenative and formant synthesis Formant synthesis does not use any real human speech sound at runtime Sound output of synthesized speech is created by an acoustic model artificial speech The speech sound is not natural and somehow robotic sound but this method of technology is highly reliably intelligible Even at high speeds it can produce the speech without glitches Also a formant synthesizer is much smaller program than a concatenative synthesizer since the latter uses the reco
18. how tactile representation must work best for some particular type of information and so do audio and haptic representations The study investigates what type of information is best represented by what mode This result helps blind students to learn more efficiently 12 3 Literature Review This chapter presents the literatures of the subjects at the core of this project 3 1 Computer Interaction and Blind Users Computers have become an indispensable tool in our life Its technology and the Internet greatly affect our modern life style This fact also applies to people with visual disabilities Computer technology can offer them independence in learning receiving information communicating with others and working However these potential benefits of computer technology are not fully exploited by people with visual disabilities Blind computer users in Iceland rely mainly on screen reader software and or Braille display to access information presented in graphical user interfaces Neither the screen reader nor Braille display is a complete solution for accessing the graphical information in window based systems Window based systems present not only a problem with graphical manipulated information but also the spatial organization of graphical objects in the screen Therefore blind users commonly use the tab or keyboard to move through menus icons and text areas instead of using a pointing device such as a mouse Though blind
19. on or by the haptic device The GHOST SDK does not generate the graphical representations of the objects but provides the graphic call back mechanisms to facilitate integration between the haptic and graphic domain SensAble Technologies Inc In order to generate the graphical representations of objects the OpenGL graphical package is used in this project OpenGL is a software interface to graphics hardware for producing interactive three dimensional applications and is designed to be platform independent Figure 3 1 PHANToM Desk Top haptic device 23 General Principles of Haptic rendering The most important feature of haptic interaction modality is the bidirectionality With audio and visual channels information and energy flow into unidirection which means from machine to the user However the haptic modality exchanges information and energy between the user and machine Salisbuty Conti et al 2004 As it is seen in Table 1 the perception bandwidth in haptic modality is low Fritz and Barner 1999 but if the characteristic of bidirectionality and the dynamic nature of the kinesthetic sense are utilized in proper way it might be possible to modify the visual perception for the blind users In the case of the PHANToM Desktop device when a user manipulates the pen like stylus of a haptic device the position and orientation of stylus are detected by encoder As a user moves the real stylus the simulated stylus mo
20. participant in order to find the background information prior to the task performance Questions are focused on their experience in using screen reader or Braille display with particular applications This information is combined together with the result of the after task and after test interview to evaluate the test results After Task and After Test Questionnaire Prepared questions are asked prior to the short interview After task questionnaire uses yes no answer and after task questionnaire uses likely scaled in 5 levels and subjective answers 53 After Task and After Test Interview Since this test uses the reflection method test personnel conduct a short interview immediately after each task and test are completed We are interested in collecting the subjective data from the participant s experience The interview is developed based user observations during the session Questions in interview focus on level of user s emotion and difficulties during the test session Test Time The approximate test session is no more that two hours unless participants wish to quit before they complete a test Post test Review After the test session is completed the tester reviews the videotape record of the test session and records any noticeable issues 5 3 4 Task Scenario Tasks to be evaluated during these testing are selected from ones that are thought to be most common and frequently performed by the most general computer us
21. using 5 point liker type scales and yes no response The questionnaire asks specific questions about users own task performance The short interview is developed based on the observation during the test session Any user s verbal comments and physical expression that indicate user s emotion during the test session are noted for interview During the usability testing users performance is video recorded in order to collect more detailed information on their performance 48 5 3 Design of Experiment 5 3 1 Overview 3 participants with visual disabilities Age 13 18 School students General orientation of the test environment and the Exper kaenen flow of the test sessions Treatment Material Experimental Tasks Pre test Questionnaire a Verbal representation of pre test questionnaire b Verbal representation of 7 different task scenarios for the audio interface and 5 different task scenarios for the tactile interface c Verbal representation of post task questionnaire Verbal representation of post test questionnaire Perfom Task Post task Questionnaire d Post task Short Interview Keyboard Input Audio Tactile Output Post test Questionnaire Post test E stati Quality Characteristic and Metric 1 Efficiency 1 1 Temporal Efficiency a The time to accomplish each task 1 2 Level of Mental Load a Level of easiness to work only with screen reader Braille display to accompl
22. video games that allow players to feel and manipulate the tools solids or fluid and the industrial applications integrated with CAD systems that allow designers to manipulate some mechanical components The result supports the objective 2 Objective 2 To explore the use of haptic cue for mathematics learning The second objective is also related to haptic perception Since one of purposes in the study is to support the learning mathematics the study explores what type of information can be presented haptically in order for blind students to learn mathematics independently By examining the features of haptic device with users the study seeks the way to support the mathematics learning for blind students As it is described in the Chapter 3 some researchers worked on the graph representation of scientific data with haptic machine We focus on a subject of geometry in mathematics and explore how haptic cue can be utilized to learn geometry Objective 3 To investigate the effectiveness of information perception with audio and tactile sensory cue for learning The third objective is associated with the currently used solutions The study examines the screen reader and Braille display to see whether users are perceiving information effectively so that they are able to perform common computer tasks efficiently Each sensory cue has different ways of encoding the information and storing into human memory system If each sensory is n
23. Cube ccseeees 45 5 Experiment with Audio and Tactile Mode cccccceeseeeeeeeeeeeees 46 Fo Pad Ia oig ole LU io a PA EEEE 46 5 2 Research Methods 0 ccc etic idee tet diene cele hed 47 5 3 Design of Experiment lt s 2caceiedndeee epee seie ted 49 Bc Sid OVGIMIOW 5 F nase TEE heb tend soba EET 49 5 3 2 Problem Statement and Usability Test Objectives 50 5 3 3 RESOUICOS pikapil pripara kreiki rnaak elect he eles 50 5 9 4 TOSE SSSS OS sac ces tase anc sive season Bearers satin ote 52 5 34 Task SCN al lO eeneioe a e A E 54 5 3 5 Quality Characteristic and Metrics cccceeeceeeeeeereeeeees 56 6 Experiment with Haptic Mode 0 cccceeeceeeceeeeeeeeeeeeeeeaaeeeeeeeeeees 59 6 Tintrod ctionNi reene lenctenann ovcesewnaetecaen ae E a E tins 59 2 Researchh MeMO S a a a ea aa e niaii 60 6 3 Design of Experiment ccccccceeeeeeceecceeeeeeeeeeeeeeeeeeeeeeeeeeenees 62 7 Results with Audio and Tactile Mode eeceeeeeeeeeeeeeeeeeeeeeeeeeeeees 72 8 Results with Haptic Mode cccccceecceeeeecceeeeeeeeeeeeeeeeeeeaaeeeeeeeeeees 75 8 1 Completeness of Task Performance Task 1 6 ceeeeeeee 75 8 1 1 Identifying OD SGU siecctiase eat iasrransadeieteaant aie vanianea kine 75 8 1 2 Pointing Positions on Objects cccceeeeeeeeeeeeeeeeteeeeeeeees 77 8 2 Result from After Task Questionnaire Task 1 6 0 ee 78 8 3 Co
24. OST SDK 2 Objects residing in the closed space i e bounded space with virtual walls with the default force effect which is calculated by GHOST SDK 3 Objects with magnetic effects residing in the free space In addition to three representations of sphere and cube the following geometric objects are presented as well 1 Cone residing in the bounded space with default force effect 2 Cylinder residing in the bounded space with default force effect 3 Torus residing in the bounded space with default force effect 4 Cube in rotated position residing in the bounded space with default force effect 38 4 2 Objects reside in the free space Figure 4 1 Figure 4 2 Sphere in free Space Cube in free space When object reside in the free space as seen figure 4 1 and 4 2 the user feels the object when a collision occurs otherwise the user can move the stylus around in the virtual space freely and even off from the screen space 4 3 Objects reside in the bounded space Figure 4 3 Figure 4 4 Sphere in bounded space Cube in bounded space 39 When objects reside in the bounded space as seen in Figures 4 3 and 4 4 the user is able to move the stylus only inside this bounded area The area is bounded by five walls top bottom back right and left It is constructed like a box shape except the front wall The box shape is open at the front so that the user moves the stylus as if they insert it into the box The five walls re
25. Task scenario Organize a contact in an e mail program This task was performed only with a screen reader One participant was unable to complete this task A second participant was not satisfied with his performance As in the case of the task of sending e mail here too were problems that were due to insufficient usability and accessibility Software faults were also present Recommendations In case of software faults automatic recovery that returns the system to the same state will undoubtedly help Additional tracing can provide better recovery Since it is more difficult to assess the state of the system for blind people there is less tolerance for faults and increasing demand for higher reliability Open problems In simple data entry applications such as an address book it is worth investigating whether audio input with simple one word commands is applicable Another way might be to parse free form text instead of a form This would free the burden of the blind person of navigating within a form and provide a more direct manipulation 86 Task scenario Editing an MS Word document Screen reader None of the participants were able to complete this task efficiently Two gave up and the third was not satisfied with the time it took A number of problems were recorded mostly under the category usability and accessibility but also under the category cognitive load Braille display None of the participants were able t
26. The information can be perceived by users more directly in audio than in haptics Yu and Brewster 2003 3 8 Screen Reader and Braille Display The most commonly used solution for blind people to access computer presented information is screen reading software The screen reading software is loaded into the memory at system start up Screen reading software is used with a speech synthesizer or and a Braille display The role of screen reading software is to capture the contents displayed on the screen including graphic object labeling and send this information to speech synthesizer or Braille display The screen reading software takes the raw text including numbers and abbreviation to converts into written out word equivalents text normalization After text 30 normalization process it assigns phonetic transcriptions to each word and divides and marks the text into prosodic units such as phrases and sentences The screen reading software combines the phonetic transcriptions and prosody information into symbolic linguistic representation to synthesizer or Braille display http www absoluteastronomy com encyclopedia S Sp Speech_synthes is htm The screen reading system used by blind students in Iceland is called Dolphin Supernova This screen reading software reads or spells out any text displayed on screen including menu and emails It also reads icons and Internet graphics labels and announces font and style information e g
27. To investigate how effectively blind students perceive computer oriented information with audio and tactile cues We investigate the currently used solutions to see how blind students access information presented by computer These solutions to be assessed are screen reader and Braille display This thesis is divided into eleven chapters The second chapter provides research purpose objective and hypothesis and is followed by the third chapter which presents the background on the topics to be covered In chapter four the haptic prototype is described Chapter five and six describe the design of research experiments Chapter seven and eight provide the results of experiments and chapter nine and ten present the analysis driven by research methods and the final chapter presents conclusions along with recommendation to those who design the user interfaces for the blind computer users and open questions that are still left unanswered for the future research 2 Research Objective and Hypothesis 2 1 Overall Research Purpose This thesis investigates the potential benefits of multimodal user interface for blind computer users in learning process There are currently several research studies about multimodal user interfaces for blind people Fritz and Barner 1999 Oakley McGee et al 2000 Brewster and Yu 2002 Jacobson 2002 to access the computer presented information Also there are many research studies done and real life applications d
28. amid or diamond e Torus has some round features Several participants also felt some level of realism even though they did not feel objects as realistically as touching with their fingers However the study showed that the accuracy and detail of virtual information they perceived is lacking in the haptic only interaction with the PHANToM unit Participants were not able to understand the orientation of an object rotated cube and extra information presented around the object 104 objects in bounded space In order to use haptic machine for blind students to learn geometry independently greater accuracy of information perception is required Sphere must not be of an egg shape and a cube should not be a rectangular box Hypothesis 2 Different presentations haptic effects and reference points of the same geometric object will improve the perception of haptically presented information In the experiments we have tested several issues presented in the Sj str m s guidelines Sj str m 2002 see Table 6 1 and 6 2 to examine whether different effects and presentation of objects he suggested helped participants to understand virtual information better 1 Use rounded corners rather than sharp ones When participants tried the cube the sharp corners were keys for them to understand this object One participant commented that for this person the cube was much easier to understand than sphere because of the sharp corners
29. ants were not able to differentiate between three dimensional and two dimensional space We are not used to touching three dimensional objects except in the real world The problem of understanding three dimensional space may be solved by training For the cube participants responded with rectangular shape and house up side down Two out of five participants recognized the cube in the first response but one did not have an idea what it was for the first time he she tried the cube in task 2 Verbal comments made by the participants during these tasks show the uncertainty of their answer even when they answered successfully at the end They might have guessed their answer Some participants commented that they were not able to identify the proportion of objects e g cube vs rectangular box and sphere vs egg shape This point is discussed in the work done by Sj str m and his colleague as well Magnusson Rassmus Grohn et al 2002 A more detailed experiment is needed to examine how users understand the proportion of one object to another The cylinder and the cone are considered less familiar shapes When these objects were tested the level of assistance received from the test personnel was low except in one instance out of total of 20 Appendix F 96 A cone has a very sharp point on top round surface and a sharp corner at the bottom These facts might have helped the participants as anchors to connect with their familiar obje
30. by using a haptic device PAHToM by SensAble Technologies Inc The results of that study were promising as the percentage of persons that recognized the object was as high as 96 and always at least 76 Magnusson Rassmus Grohn et al 2002 Stephen Brewster studied how haptics could be used to represent graphs of complex statistical and abstract mathematical data Brewster and Yu showed that different types of graphs represented by audio and haptic feedback have more accessibility than a manually representeation of paper paste and string used for static graphs Yu Brewster et al 2000 Yu and Brewster 2003 Through the potential of haptics has been shown in experiments the need of further investigation is considered in order to design the application in a real context We need to research where and how these potntials are best suited and how other human senses can be ustilized in order to interact with computer in most effective way for blind users Objectives of this study are 1 To investigate the effecto fo haptic cue on how accurate users perceive information without visualiztion This result can support the objcective 2 2 To explore the use of haptic cue for blind students to learn mathematics The goal of this study is to find the way to support blind students for independent learning of mathematics We focus on a subject of geometry in mathematics and explore how haptic cue can be utilized to learn geometry 3
31. ch of consciousness where humans examine evaluate transform and compare different mental representations Wickens and Hollands 2000 Human working memory has limited capacity to hold information and its limit is controlled by time Wickens and Hollands 2000 According to G Miller Miller 1956 the maximum capacity of human working memory is 7 2 items chunks of information when people pay full attention to receive that number of items Wickens and Hollands 2000 Main use of working memory is not holding the new information Sweller et al describe the common use of working memory as to process information in the sense of organizing contrasting comparing or working on that information in some manner Sweller Merrienboer et al 1998 Sweller et al concluded that humans are probably only able to deal with two or three items of information simultaneously when required to process rather than merely information Sweller Merrienboer et al 1998 The new information held in the working memory is encoded into a more permanent form for the long term memory Learning refers to this encoding process Cognitive Load Theory states that human learning is optimized when the load on the working memory is kept minimized Then changes to the long term memory are facilitated in the best manner The theory is best known for designing effective learning materials by using multimedia 16 The learning materials have to be desig
32. ch others weaknesses 94 10 Analysis and Discussion Haptic Mode 10 1 Accuracy of Information Perception 10 1 1 Correctness of identifying geometric objects All participants completed tasks 1 6 in varied time see Appendix F This implies that all participants responded with the correct answer at the end of each task One important issue we need to note here is that the participants completed these tasks with some assistance given by test personnel The level of assistance they received is recorded in Appendix F The reason for giving assistance to participants was that none of participants has experienced with virtual reality or haptic devices Four out of five participants were children and three out of five were blind Children tend to ask questions when they are uncertain about what they are doing Hanna Risden et al 1997 Blind people have this tendency as well especially blind students Blind students in Iceland learn at school with constant assistance and tend to ask questions whenever they are not sure of what is going on For running a usability test with young participants it is important to make them comfortable and encourage them to accomplish tasks Therefore in the analysis here we focused on their first responses recorded during the session 95 In the case of sphere shape task 1 3 6 the participants responses were circle egg circular plate knife cover and a half circle The problem here is that the particip
33. cipants average response was 4 5 The participants did not give high ratings for the level of ease to manipulate the PHANToM machine The average point after each task was never 5 and overall rating after the test was 3 5 One user had a hard time to feel the difference between the force output and the machine s stiffness The test personnel held this person s hand to teach the difference of feeling Another participant had a hard time to hold the pen like stylus The test personnel suggested blind participants to hold the stylus as they hold a pen but they seldom use a pen and did not understand how to hold it 10 3 Effectiveness For 10 instances out of 50 in total participants were not able to accomplish the task even with help provided by test personnel We are not sure whether we can conclude that a haptic interaction with a virtual object is effective or not since most of the participants received some assistance though the level of assistance varied to accomplish the tasks However we need to focus the results on tasks 7 and 8 to 102 identify the cone and cylinder Most participants completed these tasks with little assistance and the time required was not longer than for tasks 1 2 3 and 6 The objects in tasks 7 and 8 were considered less familiar objects and have more complexity than the sphere and cube We need to test a greater variety of objects to prove the effectiveness of haptic interaction 10 4 Satisfa
34. ction Users satisfaction was measured by the after task questionnaire 4 and after test questionnaire 8 After task questionnaire 8 asked about how users felt after experiencing the haptic interaction They all responded with positive answers such as interesting fun and amazing The satisfaction was also measured by users confidence to manipulate the PHANToM machine The average point rating was always above 4 on the 5 point liker type scale Even though all participants failed tasks 9 and 10 they all showed a positive feeling towards haptic interaction 10 5 Analyze the research hypothesis Hypothesis 1 by using haptic sensory cue people are able to perceive geometric properties of the simple familiar objects such as cube and sphere 103 This hypothesis is analyzed by the results discussed in 8 2 the accuracy of information perception Several pieces of evidence showed that participants were able to perceive geometric properties of virtual objects All participants understood the following features of objects before test personnel provided assistance e Sphere was something round e Cube was something with corners e Cone has some narrow top round surface which became larger and larger as they moved the stylus to the lower part of the object and the flat surface at the bottom e Cylinder has a flat surface on top and round surface below e Rotated cube has some shape like house roof pyr
35. cts The rotated cube was one of the difficult objects to identify and all participants failed this task All of them recognized the top part as house roof diamond or pyramid However none of the participants was able to recognize the lower part of this object One of the reasons was that they were not able to have the stylus stay on the surface of objects At the edge of bottom of the pyramid or roof the stylus slid down and they lost contact with the object For sighted people the cube in rotated position gives a more realistic view of the three dimensional object However the unexpected position of an object made the participants confused Sj str m also discussed this unexpected object orientation Sj str m 2002 The torus was the most difficult object to identify and they all failed this task All participants explored only a small portion of this object One of the reasons was that there is a hole in the middle of this object so that they were unable to have a stylus stay on the object It was easy to lose contact with the object Sj str m and his colleagues Magnusson Rassmus Grohn et al 2002 tested an object with much finer details They asked test participants to identify a grand piano and a stool Their result showed that 20 out of 24 participants were able to recognize these 97 objects In their work it was not mentioned whether they provided some help or not However our results showed that even with much simp
36. cube use the default force The force is sent when the virtual stylus collide with these object Rotated cube is exactly same as one in Figure 4 4 except it is rotated 45 degrees around x y Z axes Torus has a donuts shape and users receive the force only when the virtual stylus is on the ring shape 45 5 Experiment with Audio and Tactile Mode 5 1 Introduction Visually impaired students in Iceland use screen reading synthetic speech software audio and Braille display tactile This experiment assesses the way they use these solutions to accomplish certain tasks by using the usability test techniques and finds areas that need improvement or can be assisted by some other methods By assessing the usability of each tactile and audio mode the research hypotheses 4 in Section 2 3 Interacting with computer with single mode tactile or audio has limitation for perceiving study materials but each mode can be used in its suitable presentation of information is investigated The purpose of these tests is to collect information about how blind users work with computers in the Windows environment with the screen reading software with synthetic speech or with the Braille display output and whether they experience difficulties performing simple and more complex tasks The rest of chapter 5 describes the following the experiment method the problems to be tested participants the test environment tasks to be tested roles of test p
37. ds to be persistent i e available as long as needed One recommendation Leporini and Patern 2004 that has been made to increase satisfaction is to provide with each web page a short sound that tells the user that it has been loaded While this can be a good suggestion one has to be careful that a short audio signal is very transient Our case study showed that such feedback would not merely increase satisfaction but is likely to increase effectiveness or efficiency Open problems Communication is particularly important to blind people in order to break their isolation Textual e mail is an example of a translated solution for blind people where a native approach would be more helpful We can propose that e mail is popular communication instrument because it is asynchronous i e the receiver does not have to be available and the sender can enter text and edit it until he she is satisfied Finally the receiver can at his leisure read and re read the e mail For a blind person a more direct approach would be to allow him her to record his message with an editing tool that can revise the message if needed If he she is communicating with another blind person he she can listen to it as many times as needed just as he she had done if the message had arrived in textual form If the recipient is sighted he she might have an 85 option of translating the audio message to text This technology is available but in limited languages
38. e and gravity of three dimensional objects in the virtual environment Most Haptics study have been focused on the development of hardware in the research institution but in recent years commercially produced Haptics device made it possible to focus on the application side of Haptics study In this study the PHANTOoM Haptic device from SensAble Technology Inc http www sensable com is used PHANToM Haptic Device and GHOST SDK There are several haptic device models from SensAble Technologies Inc In this study the PHANToM Desktop Device is used This device has the Encode Stylus which provides 6 degree of freedom positional sensing http www sensable com The user holds the pen like stylus which provides force feedback in x y and z coordinates and kinesthetic motion of yaw left and right pitch up and down and roll 360 degrees The force is received by users based on one point interaction which means that users feel the object in the three dimensional virtual environment as touching the real object with a point of pencil 22 Haptic applications with the PHANToM Desktop device are developed with the GHOST SDK General Haptics Open Software Toolkit The GHOST SDK is a C object oriented toolkit that allows developers to create haptic environments which includes generating the haptic scenes manipulating the properties of the scene and objects within the scene and controlling the resulting effects
39. e pages long document This person was not satisfied with the time he spent On the other hand this participant spent 38 minutes to edit a two pages long document with Braille display and he was Satisfied with the time he spent 82 Observations recorded during the task to edit the document tell us that with Braille display participants had good control over the information flow Especially for the task finding the specific position in document they commented that Braille display work better over screen reader The reason relates to the nature of listening and touching reading Parkin and Aldrich described reading and listening perception as Listening is a much slower process than reading and many of the strategies available to the reader e g highlighting underlining and skimming cannot be carried out with tape More importantly listening is essentially a passive process making the listener far more vulnerable to distractions than the reader Parkin and Aldrich 1989 9 3 Analyze observations Observing blind students carrying out the tasks was satisfying because we learned about their problems and their accomplishments We discovered that the assistive technology was built and provided to adapt the blind world to the sighted world In this section observations during experiments are discussed in each task For the experiment with screen reader there are three participants and for the experiment with Braille display t
40. e the mental load and this is not an efficient way of learning according to Sweller Sweller 1988 In this sense using Braille display for this type of tasks creates less mental load because participants are able to 93 control flow of information in a better way Participants made comments that they were able to think at the same time they touch the line of information This comment implies that solving a problem by listening is not an efficient way to work In the task of searching Internet similar phenomenon occurred The amount of information in each web page that participants had to scan is a lot more than the readers had to because every page is new and they had to listen or touch everything written until they found things they were looking for Aldrich and Parkin described the lack of control over information flow such that since the listener does not have good control over the information flow they become passive and this passivity leads to decrease in the concentration Aldrich and Parkin 1988 This implies that the listener needs to pay more concentration to perceive same amount of information than readers In this study we did not include the experiment using both screen reader and Braille display together However the participant comments using screen reader for scanning information and Braille display for detail work which require more focus on that information confirms that the two modes reciprocally cover ea
41. eading 14 and writing mathematics such as complex equations and they are developing tools which translate the Nemeth Code and regular mathematical expression into LaTex and vice versa Karshmer Pontelli et al 1999 The Nemeth Code is used among blind students to read and write mathematics Sribunruangrit et al developed the Braille Box to access the graphical information on a computer Salisbuty Conti et al 2004 It allows users to explore the figure such as a black line on a white background and distinguish the colors by sound Although such tools are developed the most commonly used solution for graphs is manually created with the rubber bands on a cork composition board mounted with a rubber mat which has been embossed with a grid of 1 2 inch squares For geometric objects teachers describe the object verbally and use the real objects to touch 3 3 Cognitive Load Theory The Cognitive Load Theory established by Sweller Sweller 1988 states that the best learning process occurs under conditions related to the human cognitive architecture The learning structure is described from the human information processing system with long term memory and working memory Success or failure of a human learning process depends on how a person s working memory and long term memory are used Working memory is the temporary information storage that 15 humans use to keep new information This memory is used as a kind of workben
42. ed better than auditory Severin 1967 Sewell and Moore 1980 Even in the case of four line poems Menne and Menne 1972 and nonsense syllables Chan Travers et al 1965 the retention of information is better for visually presented format Najjar 1998 Graphically presented information such as drawings and images are frequently used to increase the effectiveness for leaning process because in many cases images can provide more information instantly than textual format Several studies Lieberman and Culpepper 1965 Paivo and Csapo 1973 Nelson Reed et al 1976 show that common objects presented as pictures are recognized and recalled better than their names presented in textual form Najjar 1998 Reading is an interaction between human and printed information The reader can select where to read and what to read within the written information But listening to the same information is a passive process Unlike listening to information eye movement over the printed information catches the paragraph separation size of texts words and sentences At the same time eye movement organizes printed 20 information 3 5 Haptic Interface Haptics is the sense of touch which includes both tactile perception and kinesthetic or proprioceptive perception Fritz and Barner 1999 Tactile perception pertains to the cutaneous sense but more specifically the sensation of pressure rather than temperature or pain Kinesthetic perception perta
43. ement of network connection the accessibility issues have increasingly been raised The International Standards Organizations ISO has released its first edition of the technical specification ISO TS 16071 Ergonomics of human system interaction Guidance on accessibility for human 34 computer interfaces in 2003 ISO TS 16071 is the guidance on accessibility for human computer interface and has developed for use by designers of development tools and style guides user interface designers developers buyers and evaluators ISO TS 16071 Prior to this release many organizations companies and institutions have published their own accessibility guidelines and standard The best known standard is Web content accessibility guideline by W3C IBM issued accessibility guidelines for general software Web applications Java applications and hardware Apple Computer committed to the accessibility issues through strengthening its operating system by referring to Universal Access features Macromedia issues some accessibility solution for creating Flash content and for web applications working with its products on the website Sun Microsystems works on Java accessibility issues and some assistive technology products including a variety of alternative keyboards and pointing devices The importance of ISO TS 16071 is providing the consistency into the huge amount of already issued accessibility guidelines and topics Th
44. entertainment None of participants has experience the haptic device previously Test Environment The testing was conducted in various places where participants felt comfortable and without noise The selection of a test place is particularly important when testing visually disable people since they usually have difficulties to be comfortable with new places http www ittatc org technical access ucd ut_conduct php Since the majority of participants are school aged children the testing site was carefully selected to make them comfortable The usability testing site was furnishes with following equipment e PHANTOM haptic desktop device e GHOST SDK v 4 0 for PC platforms running Red Hat Linux operating system version 9 or above glut and glut development package for openGL Intel 300 MHz Pentium processor 30 MB free hard disk space 64 MB RAM and e Hardware graphic accelerator 64 The testing site accommodates a tape recorder and test personnel Test personnel There is one test personnel who observes the test session The test personnel monitor and documents ongoing test activities and reads out the test instructions and task scenarios Since there is no help resource available in the prototype application participants are allowed to ask questions to the test personnel for helping them to proceed with the tasks The test personnel also conduct the prepared questionnaire and the short interview after each task and
45. ers Results from performing these tasks evaluate the hypothesis 4 in section 2 3 Interacting with computer with single mode tactile or audio has limitation for perceiving study materials and each mode can be used in efficient and effective way to perceive different information or to perform different task 54 Tasks to be tested Sending e mail The participant uses any e mail software that he she is familiar with The receiver s e mail address is provided Organizing contacts in the e mail software The participant uses the same e mail software as the one used in the previous task The participant adds a new e mail address name of the recipient post address and a phone number into the already existing contact list And after adding this new contact the participant searches for this contact from the list This task is performed only for the usability test with screen reader Editing Microsoft Words document The test participant works with Microsoft Word The test participant opens the existing file edits write new texts delete and change texts the particular file provided and saves the edited file Searching for particular information on the Internet The test participant works with some Internet search engine site First he she needs to connect to the Internet and open the Microsoft Internet Explorer The test participant may use any search engine site that they are familiar with The topic of information t
46. ers to use the visual cue to interact with the computer it is assumed that the level of participants concentration may be high and the think aloud method may interrupt their concentration There is one modification to the original reflection method Normally during a usability test session the tester speaks as little as possible and does not make a conversation with the test participant According to Hanna et al Hanna Risden et al 1997 children tend to ask for help when they are not certain what they are doing This fact applies to blind people as well The blind students who participate in this experiment normally receive constant assistance from teachers during lectures at their school They are used to ask questions whenever they are not sure about what is going on Therefore this experiment method allows participants to ask questions during the session Test participants perform certain tasks given by test personnel to 60 conceptualize the virtual information provided They answer the questions using liker type scales and respond to short interview The questionnaire asks specific questions about details of the users performance and perception The factors measured in the questionnaire are how well they are able to control to what degree how confusing when they feel virtual objects and how real they feel the virtual objects 61 6 3 Design of Experiment 6 3 1 Overview Age 11 52 RAR Experimental Treatment
47. ersonnel and evaluation measurement 46 5 2 Research Methods Several usability test techniques are used The usability method used is the modification of think aloud method called reflection method Boren and Ramey 2000 Baauw and Markopoulos 2004 combining together with questionnaires a short interview and user observation The most common usability test is combinations of two or more methods with think aloud method Nielsen 1993 It is concluded that think aloud method is not practical for following reasons Participants listen to audio extensively through screen reading synthetic speech output Participants rely on only one sensory cue to interact with computers It is assumed that more concentration is required with one non visual sensory cue to interact with computer since the bandwidth of tactile and audio sensory is much smaller than visual sensory Fritz and Barner 1999 Therefore think aloud method may increase extra mental load during performing tasks Reflection method conducts a short interview to the test participants immediately after the completion of each task and test This short interview is as effective as the think aloud method to gather the information Boren and Ramey 2000 Nielsen Clemmensen et al 2002 47 Baauw and Markopoulos 2004 on what the test participant is thinking during the test session In this study we are interested in collecting the subjective data via a questionnaire
48. estion was conducted for both sphere and cube This question was designed to evaluate whether effects or presentation differences help participants to understand the object easily The results are summarized in Table 8 1 The column Preference Effect shows how many users rank this effect 75 with 1 easiest to feel The accomplished time needs some adjustment when analyzed Most of participants accomplished tasks with some assistance by the test personnel The degree of assistance provided is labeled with H high M medium and L low The degree of assistance is labeled subjectively based on the audio tape review and user observations Table 8 1 Average time to identify the object Effect Object tested Time sec Preference Sphere in the free space 96 0 3 out of 5 Cube in the free space 150 0 3 out of 5 Sphere in the bounded space 110 0 1 out of 5 Cube in the bounded space 64 0 3 out of 5 Sphere with magnetic effect 35 6 3 out of 5 Cube with magnetic effect 103 0 1 out of 5 Note Data indicates user s preference of effects used with identified objects Data is read as how many users rank this effect is easiest to feel Though all participants accomplished the task to identify the object within certain time with assistance of test personnel the first response was often different Table 8 2 shows such responses 76 Table 8 2 Users response for identifying object Task 1 6
49. et information at the same time they are able to think On the other hand with screen reader they need to memorize all information in their heads before solving the problem This caused extra mental load during learning There were no recommendations or open problems derived from this task Task scenario Solve a logic problem Screen reader amp Braille display All participants were able to solve the problem with both screen reader and Braille display With screen reader 91 one participant was not satisfied with his performance although he solved it the fastest When this task was performed with Braille display participants did not have to read touch as many times as with screen reader Recommendations Assistive technology needs to make information persistent In the case of a screen reader this means that the student needs to listen to the problem description repeatedly Continuous confirmation is needed from the computer that the student is working on the solution in the right direction In contrast to the calculation problem the logic problem was very difficult because everything had to be done only in their minds If there is some solution for providing an external memory as the calculator or the spreadsheet were in the case of the calculation problem we believe their way of learning changes quite effectively Open problems Since auditory information is transient it might help students to decompose the pr
50. eveloped so that blind students can access the abstract mathematical information These two areas of works give great benefits to the community of blind people This study combines ideas of these two areas of work based on the concept of the Cognitive Load Theory established by Sweller Sweller 1988 We seek effective and efficient ways of interaction with computer by utilizing more than one human modality without visualization The study aims to emphasize representation of computer oriented information for mathematical learning for which blind students have difficulty in staying independent while learning and to achieve similar goal as sighted student does 2 2 Research Objectives Objective 1 To investigate the effect of haptic cue on accuracy of information perception One of the objectives of this research is to investigate the accuracy of information perceived through haptic interaction without visualization In order to use the haptic sense correctly in the multimodal interface it is important to examine how accurately users are able to perceive virtually presented information through the haptic device Accurate perception includes how well users are able to manipulate haptic device and to what degree Haptic device is often used to increase the sense of realism in a virtual environment Many applications developed with haptic devices are often combined with visual cues For example surgical simulators for medical training
51. fect helped users to trace the surface of the virtual object easier Adachi Kumano et al 1995 106 The above evidence shows that extra effects on the object enhance perception 3 Make sure that the models are haptically accurate and work without vision The magnetic cube did not have haptically accurate effects The force was not calculated in real time so that force was sent not smoothly It sent the right force when the participants moved the stylus slowly When the stylus was moved fast the participants did not feel the right force and got confused This fact showed that the haptic model needs to send the accurate force in a natural way Much research has been done to develop better algorithms to support creating the haptically correct effects Anderson 1996 Fritz and Barner 1999 Raymaekers DeBoeck et al 2005 4 Be aware that the orientation of the object matters Rotated cube was used to examine this point Participants experienced three cubes which were haptically represented differently before they tested the rotated cube The rotated cube is similar to the one in bounded space except for its orientation All participants failed to understand the orientation of the object They understood only a portion of the object 107 Though it was difficult to understand the orientation of the object as Sj str m pointed out in real context objects are represented in various orientations A new experiment is required
52. h certain tasks was similar to the audio user interface while the users satisfaction rate was higher Haptic user interface was tested with a prototype application developed for this study on a haptic machine PHANToM In the haptic experiment we have focused on the accuracy of information perceived by users through haptics Users were able to guess the shape of virtual object but identifying the details of the object with the prototype is not accurate for the purpose of mathematical learning use At this point we concluded that haptic only perception has a limitation to interact with computer We presented the effects of the single mode as a result of this work It is important to understand how blind students interact with computer through the single mode in order to develop the effective and efficient multimodal user interface By understanding the effect of each mode we are able to optimize the use of each mode when combining them together The study analyzed the recommendations and open questions directed to audio screen reader and tactile Braille display user interfaces The further research is required to investigate the strength of haptic interaction We have presented the suggestions of further research and hypothesis that needs to be evaluated in the future work Content Te INtTHOGUCHION EAEE ees a gaes E E a aedea sats Rosine Giaadde Meuavandenaatne deh ganaeds 1 2 Research Objective and Hypothesis
53. he standard has to be practical and must be widely used by the wide range of audience addressed by the standard ISO TS 16071 tries to cover many topics in accessibility but there needs to be more depth in each guideline in the section 7 Also the guideline must cover the 36 situation that multiple accessibility features applied for different users who have different accessibility demand For example when the web application is developed the developers often do not know so much about what types of users will see and use their products It might be used for both visually impaired users and auditory impaired users at the same time Also to make standard more practical the standard way to assess the accessibility is needed to ensure that the products guarantee the accessibility for certain levels for consumers This will benefit the designers and developers of interactive systems because the test procedure keeps the level of accessibility quality 37 4 Haptic Prototype 4 1 Introduction The prototype application was developed with openGL for graphical rendering and GHOST SDK for haptic rendering along with C in Linux environment Fedora Core 1 The prototype application consists of three main parts which represent the two simple geometrical objects sphere and cube in three different ways These objects are presented as follows 1 Objects residing in the free space with the default force effect which is calculated by GH
54. here are two participants These two participants also tried experiment with screen reader 83 Task scenario Send E Mail Screen reader The participants solved this task efficiently There were operating systems network problems incomprehensible error messages that did not catch the user s attention and navigational problems within a web browsing application Sighted people also have problems with incomprehensible error messages and they too can overlook error messages Sighted people conclude that there are network problems from an inactive browser a message indicating that it is still attempting to retrieve data or error on page Braille display One participant did not solve this problem because the new mail system was just installed and this person was not able to use it Another participant solved successfully but encountered the error Recovering from errors for blind users is more difficult task than for sighted people because encountering a new situation is often difficult task for blind users in any circumstances Recommendations Provide additional feedback on network connections and processing status The feedback of the status of all systems is particularly important to blind people and they often rely on asking an assistant The error feedback is particularly necessary for blind people It needs to be 84 appropriate to that particular situation and to provide easy to follow instruction The feedback also nee
55. ill change not only the way visually impaired students study science but also the way they use computers Objective 3 To investigate the effectiveness of information perception with audio and tactile sensory cue for learning Hypothesis 4 Interacting with computer with single mode tactile or audio has limitation for perceiving study materials and each mode can be used in efficient and effective way to perceive different information or to perform different task Currently blind students use screen reading and or Braille display to interact with computer when they study at school Though these students are able to listen or feel to be learned information it does not imply that they perceive this information in efficient way for effective learning Sweller Sweller 1988 states that overloaded information to the human memory affects human s learning process In order to provide the efficient learning material for blind students it is important to understand how they perceive computer oriented information through common computer tasks Sweller s cognitive load theory is often 11 applied for designing multimedia learning with utilizing more than one mode Sweller Merrienboer et al 1998 Sweller et al suggests combining different sensory cue to increase the efficiency and effectiveness in learning Since blind students do not use visual cues which have the highest information perception bandwidth we need to understand
56. ins to the feeling of motion relating to sensations originating in muscles tendons and joints Proprioceptive perception is the sensory information about the sate of the body including cutaneous kinesthetic and vestibular sensations Oakley McGee et al 2000 Table 3 1 Properties of Sensory Channel Fritz and Barner 1999 Visual Audio Tactile Kinesthetic Physical Mechanism Light wave Sound wave Suiface koree length texture velocity Information Flow amp Unidirectional Unidirectional Unidirectional Bidirectional Characteristic Passive Passive Passive Active Information Bandwidth 10 10 10 10 teas Note the table was adapted from Fritz and Barner Fritz and Barner 1999 Webopedia http www webopedia com describes that Haptics in the computer study is the science of applying these perceptions and control the interaction with computer applications by using haptic input output devices The technology of haptics has been developed and improved 21 in the fields of robotics virtual reality and artificial intelligence Most Haptics devices are robotic equipments which provide the kinesthetic force feedback As an input device it tracks the user s physical manipulation and as an output device it provides the touch sensation coordinated with events in the screen By utilizing this kinesthetic force feedback users are able to perceive the information such as shape textur
57. ionnaire 2 asks whether it was easy to point out the specific place of the object task 1 6 For 4 instances out of 6 in total the average response was 5 on a 5 point liker type scale For the first two tasks average was 4 5 and 4 2 The more they touched the objects the easier it felt to control the object 10 1 3 Level of realism in virtual reality The level of realism was measured by the after task questionnaire 4 which asked whether participants were able to feel the object as easily as touching is with their real fingers For 4 instances out of 30 participants responded with over 3 points on a 5 point liker type scale The average through tasks 1 6 was 2 8 points which indicated that they could not feel the object as well as touching with their fingers If one touches a real object with a pen point one may not feel it as well as when touching with fingers One observation that needs to be pointed out is that during the test session three blind participants started using both hands as they tried to touch real objects Compared with using their fingers the level of realism is low but the observation tells us that they did feel some realism in the virtual environment One participant commented that the haptic touching was much more realistic than this 99 person expected Another participant described haptic interaction with the PHANToM as being like touching the real object with some tool Three blind participants almost used ano
58. is is the general guideline to cover the accessibility issues in interactive systems ISO TS 16071 ISO TS 16071 is unique compared with other guidelines or standard which many organizations and companies issued by focusing on some particular interactive system such as Web content 35 ISO TS 16071 approaches accessibility as an extension of usability Even though the web page is readable by the screen reader if it is very difficult to accomplish intended task this web page is not accessible for those using the screen reader The guidelines become practical when usable by the audience whom this guideline addresses Audience may not be professionals in the usability engineering nor accessibility topics If considering the non professionals in accessibility vocabularies in the guideline may be unclear The wording in ISO TS 16071 is very carefully put together so that it does not address any particular design or products It is understandable the way the vocabularies are used since this guideline s purpose is to provide the wide range of accessibility solutions to interactive systems including office applications web pages and multimedia On the other hand this fact makes it difficult for some kinds of audience to follow the contexts of guideline It may not be easy to address the guideline to wide range of audience and products The contents become so general that it is not easy to use as guidance for an ordinary programmer T
59. is orthogonal to this plane 4 Compute the intersection point on the plane with vector n see figure 4 7 5 If HIP is inside the inner cube the direction vector n n is toward outside And if HIP is between inner cube and outer cube the direction vector n n is toward the axis which is parallel to the plane closest to HIP Using Hooke s laws the anchor of the spring is 43 the axis of coordinate system and the length of the spring is a half of the side s of cube If the magnitude of vector from HIP to the axis is larger than s 2 the effect is as if the spring is pulled and the spring tries to pull back toward the axis If the magnitude of this vector is smaller than s 2 the effect is as if the spring is pushed and the spring tries to stretch toward outside The length from the intersection point see Figure 4 6 to the y axis is s 2 6 Force k s 2 n n where k is the constant integer l Graphically 4 presented cube l Boundary cube The are that the magnetic effect is applied Intersection point on the plane with vector n Figure 4 6 Figure 4 7 Concept of magnetic cube Calculation of magnetic cube 44 4 7 Cylinder Cone Torus and Rotated Cube Figure 4 8 Figure 4 9 Cylinder in bounded space Cone in bounded space Figure 4 10 Figure 4 11 Torus in bounded space Rotated cube in bounded space Cone cylinder torus and rotated
60. ish each task b Level of easiness to use screen reader Braille display 2 Effectiveness a Number of tasks completed 3 Satisfaction a Level of satisfaction with the time spent for each task b Level of satisfaction with own performance for each task Figure 5 1 Overview of Experiment with Audio and Tactile Mode 49 5 3 2 Problem Statement and Usability Test Objectives The goal of the two usability tests is to gain data relevant for user interface designers The designer needs to understand how different human modes can be utilized in user interfaces to increase the accessibility for blind people Therefore these tests collect information about the correlation between human mode and computer interaction by using the screen reader or Braille display The test assesses the hindrances and aids of the screen reading software synthetic speech and the Braille display as well 5 3 3 Resources Test Participants Three blind test participants are selected from school age children Age range is from 13 years old to 18 years old They have experiences in working with the screen reading software speech synthesizer and Braille display They use computers as their learning assistive tool on a daily basis Test Environment The testing is conducted at their most comfortable place such as at their home or school Since visually disable people have difficulty in adjusting to a new environment places participants must work a
61. ithm computes the collision information between the simulated stylus and virtual object such as where and how they collide Force response algorithm uses the collision state and position information to compute values such as force and torque vectors Salisbuty Conti et al 2004 The control algorithm minimizes the error between the ideal and applicable forces Salisbuty Conti et al 2004 The simulation engine uses the same interaction forces as sent to the control algorithm from force response algorithm to compute the effects on virtual objects Salisbuty Conti et al 2004 In order to feel the 25 object in the virtual reality as real as possible improvement of algorithms in the haptic rendering component is an active research area Computation of Contact Force Figure 3 3 shows how the surface contact point is used in order to compute the contact force for the PHANToM haptic device SensAble Technologies Inc This is also called the 1 DOF direction of freedom interaction Salisbuty Conti et al 2004 In this thesis only this calculation is used In the visually representation of the object the user sees the position of simulated stylus on the virtual object only as if the position is at the surface contact point Let the surface point be Ps and the actual stylus point Pa The force is calculated by using the Hooke s spring law Force F is a multiplication of constant k which is known as a spring constant and x which is a di
62. jects by letting participants select the real object Magnusson Rassmus Grohn et al 2002 Because their test participants chose the real object which they were able to touch both hands as their answer we assume that participants might have been able to guess the object rather than identifying objects correctly There might be data errors in identifying the properties of objects accurately in their study In order to provide the independent learning solution to blind students it is important that they do not guess haptically represented information Studying to what extent people are able to conceptualize the simple geometric object with their geometric properties and to manipulate the device in the virtual environment develop the idea of more concrete application for blind students Hypothesis 2 The different presentations haptic effect and reference point of the same geometric object will improve the perception of haptically presented information Sj str m developed the haptic interaction guidelines Sj str m 2002 In this study we selected following six suggestions from the guidelines to test 1 Use rounded corners rather than sharp ones 2 Consider different representations to enhance different properties 3 Make sure that the models are haptically accurate and work without vision 4 Be aware that the orientation of the object matters 5 Provide well defined and easy to find reference points in the environment 6
63. k the level of difficulty to perform the task are used to measure the mental load during the task performance Bevan and Macleod 1994 In this study the after task questionnaire asked whether it was easy to accomplish the task with screen reader Braille Display Answers were either Yes or No In order to measure the level of mental load in details it was necessary to apply 81 liker type scaling In after test questionnaire questions 1 5 ask about the level of ease to work with screen reader or Braille display with 5 point liker type scaling Response from participants P2 and P3 who have participated in tests both with screen reader and Braille display indicate that ease of use and work with Braille display is higher According to Bevan et al this result implies that mental load during the task performance was lower with Braille display 9 2 Effectiveness With the screen reader in 5 instances out of the total of 20 participants failed to perform the task With Braille display there was 1 failure out of 10 instances In this case this participant s mail sever was not connected because the school just installed the new mail system on the day when the test was conducted Both with screen reader and Braille display editing MS Word document and searching requested information over the internet were most time consuming tasks Only one participant accomplished the editing task with screen reader and took 40 minutes to edit a thre
64. lay information and the way humans interact with them Visualization is increasingly important as a means for people to understand complex information Visualization is concerned with exploring data and information in such a way as to gain understanding and insight into data Edwards 1995 Computer based visualization techniques depend almost entirely on high resolution graphics For blind users the problem of using such displays is vast Fritz Way et al 1996 The access to and learning of mathematical information has been one of the greatest challenges for blind people since abstract mathematical information is generally represented by graphs equations or other highly visual manners According to Blindraf lag islands Blind Association of Iceland visually impaired students have serious difficulties in understanding abstract scientific information which leads them to abandon studies of disciplines with a strong scientific component Most commonly used techniques for displaying computer information non visually rely mainly on screen reading synthesizers operating with scanner and optical character recognition software audio interface and Braille cell displays and Braille embossing printers tactile interface To visualize and item like a picture blind people use tactile imaging Tactile imaging is the process of turning a visual item into a touchable raised version of the image so that this tactile rendition faithfully re
65. learning at school is receiving information with attention Listening to information is more difficult than visually scanning the same information due to the transient and temporal nature of audio Arons 1997 The auditory sense can receive the incoming information from any direction and the auditory input has transitive nature There is no similarity to visual sense which allows human to scan information and use the selective attention Wickens and Hollands 2000 The auditory input such as sentences or words is heard and then ends On the other hand the written word remains on the page Therefore auditory processing requires holding on to a stimulus before it disappears Auditory processing has pre attentive characteristics which are more critical than in vision Wickens and Hollands 2000 Since the information loss in working memory occurs when it is displaced by the presence of other information new information that has been attended Pashler and Carrier 1996 receiving to be learned information by listening especially during class lecture is not effective unless the listener can control the amount of information listened to 19 When humans retains information over longer periods of time visually presented information text appears to be better than auditory for communicating verbal information Najjar 1998 When verbal information is a list of words and instructions visually presented information has been shown to be retain
66. ler object with high level of assistance our participants had no clue and could not give any response about this object Further testing is required to confirm how much detail of virtual objects people are able to understand with a haptic mode One question raised related to the work done by Sj str m and his colleagues Magnusson Rassmus Grohn et al 2002 is whether users are able to understand a virtual object which is unknown for them in their real world especially for younger blind users whose experience in the real world may be much more limited 10 1 2 Achieved completeness working with virtual objects The achieved completeness of working with virtual objects was measured by how accurate and how well the participant placed the stylus on specific positions of the objects Once participants understood the object they all successfully placed the stylus on the specific position of the object There were few instances where the participant did not place the stylus on the exact positions but they all understood and felt the virtual space correctly and manipulated the machine accordingly This is an important fact when we develop the useful application with haptic mode for blind users which helps them understand information not accessible by the current solutions The amount of time to place the 98 stylus on the object only task 1 6 is considerably short and this time became shorter as they experienced more objects After task quest
67. mmary of the after task questionnaire with average scaled points Values were rated with a 5 point liker type scale where 5 points indicates strong agreement After task questionnaire question 1 Q1 is used to measure the level of easiness to identify shapes question 2 Q2 for the level of easiness to manipulate a device question 3 Q3 for the level of user confidence to manipulate a device and question 4 Q4 for the level of realism in virtual reality Table 8 3 Summary of after task questionnaire Task Q1 Q2 Q3 Q4 Task 1 3 75 4 5 4 25 Task 2 4 42 42 24 Task3 4 25 5 4 5 3 Task 4 4 4 5 46 3 2 Task 5 4 6 5 48 3 4 Task 6 3 6 5 42 24 8 3 Completeness of Tasks Task 7 10 Tasks 7 10 are to identify less familiar objects see Table 6 2 in section 6 3 5 In these tasks only the short interview was conducted Table 8 4 shows the average time to identify the objects All participants failed to identify a cube in rotated position and a torus in bounded space even with assistance from test personnel In the case of torus shape none of 78 the participants had any clue what it was Cube in rotated position was identified as a house a pyramid a triangle or a diamond Table 8 4 Average time to identify the objects Task 7 10 Object tested Time sec Cone in bounded space 72 0 Cylinder in bounded space 53 0 Cube in rotated position All failed Torus in bou
68. motions After each task test personnel give out a prepared questionnaire and conduct a short interview which is developed based on user observations After the testing test personnel reviews the videotape recording and records further observers ions which indicate user s performance level and emotions 5 3 4 Test Sessions Test Administration The evaluator has a minimum background to accomplish the test tasks and knowledge about screen reader and Braille display The test personnel provide the experimental treatment The test participants receive the test scenario by the diskette and or one of test personnel reads out the test scenario The test participants listen to or touch the task instructions with the screen reader or Braille display and may ask 52 any questions to clarify directions When the test participant is ready to perform a task the test personnel starts the clock to measure the time to accomplish the task After the completion of the test session and the review of the videotape recording it is where an evaluation is completed Experimental Treatment The test participants are given a general orientation of the test environment and the flow of the test sessions General Conduct of Test Session Video Recording The purpose of the video recording is to capture the test session on tape so that it can later be evaluated Pre Questionnaire Pre questionnaire Appendix A Appendix B is given to each test
69. mpleteness of Tasks Task 7 10 ccccceeeeeeeeeeeeteeeeeeeeeeeeeees 78 8 4 Summary of Observations 0 ceeeeecceceeeee eee eeeeeeeeeeeeeeeeeeeeeeees 79 Oe Ti E A iin EAEE A Awd cence Aes 81 9 2 EA ETATE e E denvncatsnuerantsanencatannseastdensscatsausesaes 82 9 3 Analyze ObServati nS sirinin le Re eke 83 9 4 Analyze the hypothesis 4 cccccccscnetesteccenestensecdsteeeenesteneececteecens 93 10 Analysis and Discussion Haptic Mode cccccsseeeeeeeeeeeeeeeeees 95 10 1 Accuracy of Information Perception eeeeeeeeeeeeeeeees 95 10 1 1 Correctness of identifying geometric objects 00 95 10 1 2 Achieved completeness working with virtual objects 98 10 1 3 Level of realism in virtual reality eee 99 10 2 Efficiency tc Ca a Eee Oe BEES a alison ates Soe 100 10 2 1 The time to identify geometric objects ee 100 10 2 2 The time to point out the position of virtual objects 101 10 2 3 Level of ease to identify shapes of objects and to Manipulate AOVICE ccccccccceeeeeeeeeeeeeeeeeaaeeaeeeeeeeeeeeeaaaeneeeeeeeees 101 10 3 Effectiveness inunan a a a SEE 102 10 4 Satisfaction i i Me ae eee ee 103 10 5 Analyze the research hypothesis ce eeeeeeeeeeeeeeeeeeeeeeeee 103 Ti CONUS ON 2 2 20 5 60288 cee ks Ae eae eA ae Ab a a 110 1 Introduction Graphical user interfaces represent a fundamental change in the way computer disp
70. nality and preferences is recommended Comparison of available tools and evaluation of best suitable applications needs to be continuously up to date such as comparison of web browsers with respect to implementing accessibility guidance Gunderson 2004 Open problems It is unrealistic to suggest that ads should be removed and even removed for blind students because they are consumers too It is not a solution to change the graphics to text because it will not diminish their cognitive load An intelligent filter is needed to remove information that increases the user s cognitive load by appearing repeatedly 90 Task scenario Solve a calculation problem Screen reader amp Braille display All three participants with screen reader and all two with Braille display were able to solve the problem and were satisfied with the time they spent on it They were free to use any tool but chose to use Excel and a calculator When sighted people read or solve some mathematical problem they use external memory to draw an idea or take memos For blind people perceiving mathematical information with currently used solutions is difficult because often they are not able to use any resource for the external memory In the calculation task the computer calculator and a spreadsheet were used as external memory which may explain why it was so easy When this task was tested with Braille display both participants commented that they were able to g
71. nded space All failed 8 4 Summary of Observations The summary of user observations Table 8 5 during the test session was Classified based on the problem classification of haptic interaction developed by Sj str m Sj str m 2002 He classified the problems of haptic interaction into three and discussed these problems from five aspects A subset of his classification is used to summarize the observations in this experiment Sj str m classified problems into three and relate them 1 to touch interaction in between real world and virtual reality 2 to virtual interaction of any kinds and 3 to one point interaction Since these three problems are interwoven and closely related we do not classify our observation according to these problems Our observations are discussed from four aspects 1 Object 2 Navigation and Overview 3 Context and 4 Learning 79 Table 8 5 Observed problems in haptic interaction Object Naviagtion Details of a complex object are hard to be understoo by users The accuracy of information perceived is not as high as the one through real hands The orientation of object may confuse users to understand the virtual objet Unfamiliar object is difficult to be understood Incorrect effects on the object made more difficult to interact with one point Since one point interaction allows users to touch one point on one object it is almost impossible to overview the
72. ned to keep the learner s cognitive load at a minimum during the learning process in order to increase the effectiveness of instruction Sweller Merrienboer et al 1998 The working memory model that Baddeley Baddeley 1992 explains has two slave systems for visual and spatial information and for acoustic information When instructional information is presented in both visual and auditory senses at the same time the two slave systems are addressed and the capacity of working memory is used more efficiently Baddeley 1992 Tabbers Martens et al 2004 If some complex information such as He is a brother of my grandmother s husband s cousin s wife is expressed in a pictorial format like a family tree it is much easier to understand and information held in the working memory is reduced To be learned information with complicated elements individual information inflicts the cognitive load because humans remember elements at the same time when they develop the relationship between elements This is one of the reasons that learners have difficulty to learn new material It is important to think about how the learning materials instructional materials need to be presented and designed in order for achieving effective learning This issue becomes more crucial when learners have 17 some disabilities and rely on limited number of modalities to perceive information 3 4 Reading vs Listening to Information and Memor
73. nowledge based economy The eEurope 2002 Action Plan was carried out as creating the foundation of the knowledge based society One of the examples is the increase of Internet connectivity which supplies the infrastructure to carry out certain activities of the future information society eAccessibility The Action Plan was aimed at the inclusion of all citizens such as 33 elderly young and disabled people into this Information Society as it can be read from its subtitle an Information Society for All What is the Information Society for All One of the emphases in the Action Plan was e Accessibility Improving disabled people s access to the Knowledge Based Society European Union Information Society http europa eu int information_society eeurope 2005 index_en htm Its aim is to include disabled people more easily into the society and its labor market by developing an accessible information society Another important and noticeable subject in the Action Plan 2005 is to put users at the center of eEurope society In the IT society human factors play more important roles than ever as seen in eEurope project Accessibility issues are not any special steps during the development of new software products but need to be a part of the software development cycle like the usability issues Accessibility Standard and Guidelines In recent years as the number of web sites and computer users has increased with the improv
74. nterface designers concerning haptic mode to interact with virtual environment Such data involves how well and to what degree of accuracy users are able to conceptualize haptically presented information Moreover the test assesses the hindrances and potential aides of haptic devices for blind users through performing the following tasks identify different shapes of geometrical objects manipulate the device to touch different parts of objects compare different effects and environment setting that objects reside Test participants answer a pre test questionnaire to collect background information about computer use and computer game experience after task and after test questionnaire and a short interview immediately after each task and the whole test to extract detailed and specific feedback about issues described above 6 3 3 Resources Test Participants Five test participants are selected Age range is from 11 years old to 55 years old but all but one of them are students Two participants are sighted computer users and participate in the test with a blindfold while the rest of participants are blind from childhood Four out of five participants have experienced interactive computer games with special 63 computer controllers such as joy sticks Three out of four participants experienced in computer games play games on an every day basis All participants have computer experience and use computer every day for their study work and
75. o be searched for is provided in the scenario This task is performed only for the usability test with screen reader 55 Searching particular information from a specific web site The test participant retrieves requested information from a particular web site Solving mathematics problem requiring logical thinking The test participant solves mathematical essay problem The problem is selected from the Icelandic 5 grade text book The problem is in simple format with at least 10 sentences Solving mathematics problem requiring calculation The test participant solves mathematical essay problem The length of problem is about 30 sentences and problems require calculating the several different items The problem is selected from the Icelandic 5 grade text book 5 3 5 Quality Characteristic and Metrics This experiment collects data based on the usability metric described in the ISO 9241 which consist of the aspects effectiveness efficiency and satisfactions In this study we measure these aspects by evaluating the specific target users after performing tasks by using a computer as an assistive tool In addition to three usability aspects we evaluate the level of users mental load working only with screen reader or Braille display The summary of quality characteristic and metrics is described in Table 5 1 56 Table 5 1 Quality Characteristic and Metric Quality Characteristic Metrics Instrument TE Temp
76. o complete this task One person spent too much time to complete although this person was satisfied with the time took The other one finished much earlier but this person was not satisfied with the time Comparing with the case when they used screen reader they looked more relaxed because they had better control over the information flow They moved to the next line after they understood by touching Though screen reader also has some control such as stop the synthesized speech change the speed or move between lines our participants did not used these features The way they control the information flow with Braille display is similar to the reading technique The blind users are able to spot some particular place in the document and to focus on a particular content easily Recommendations Bookmarks indices table of contents or keywords for long text documents are needed that will enable the blind person to scan a long document easily Leporini and Patern Leporini and Paternd 2004 87 point out that one of the main navigational problems is excessive sequencing in reading information i e that instead of directly accessing a certain paragraph the blind user needs to listen first to the preceding paragraphs Sighted people use paragraphs for visual scanning Indices to numbers can be helpful in a similar way Sighted people will easily be able to spot and go directly to paragraphs containing numbers but blind people do not have
77. oblem into sections Parts of the solution might be connected directly to the problem description Since there was apparently no tool that came to the students minds it is a research question what form or tool can complement current text editing and spreadsheets as a native tool for the blind 92 9 4 Analyze the hypothesis 4 Hypothesis 4 is Interacting with computer with single mode tactile or audio has limitation for perceiving study materials and each mode can be used in efficient and effective way to perceive different information or to perform different task From experiments both with screen reader and Braille display participants used only single mode The hypothesis was established from the issue that audio and tactile perception has much lower information bandwidth than visual perception By the nature of reading there is much more control over the incoming information and it allows the reader to perceive this information more accurately and efficiently The materials to be read stay in front of reader s eyes and readers do not have to memorize all information written there As it was seen in the task solving mathematics participants had to listen to the problem over and over again until they memorized all necessary information to solve problems Calculation was easier for them because they added numbers as they listened by using the calculator Keeping the information and solving problems by using this information increas
78. ollected here is the experience of computer use and computer games machines such as PlayStation The test examines if the game experience affects the performance with the haptic device since these games use a joystick and other hardware devices to interact with on going screen action This background information is combined with the results of the post task and post test interview to evaluate the test results 66 After Task and After Test Questionnaire Prepared questions are asked prior to the short interview Answers to the prepared questions in the post task interview are Likely scaled in 5 levels For post test interview the participant give both Likely scaled and subjective answers After Task and After Test Interviews The tester conducts the short interview immediately after each task and a whole test The interviews are based on some noticeable points during the test session The tester observes the participant and notes down any noticeable points from participants verbal comments facial expression and physical expression Test Time The estimated time of the test session is no more than one and a half hour The approximate time of performing each task is about three minutes This approximation is based on two pilot tests If three minutes past the test personnel may offer the help or suggestion Post Test Review After the test session is completed the test personnel reviews the audio recorded tape of the test session
79. orary a The time to accomplish each Efficiency task a Level of easiness to work only Rey Mie Efficiency revanar with screen reader Braille A3 Ate Q5 Mental display to accomplish each Ate Q4 Ate Q8 Load task b Level of easiness to use screen reader Braille display Ate Q2 Effectiveness a Number of tasks completed a Level of satisfaction with own Satisfaction performance for each task Ata Q 1 b Level of satisfaction with the time spent for each task Ata Q2 Note Ata Q After task questionnaire Ate Q After test questionnaire see Appendix C Efficiency To see how efficiency relates to the level of effort to accomplish the task Since the study is interested in collecting data to see the level of users mental load efficiency is classified into two aspects temporal efficiency and level of mental load human efficiency Bevan and Macleod 1994 Temporal efficiency is measured by the time to complete the task The level of mental load is measured by user s responses in the after test questionnaire which indicate the level of easiness to work only with screen reader or Braille display and the level of easiness to use screen reader or Braille display Effectiveness To see how effectiveness relates to the target goals of 57 the task with accuracy and achieved completeness To measure effectiveness we count the number of task completed Satisfaction To see how satisfaction rela
80. ot used correctly to perceive different types of information in this case audio and tactile information the users are not able to interact with computers efficiently and effectively This point is particularly important for learning process Learning materials must be presented in an appropriate format to make optimal use of the different modalities 2 3 Research Hypothesis In this section research hypotheses and evidences to build such hypothesis are described These hypotheses are established based on four research objectives described in section 2 2 Objective 1 To investigate the effect of haptic cue on accuracy of information perception Hypothesis 1 By using haptic sensory cue people are able to conceptualize geometric properties of the simple familiar objects such as cube or sphere Sj str m et al examined whether blind users were able to identify the objects with small details such as piano or chair They claimed that small details such as legs of piano were difficult to be identified They also used much simpler geometric objects as test samples and their results showed that 80 of test participants were able to identify Magnusson Rassmus Grohn et al 2002 We present the simple and familiar objects with different haptic effects and reference cue as test samples Our participants describe the properties of objects verbally Sj str m and his colleague asked their test participants to identify the geometric ob
81. pace bounded space v 1 V 2 V 5 V 6 1 V 2 V 5 V 6 Rotated position in HIGH Cube bounded space V 1 V 2 V 4 V 5 Note represents the level of familiarity with an object to be identified represents the point to be validated in Sjdstr6m s guidelines described in table 6 1 In each task the participant is requested to identify the shape and to place the stylus on top bottom front back right and left sides of the objects 6 3 6 Quality Characteristics and Metrics This experiment collects data based on the usability metric described in the ISO 9241 Usability consists of the aspects of effectiveness efficiency and satisfaction with which specified users achieve specified goals in particular environments Addition to these aspects the experiment collects data relating to the accuracy of information 69 perception The summary of quality characteristics and metrics is described in Table 6 3 Metrics for efficiency see Table 6 3 is used to measure the efficiency aspect rather than the satisfaction aspect because this point is related to the ease of manipulating a device Accuracy of information perception The accuracy of information perception relates to the target goals of the task with correct response and achieved completeness To measure the accuracy of information perception the following data is used the answer to identify object shapes achieved completeness to place a stylus on virtual object and
82. presents the original information There are textual and pictorial components to such information Fritz Way et al 1996 However when representing the pictorical mathematical information such as graph drawing and tables it becomes very difficult for blind users to understand or learn by tactile imaging and screen reader Such pictorial information has static and dynamic characteristics Perceiving informatin through hearing is not the most efficient way to work with human s cognitive mechanism Working in the auditory modality there is often an inadequate external memory and poor control over the flow of information from that source Stevens and Edwards 1994 People now study the possibility of using haptic modality for visually impaired users to communicate more interactively with comuters The technology of haptics has been developed and improved in the fields of robotics virtual reality and artificial intelligence The haptic technology is now becoming mature enough so that people can focus on its application Several studies by Calle Sj str m have shown the potential of haptics to explore graphically represented computer information for visually impaired users Sj str m and Rassmus Grohn 1999 Sj str m 2001 Sj str m Danielsson et al 2002 Sj str m Danielsson et al 2002 He has examined if visually impaired users are able to descibe and understand three dimensional objects created in a virtual environment
83. rded voice of humans and patches this sound together resulting is a quite large voice database Dolphin Oceanic Ltd describes the formant based system as it has total control over all aspects of the output speech a wide variety of prosody or notation can be output conveying not just questions and statements but a variety of emotions and tones of voice http www dolphinuk co uk 3 9 Accessibility Participants in the European Union initiated a project called eEurope to create a more sophisticated information society One of the central issues in this project is e Accessibility which is to develop a more 32 accessible information society for a wide range of people Accessibility issues have become more important than ever as computer population increased and many corporations and institutions have developed accessibility guidelines used for their products In February 2003 ISO published its first edition of guidance on accessibility for human computer interface ISO TS 16071 Guidance on accessibility for human computer interface ISO 16071 is reviewed along with the discussion about the demand of accessibility in the society Demand of Accessibility in IT Society eEurope As a strategy of economic social and environmental renewal among participants in the European Union the eEurope 2002 Action Plan was endorsed eEurope s long term goal is to create an information society in Europe and to develop a k
84. splacement of the spring from its natural length such that F kx The x in this case is Ps Pa 26 Simulation Simulation Engine Collision Force Graphic Haptic _ _ 5 js Device Detection Response Engine gt Screen Control Algorithm Visual Rendering Haptic Rendering Figure 3 2 Haptic Rendering Algorithms Salisbury Conti and Barbagli 2004 Note Figure is adopted from Salisbury et al Salisbury Conti and Barbagli 2004 Simulation component computes the virtual environment s behavior over time Haptic rendering component computes the virtual environment s force responses Visual rendering component computes the virtual environment s graphics If sound is used along with graphics the sound rendering is combined in here together The output from the Haptic Device to the Collision detection algorithms is position information of haptic interface point Salisbury Conti and Barbagli 2004 Collision detection algorithms output the information about collision between the object and the stylus and the position Force response algorithms output the ideal force Control algorithms output the force to the user 27 O Previous Position of Simulated Stylus Object Surface Surface Contact Point Spring New Position of Simulated Stylus Figure 3 3 Contact Point SensAble Technologies Inc
85. strict the movement of stylus The big difference from ones in the section 4 2 is that the size of space surrounding the object is much smaller in this case 4 4 Objects with magnetic force effect reside in the free space The special force effect is applied in this case The graphics of objects are same as Figure 4 1 and 4 2 The user can move the stylus freely until the stylus collide with objects As soon as the collision occurs the stylus is stuck with the object The user is able to move the stylus only on the surface of objects The computation of the magnetic force effect for the sphere is based on the sample program provided by SensAble Inc 40 4 5 The computation of magnetic force effects for a sphere The main concept used in the computation is the Hooke s spring law The graphically presented object is surrounded by the sphere shaped boundary Inside the boundary the force is applied The radius of boundary sphere is slightly larger than the graphically presented sphere The force is applied as soon as the virtual stylus entered the boundary sphere The simulated magnetic attraction is toward the center of sphere Computation is done by using direction and magnitude of vectors The steps of computation are as follows 1 Detect haptic interface position HIP the position of the virtual stylus in the coordinate system 2 Calculate the magnitude of vector v v v starts from HIP to the center of sphere
86. t did not show a relationship between game experience and efficiency of haptic information perception 10 2 2 The time to point out the position of virtual objects Another measure of efficiency is the time to point out the position of virtual objects As described in 8 1 2 once participants understood the objects all participants pointed out the position of virtual objects in an efficient manner The efficiency improvement can be seen by comparing the time between task 1 and task 6 This fact indicates that training may improve the efficiency of user performance In task 5 whose object was rated easiest to understand all participants move the stylus instantly This may indicate that added haptic effect on the virtual object affects the efficiency of working with haptic mode in a virtual environment 10 2 3 Level of ease to identify shapes of objects and to manipulate device These points were examined by after task questionnaire 1 2 and 3 and after test questionnaire 4 These questions ask participants whether it is easy to identify to point out position and to control the machine The 101 responses are users evaluations of their own performance The level of ease to identify the object increased by experience except for the time that the magnetic cube was tested which many participants rated as the most difficult object to understand The level of ease to point out the position was always high Even for the task 1 parti
87. t surprising and similar findings have been reported in a study by Pernice Coyne and Nielsen Pernice Coyne and Nielsen 2001 as cited in Gunderson 2004 that reported that whereas users in a control group finished about 75 of web related tasks blind participants completed the tasks about 12 5 of the time Task scenario Search the particular information from particular site This task was performed only with Braille display Both accomplished this task One effectively completed but the other one ran into accessibility problem In this task the municipal web site was used Most of content was accessible i e displayed on Braille board but overflowing information made it hard for them to find what they were looking for The menu was too long Poor usability web design affect 89 blind user directly and severely Poor usability sometimes makes contents inaccessible for blind users The depth of site links needs to be considered Since for blind people scanning the web content is not as easy as sighted people they need to listen to touch contents of web pages every time they encounter the new page The deeper the depth of web page links the more they need to read One participant commented that he avoids using the Internet because working with the Internet for searching information is very time consuming and he she encounters all kinds of problems which he she cannot solve alone Recommendations Training of existing functio
88. t the most 50 familiar place Also at their home or school all the necessary facilities are available to them Working with these tools they need a separate number keyboard if they are not attached with the regular keyboard and speakers or headphone which are all available at their computer working place The usability testing site must be furnished with the following equipment e A desk on which computer equipment can be set up to perform test tasks e Intel P 400 MHz equivalent or higher 125 MB free hard disk space 128 MB Memory or higher e Sound Blaster compatible sound card e PCI or AGP graphics card e Windows 98 or later version of Windows e Microsoft Internet Explorer v5 5 or later e Microsoft Office 98 or later version e Any e mail software that participants are familiar with e Number keyboard e Internet connection e Speakers or a headphone The testing site accommodates a video camera and the test personnel 51 Test Personnel There are two test personnel for each test session One monitors and documents ongoing test activities The other one reads out the test instructions task scenarios and pre test after task and after test questionnaire Test personnel are not allowed to speak while a test participant is performing test tasks unless a participant needs some assistants from them They record observations such as user s verbal comments and facial physical expressions which may indicate user s e
89. tes to the comfort and overall acceptability of the task All subjective data is used to validate this point 58 6 Experiment with Haptic Mode 6 1 Introduction This section describes the experiment relating to haptic cue The experiment assesses the way that users experience the virtual environment only with haptic cues The usability test technique is applied to find evidence for hypothesis 1 By using haptic sensory cue people are able to conceptualize geometric properties of the simple familiar objects such as cube or sphere 2 The different presentations haptic effect and reference point of the same geometric object will improve the perception of haptically presented information and haptics have a potential for allowing students to conceptualize abstract mathematics information The rest of this chapter describes the experiment method the problems to be tested participants the test environment tasks to be tested evaluation measurement and experiment results 59 6 2 Research Methods The usability test techniques are used to collect experimental data The experiment is focused on collecting the subjective data by conducting short interviews combined with user observations and a questionnaire right after the completion of each task and a whole test The reflection method for use in experiment with screen reader and Braille display described in Section 5 2 is used Because this experiment does not allow us
90. the accuracy of information perception is not good enough to provide independent learning material for blind students though people are able to understand some geometric properties However our test results are not strong enough to conclude against this hypothesis 109 11 Conclusion The overall research purpose was to investigate the potential benefits of multimodal user interface for blind computer users in learning Currently the blind students access computer oriented information by using audio or and tactile user interfaces which are considered the best solutions to interact with computers for them In this study we focused on examining each audio tactile and haptic mode to see whether single use of each mode had limitations and what were the hindrance and aide of each mode when the focused users interacted with computers In this study three different usability tests with different human mode audio tactile and haptic were conducted to blind users The findings from this study provide the basis for modeling a multimodal user interface which each mode is utilized in its most optimal way so that users will be able to perceive and process computer oriented information in efficient and effective way for learning Efficient and effective learning in this study based on the Sweller s cognitive load theory Sweller 1988 Sweller stated that efficient learning process occurs when learner s cognitive load is minimized This st
91. the same time is difficult A participant solved the problem as a mental exercise without the aid Learning of drawing or written memos method A participant sought confirmation from tester on what he did He was uncertain about what he was doing 173 Table 7 2 Number of problems Problem category Screen Braille Reader Display Usability and accessibility 24 14 Software fault 6 1 Cognitive mechanism 3 3 Learning method 3 5 TOTAL 36 23 74 8 Results with Haptic Mode In this chapter results from the experiment with haptic prototype are presented The detailed results for each participant are presented in Appendix F 8 1 Completeness of Task Performance Task 1 6 8 1 1 Identifying Object All participants were able to accomplish this task with variation of help The mean times to identify the objects were calculated These are presented in Table 8 1 Though the means were calculated the performance time varies among participants For example to identifying a cube in the free space the shortest performance time was 30 seconds while the longest performance time was 420 seconds or 8 times longer The completion time for identifying the objects in tasks 1 6 reflected with the results of the after task questionnaire Il The question was Rank with number 1 3 to indicate the easiness to identify objects where 1 indicates easiest and 3 indicates most difficult This qu
92. ther hand and tried to touch it as if there was an object right in front of them Redesign of the experiment and questionnaire are necessary to evaluate the level of realism in more detail To understand the level of realism that users feel relates to the accuracy of information perception in virtual reality 10 2 Efficiency 10 2 1 The time to identify geometric objects To measure the efficiency one metric used was the time to identify geometric shapes With visual mode people can recognize all objects nearly instantly Fritz et al described the difference of information bandwidth among different senses in their work Fritz and Barner 1999 The information bandwidth with visualization is highest among visual audio tactile and kinesthetic sensory channels Comparing with tactile sensory channel visualization s information bandwidth is 10000 to 100 times more Considering this fact all participants performed within a much shorter time to perceive haptic information One of participants identified 8 objects out of 10 in total within 1 minute For some cases this person responded in 3 seconds We need to investigate further what kinds of factors associate with performance efficiency In the pre 100 test questionnaire a specific question about game experience with some game instrument was asked to see whether the level of experience in games such as PlayStation or GameCube built a skill of perceiving the haptic information Our tes
93. this ability For this type of task both participants suggested to use both screen reader and Braille display To grasp the summary of documents quickly use screen reader To edit document precisely in details use Braille display Open problems Short keys and other tools that require blind students to be experts at using assistive technology was of limited use since it adds to the cognitive load Blind people have to remember all short keys in order to remember One participant commented that the user manual for screen reader and Braille display is very thick and this person does not feel like reading it and cannot memorize it all Our study confirms in this aspect the result of another study by Theofanos and Redish Theofanos and Redish 2003 Instead applications need to provide interactive help or training that eases user s cognitive load but still provides efficient processing 88 Task scenario Search the Internet with Google This task was performed only with screen reader Only one of the three students was able to complete the task Navigating between contexts proved hard e g between a list of results and a particular web page that the student wanted to examine further As in the case with text processing long documents are also a problem Pop up information interfere students work but this can be turned off Graphical images icons and irrelevant information to the content such as ads interfered This result is no
94. to support users to understand the orientations of objects 5 Provide well defined and easy to find reference points in the environment This point was discussed in 3 6 Avoid objects with small and scattered surfaces Normally objects have specific details to represent their features In our experiment as an example of the object with small surface participants tried the torus shape Torus shape is a donuts shape ring This object has more detailed features than sphere and cube but it is not as complex as a grand piano or a chair that Sj str m and his colleagues tested Magnusson Rassmus Grohn et al 2002 All participants failed to recognize the torus and had no idea what this object was Evidences analyzed along with Sj str m guidelines indicate that people are able to understand some geometric properties of objects However when the accuracy of the information understood and the detail of object 108 are concerned the haptic only interaction with PHANToM still has limitations for our purpose Hypothesis 3 Haptics has a potential for allowing students to conceptualize abstract mathematics information This hypothesis is supported by the results from hypothesis 1 and 2 The experiments in this study focus on possibilities of geometry learning When considering the PHANToM for learning geometry the study showed that there is some limitation such that details of information are difficult to understand and
95. udy however was not able to provide the potential benefit of multimodal user interface as a result though we have shown the 110 limitation of use of single human modality Further studies are needed to present the benefit of multimodal user interface Followings are suggested work needed in the future Audio and tactile user interface this study found evidence than single use of audio and tactile mode for computer interaction has some limitation However the experiments in this study were not designed to provide the solution for this limitation The future works need to look solutions for limitations of audio and tactile user interfaces 1 Investigate the area where audio information perception works best 2 Investigate the area where tactile information perception works best 3 Investigate the efficient way to perceive the information by combining two audio and tactile modes for computer interaction 4 Explore the learning model by using audio and tactile modes for effective and efficient learning Haptic user interface this study showed that the users were able to perceive the geometric properties of virtual objects through haptic only interaction The study also showed the weaknesses of haptic interaction based on our prototype The weaknesses are 1 users do not perceive virtual information accurately and 2 users do not perceive the small 111 details of virtual objects In order to use haptic interaction as
96. ves and if the simulated stylus collides with the object in the virtual environment the reaction force is calculated This force is sent back to the user through the haptic device There are two types of haptic interactions One is point based and the other one is ray based interaction PHANToM Desktop Device in this project is a point based interaction device and only the end point of haptic device interacts with the virtual object This end point is called as the end effector point or haptic interface point HIP The location of HIP is calculated in the coordinate system of the haptic mechanism via sensor 24 and then transformed to the coordinate system in the virtual environment VE where the object resides By using the stylus position information haptic rendering algorithms compute the appropriate output force The computed force depends on the requirement of haptic applications Haptic Rendering Algorithms Salisbury et al state that haptic rendering algorithm computes the correct interaction forces between the haptic interface representation inside the virtual environment and the virtual objects populating the environment Salisbuty Conti et al 2004 The haptic rendering algorithm that is described by Salisbury et al is shown in Figure 3 2 There are three main subalgorithms collision detection algorithm force response algorithm and control algorithm Salisbuty Conti et al 2004 Collision detection algor
97. virtual scence It is easy to get lost in the free space It is easy to lose the contact with the object while interacting with one point touching Context Learning Even with simple and familiar object the context information helped Understanding the force effects does not come naturally Experience made users more easily to manipulate the machine and understand the force effects It is as difficult as touching a real object by a point of a pencil Improving this skill by training is a question 80 9 Analysis and Discussion Audio and Tactile Modes 9 1 Efficiency Bevan et al described efficiency as how much resource is consumed on achieving a task Bevan and Macleod 1994 From a user s perspective spent time or effort required to complete the task are considered as resources Bevan et al Bevan and Macleod 1994 defined two types of efficiency One is the temporary efficiency which is the time spent to accomplish the task Another is the human efficiency which is the level of the human s cognitive load during the task performance In this study we are interested in understanding the human efficiency during the time when participants work with screen reader or Braille display Sweller Sweller 1988 stated that in order to learn effectively the human s mental load needs to be kept low Level of mental load was measured by questionnaire Bevan et al stated that questions which as
98. when all tasks are completed After testing the test personnel review the recorded tape 6 3 4 Test Sessions Test Administration The test personnel have a minimum background to accomplish the test tasks He she provide the experimental treatment All the test scenarios are read out and prototype application is run by the test personnel when participants are ready When the participants are ready to perform a task it is when the time is started to be recorded After the completion of the test session and the review of the audio recorded tape it is where an evaluation will be completed Experimental Treatment 65 The test participants are given a general orientation of the test environment and the flow of the test sessions The participants receive the short training to learn how to manipulate the haptic device and short discussion about the general concept of virtual environment haptic device and haptic interaction General Conduct of Test Sessions Audio Tape Recording The purpose of the audio tape recording is to capture the test session on tape so that it can be evaluated later on Help Resources Since there is no help information available in the prototype application they are allowed to ask questions to the tester during the test session Pre Questionnaire Pre questionnaire is given to each test participant in order to find their background information prior to the task performance The particular information c
99. y Human memory has three phases which it undergoes acquisition learning storage retention as affected by rehearsal consolidation and forgetting and retrieval recognition and recall Wickens and Hollands 2000 These three phases interact with different memory systems For educational learning long term memory and working memory are considered most important Information enters the human brain through its senses visual auditory kinesthetic tactile and olfactory and is temporarily stored for a short time in the short term sensory store STSS Wickens and Hollands 2000 The retention of information stored in the auditory STSS is longer than in the visual STSS Wickelgren 1997 Information entered into the short term sensory storage is not processed further unless the person pays attention to this information this stage is called perception In other words unless humans pay attention to the incoming information through visual or audio sense this information disappears is forgotten in a very short time Information entered auditory STSS tends to remain longer than in visual 18 STSS However this fact is not always true when humans pay attention to information receiving for example during a lecture Information is processed differently in working memory when humans pay attention to it rather than when they do not We focus on the working memory process and the process receiving information with attentions since

Multimodal User Interface Design: Increasing Accessibility to

Contents

Download Pdf Manuals

Related Search

Related Contents