- alt.chi 2013
Contents
1. 21 These potentially allow more convenient eyes free use of mobile devices In addition to this major companies such as Google and Microsoft are developing augmented reality glasses which overlay computer generated visuals onto the users view of the real world allowing them to see the computer output without having to direct their gaze at a specific device 10 Movement activities may also restrict where it is physically possible or convenient to use a device for example whilst it is difficult to run holding your hands out in front of you and using a device while cycling it may be physically impossible to hold and use a standard device because your hands have to be on the handlebars Even on a unicycle with its hands free ride the requirement to look ahead and see where one is going means it is extremely inconvenient to interact with a mobile device Figure 1 Figure 1 Even if hands are free it may be hard to interact In many forms of movement activity such as running fast cycling swimming or even skydiving the movement of both arms and legs are heavily constrained to particular patterns designing interaction for such situations requires a radically different approach For example skydiving photographers often use mouth switches which allow them to either bite or blow to trigger a photograph e g 18 Challenge 3 Terrain Mueller et al s Sensing Body describes how in exertion games the environment aro2. The breathing action makes even mouth based interfaces such as skydivers use somewhat impractical In practice swimmers can play with this slightly by raising their head out of the water they can continue to swim and talk and look around brief hand signals can be incorporated into swimming for communication with others swimming alongside them although this is hard in murky water in competition communication often occurs through body contact From a mobile device point of view swimming also occurs in water so waterproofing is a key concern Finally the low temperature can have effects on ability to move your body accurately and also on reaction time In our reaction test the response time to a rectangle being shown onscreen before swimming was approximately 320ms whereas afterwards it was usually 360 400ms people were roughly 10 20 slower Interestingly whilst cold swimmers relate that this severely affects their swimming in our test the ability to accurately touch a point on a touchscreen was not significantly altered by the cold swimming Terrain Open water swimming happens in a complex environment with currents waves and wind all of which make a big difference both to how much mental effort is required to navigate and how much physical effort is required to swim There are also often obstructions such as riverside trees underwater rocks and chutes which require further mental and physical effort to avoid The depth
3. mentioned above satellite navigation devices have also cautioned against use while driving with some devices even automatically disabling menus when the car is moving The primary approach thus far in driving has been legislation to ban use of mobile devices However even walking can lead to social issues when people don t pay adequate attention to their surroundings and walk into others or walk across the road without looking as observed in reports of several pervasive games eg 4 6 As people move further faster and do more activities while they are carrying complex mobile devices such as smartphones this disconnect between the device design and the use in practice is only likely to increase As an extremely simple example of design for interaction in motion we can consider one of the most common tasks on mobile devices sending an SMS or email message whilst walking This causes problems solely because you are looking at the screen people have their hands available to interact so the touch interface is not a problem Potentially we could apply a design used in driving the rear view mirror except by using the camera on the device to show an onscreen front view mirror so that people can see what they are about to walk into Such apps do already exist on several platforms eg type n walk 8 however they are as yet imperfect because on current devices the camera angle is not correct To work effectively this would
4. of water may make it impossible to stop swimming although swimmers can tread water if they need to pause Finally the cold temperature means that swimmers have to swim hard in order to keep warm this constrains both how much it is possible to swim slowly and relax and how long it is possible to stay in the water All of these factors will mean that the ability to interact with a device will vary greatly depending on the current situation even more so with cold swimming because stopping to interact is much less feasible and potentially dangerous and we certainly cannot assume any ability to attend to a device quickly Other people Cold swimmers often swim in groups who need to communicate with each other for navigation avoidance of hazards and to monitor each other In particular cold swimmers try to be aware of signs of severe hypothermia such as extreme confusion as these can lead people to continue swimming when it is no longer safe for them to swim and may operate buddy systems to minimize this danger This large number of tasks that are already going on is a challenge for interaction design as we must take care not to harm any safety critical communication although the current difficulty of communication and monitoring between swimmers does suggest a potential for technology development Further to this there are other water users such as boaters fishermen and regulatory authorities such as police or landowners how sw
5. on many older mobile devices there are specific buttons for functions such as answering and ending calls on a mobile phone changing tracks on an mp3 player or announcing heart rate on a sporting device Due to the tactile nature of physical buttons these can at least be used while walking and without looking at the device Recently some developers have tried to create similar in pocket interaction interfaces using touchscreens and audio feedback 19 These work well when there is a limited range of actions to be done but when a whole system has to be controlled eyes free it becomes too complicated to use in the pocket as demonstrated both by the comparison with Apple s more generic VoiceOver accessibility software in the study above and the long step by step lists of how to do something as simple as set a clock alarm using eyes free input on a mobile device eg 1 Voice commands are also possible typically in sync with a specific physical button action such as used in Apple s Siri voice control 3 These are becoming usable in some situations such as in cars but in less controlled outdoor environments they are extremely susceptible to environmental and wind noise Siri and its equivalent Google Voice require a connection to the internet to function While in theory our devices may be always connected in practice this may not be the case especially when participating in more extreme mobile activities eg fell runn
6. Mobile Interaction Does Not Exist Joe Marshall Mixed Reality Lab Department of Computer Science University of Nottingham Nottingham NG8 1BB UK joe marshall nottingham ac uk Paul Tennent Mixed Reality Lab Department of Computer Science University of Nottingham Nottingham NG8 1BB UK paul tennent nottingham ac uk Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page To copy otherwise or republish to post on servers or to redistribute to lists requires prior specific permission and or a fee CHI 13 April 27 May 2 2013 Paris France Copyright 2012 ACM 978 1 XXXX XXXX X XX XX 10 00 Abstract Most mobile systems are stop to interact designed for active interaction only when a user is standing still paying visual and mental attention to the device However people are increasingly carrying and using devices while undertaking a wide range of movement activities such as walking cycling running Some existing systems such as Apple s Siri aim for hands and eyes free use but they do not consider the wider challenges of interaction during movement We describe the challenges of system design for active mobile interaction These interaction in motion challenges are discussed wit
7. ched they begin to stop attending to other things This effect whilst purely internal means that even though people are physically able to see feel or hear what they are interacting with as well as the environment in which they are moving it may not be possible for them to attend to both of these at the same time for example if someone is driving and making a phone call they may drive less safely 13 The cognitive load created by an interactive system is likely to change over time in particular in many systems there will be times when a user is actively addressing the system and other times when they are passively receiving output from the system this may well alter their cognitive load hence the way that it is standard in many current systems to only encourage active interaction when the user is stopped and able to pay full attention to the system indeed many in built car navigation systems go so far as to enforce this based on the car s movement The content of the interaction with a system may also alter the load on the user for example when making phone calls a general discussion with an interviewer is less distracting than having to perform a simple memory test 17 As well as device use affecting the movement activity in which a person is taking part the converse is also potentially true when considering what users may be capable of we need to consider how much cognitive load they are under The mental demands tha
8. cold may both be actively hostile to electronic devices As well as physical terrain as we move through the world we may experience varying access to services such as Wi Fi amp cellular networking positioning satellites and even electrical power for devices Moving through this digital terrain may mean that applications can expect access to services to change at any time and should potentially design to be aware of the seams where access alters 7 This reality is beginning to be understood in cellular networking where newer cellular radios allow instant handover between the four different types of radio signal available on most radios 2G 3G 4G wireless and Wi Fi without holding up connections in contrast to most current devices which will typically drop connection for a second or two to transition between network types Challenge 4 Other people This challenge refers to Mueller et al s Relating Body or how we relate to other people during interaction Many movement activities involve an inherent social element such as going for a run with friends entering competitive races collaborating to climb a rock face together or the general social camaraderie between cyclists who pass each other while riding on the road This social element of exercise has already been explored in exertion interfaces such as collaborative remote running systems 15 Social networking has been at the forefront of the uptake of ge
9. h reference to an extreme movement interaction situation cold water swimming Author Keywords Motion interaction interaction in motion mobile swimming ACM Classification Keywords H 5 m Information interfaces and presentation e g HCI Miscellaneous General Terms Design Introduction Current mobile device user interfaces are primarily based around users stopping and visually attending to a touchscreen at any point they wish to interact with the device Many applications present data to the user without forcing them to use a screen such as music navigation instructions and the audio from phone calls However this is typically a one way transfer with the screen being used to stop calls change how music is being played or set the navigation instructions As such these systems are designed around a stop to interact model this is even made very explicit in many systems for example the manual for the TomTom Go car navigation system states Important You should always plan your journey before you start driving It is dangerous to plan a route while driving 22 We would argue that these are not truly mobile systems devices are increasingly portable but performing any meaningful two way interaction while actively mobile is hard and falls outside the design parameters of the device There are a small number of devices and systems which allow a limited amount of two way mobile interaction Firstly
10. immers respond to these bodies often depends on the legal status of the water if it is contested or illegal they may wish to keep a low profile This may constrain device use for example people may wish to be quiet or to keep their body low in the water In other situations people may wish to be actively visible to other users for safety purposes Conclusions Designing truly mobile interactive systems which allow people to interact with the system without inhibiting movement through the world is in general challenging Our exploration of cold swimming as an interaction domain shows that such situations may in fact be extremely complex However given the potential for fulfilling and rich experiences which support and enhance people s movement activities it is imperative that we attempt to design for interaction in motion Acknowledgements This work is supported by the Leverhulme Trust ECF 2012 677 References 1 Accessible Android Setting alarms eyes free http accessibleandroid blogspot co uk 2011 06 how do i set and use alarms on android html 2 Alderman N amp Six to Start Zombies Run https www zombiesrungame com 3 Apple Siri http www apple com ios siri 4 Ballagas R et al REXplorer a mobile pervasive spell casting game for tourists Ext Abstracts of CHI 07 5 Blum L Broll W Miller S 2009 Augmented Reality Under Water SIGGRAPH 09 Posters 6 Boyd Davis S et al Ere be Dragon
11. ing or cold swimming Finally in applications which track movement such as navigation systems whilst the core of the application is based around stop to interact the application may track the response of the user to the output which it is presenting which can allow the user to actively interact with the system by altering their movements For example users of satellite navigation in cars may choose to use their local knowledge and ignoring an instruction from the device and it will then respond by recalculating the route taking account of the direction that the user chose Our previous work creating intense experiences by encouraging extreme exertion 11 took this approach to the logical extreme by creating a system entirely controlled by how the user chose to move Other examples include audio based mobile games such as 1831 Riot which used locational audio to present a story based on a historical riot 8 and the commercial audio running game Zombies Run in which zombies chase you as you go for a run 2 Why is Interaction in Motion Necessary The reason for the general uptake of the stop to interact paradigm is clear by assuming that people stop and are looking at a touchscreen when they interact with mobile devices we remove a vast number of variables from the situation this in turn allows the highly configurable touch screen interface to serve a wide range of purposes as can be seen by the vast lib
12. n in motion challenges described above Cognitive Load Swimming takes a lot of concentration even when pool training it is common for swimmers to miscount the number of lengths they are swimming Swimming outdoors requires extra attention both to navigate through the water and also to avoid or deal with dangers such as currents waves sea creatures boats and other swimmers This means that cold swimmers will be under quite heavy mental loads even without considering any load from interactive devices Temperature also has effects on mental acuity which further reduce the ability to operate interactive systems well for example in our mental arithmetic test the time taken to perform a simple 2 digit by 1 digit multiplication before the swim was 10 seconds or less after swim times were twice as long on average but went up to one moment of Water temp 6C pom i times Baseline 328ms mean After swim 373ms 392ms mean 330ms paseline 2 3mm 1 34mm mean After swim 3 imm 2 28mm mean Mental Arithmetic Baseline 9264ms 5989ms mean After swim 20509ms 46387ms mean Table 1 Summary of pilot swim study results complete confusion with 8 wrong answers being tried and taking over 3 minutes see Table 1 Physical constraints Swimming is a highly physically constrained activity the swimming action itself means that while swimming the hands arms body rotation and head movement are all quite constrained
13. neral purpose mobile devices and it is likely that many motion interaction systems will need to be aware of these social relationships As well as interactions between people who know each other or share a liking for the same activity our movement may involve interactions with people that we don t know which may sometimes be less positive For example we may be in a crowded area and have to take care to avoid colliding with people passing by or we may be in a quiet area and not want to visibly interact with a device in case a criminal sees it and attempts to steal it These interactions will often be strongly linked to what are essentially social elements of terrain with factors such as crime rate popularity of an area and lighting all having an effect on the possibility or desirability of interaction with a device Challenges of Interaction for Cold Swimming This section describes an exploration into the effects of cold swimming on interaction We show how the set of challenges above can be related to the activity of swimming in cold water and its effects on interaction In many countries around the world groups of cold swimmers enjoy outdoor swimming year round in all weathers Water temperatures considered to be cold or ice swimming typically range from 11C down to freezing and even slightly below moving water may freeze at sub zero temperatures Immersion in water cools the body extremely fast so swims at such tem
14. peratures are often necessarily short Most winter swimmers voluntarily undergo a certain degree of mild to moderate hypothermia every time they swim Cold swimmers do it both for the thrill of overcoming the challenges of cold water and also for the pleasure afterwards as the cold water pain stimulates the body s natural painkilling mechanisms once they exit the cold water they are left with a strong sense of wellbeing We set out to study cold swimming as a practical example for interactive device use primarily because it offers an extremely constrained and challenging application area Cold swimming is also poorly served by current technology with existing technology being primarily recording tools swim or divers watches GPS trackers cameras which record metrics such as location and speed time water temperature stroke rate etc As we show below each of the challenges we described above are strongly relevant to outdoor swimming in cold water Swimming in cold water is essentially a playful activity done for the experience of the necessarily brief immersion if participants wish to do swimming for exercise they would usually be better served by a longer pool swim As such it is a natural fit for the introduction of playful and gaming activities Swimming has already been combined with interactive entertainment successfully in a pool environment 5 although due to the enforced high energy nature of a cold water swim we s
15. raries of applications available for most modern mobile device operating systems There are several reasons why designers might wish to go beyond stop to interact and to design for active interaction whilst a user is moving these include e Movement and exercise activities such as walking running cycling swimming can be intense exciting experiences 11 As such they are a positive thing in themselves integrating physical movement into activities such as games can lead to enhanced feelings of social connection amongst players 14 and has the potential to have physical and mental health benefits for users e Mobile devices are increasingly worn at all times with many people carrying them while they take part in everyday movement activities such as commuting by bike or exercising for fun Designing systems and applications that can be used during these moments could potentially both support people in their typically optional movement activities enable the applications to be useful to them during their movement and minimize interruption by applications during movement e Whilst most current design assumes stop to interact in reality people do engage in movement activities whilst interacting As designers we should consider the reality of peoples use of our designs The most obvious is the use of mobile phones in cars which can potentially lead to accidents when people are not paying full attention to the road 13 As
16. require a small mirror to align the view forwards potentially on a newer device the camera angle could be altered or a specific camera could be fitted This would allow people to safely and politely walk whilst texting This works well for walking although it is clearly not a one size fits all text input in motion solution once someone runs or cycles they no longer have their hands free to interact How to do Interaction in Motion The recent history of mobile device interaction design has very much been a journey from special purpose devices e g GPS navigators heart rate monitors etc to small general purpose computers which are extremely portable Designing for interaction in motion may in some ways conflict with this In particular when designing for interaction in motion we have to consider two things what do we mean by interaction and what do we mean by motion When considering the range of interactions we are designing for there is clearly a tradeoff between designing for general purpose interaction and designing simpler interfaces for a more specialized purpose With existing mobile technology the movement applications referenced in the introduction have primarily been focused on a single task such as music playback or navigation It is clearly harder to design a successful interaction method for general purpose interaction during movement as evidenced by the comparison in 19 between their PocketMenu for mu
17. s Heart and Health in Pervasive 2006 157 164 7 Broll G Benford S Seamful design for location based mobile games in ICEC 05 4th international conference on Entertainment Computing 8 Cater K Fleuriot C Hull R amp Reid J Location aware interactive applications in SIGGRAPH 2005 9 CGActive Type and Walk http www type n walk com 10 Google glass http plus google com projectglass 11 Marshall J amp Benford S Using fast interaction to create intense experiences in CHI 2011 1255 1264 12 McCullough L amp Arora M 2004 Diagnosis and Treatment of Hypothermia American Family Physician 70 12 2325 2332 13 McKnight A J McKnight A S The effect of cellular phone use upon driver attention Accident Analysis and Prevention 25 3 1993 pp 259 265 14 Mueller F Agamanolis S amp Picard R Exertion Interfaces Sports over a Distance for Social Bonding and Fun in CHI 2003 15 Mueller F O Brien S amp Thorogood A Jogging over a distance supporting a jogging together experience although being apart Ext Abstracts of CHI 2007 1989 1994 16 Mueller F et al Designing Sports A Framework for Exertion Games in CHI 2011 2651 2660 17 Nunes L Recarte M Cognitive demands of hands free phone conversation while driving Transportation Research 5 2 2002 133 144 18 Paulo N Skydiving camera trigger switch http www ultimate switch com home 19 Pielot M e
18. sic playback and the use of Apple VoiceOver While Siri and Google Voice may go some way to mitigate this we have seen that they are restricted by environment and require a constant network connection As well as application we have to consider the range of movement activities which we are expecting from users for example designing an interface for use while running may be extremely different to designing for use while swimming or cycling Again it may be harder to design for a wide range of activities rather than focus on a smaller range Interaction in motion presents many new challenges for interaction design the following describes some of these challenges organized into 4 categories which are very loosely based on those used in Mueller et al s framework for designing exertion games 16 We then illustrate these challenges with reference to a particular area of interaction design which we are currently embarking on development in the design of interactive devices for use while cold water swimming This example is chosen to illustrate an extreme case of a movement interaction situation and the consequent challenges of designing interaction in that area Challenge 1 Cognitive Load Work on interfaces use whilst performing safety critical tasks such as driving has explored the concept of cognitive load essentially that a person is only able to pay attention to a certain amount of things at once and that once this limit is rea
19. t al PocketMenu non visual menus for touch screen devices in MobileHCI 2012 327 330 20 Romet T T 1988 Mechanism of afterdrop after cold water Immersion Journal of Applied Physiology 65 4 1535 1538 21 Tactus touchscreen with reshaping technology www tactustechnology com release_120605 html 22 Tom Tom Tom Tom Go User Manual http download tomtom com open manuals device ref man TomTom GO en US pdf
20. t they are placed under by the movement activity may make them unable to successfully control an interactive device or likely to make mistakes The cognitive load of a movement activity may also change over time for example as someone runs up some steps or crosses a road or passes through a crowded space they may be required to pay more attention to their running in order to avoid tripping over or collision whereas on a flat section of path with nobody else around running may only require a small amount of concentration leaving them free to interact with a mobile device Challenge 2 Physical Constraints Mueller et al describe the moving body how body parts are positioned relative to each other Both movement activities and use of interactive systems may place constraints on body position which may well conflict with each other For example as identified in 19 a major barrier to the use of many modern mobile systems whilst moving is the need to move the body and eyes in order to pay visual attention to touchscreen interfaces touchscreens allow for extremely generic devices which can run a wide range of applications but lose the tactile feedback of the physical controls on older more specialized devices Several companies are trying to combat this by designing screens which aim to allow touchscreen interfaces to be felt for example by dynamically reshaping the screen to create physically raised onscreen buttons
21. und a person affects how they experience a game When we are combining movement and interaction we describe this as terrain In previous work on running 11 we saw that when running and interacting with a system the terrain over which a person was running appeared to make a big difference to how they experienced the interaction and how able they were to concentrate on the output of the interactive system Physical terrain may appear to be relatively fixed as in hills rocky tracks tarmac roads In practice it may also be changeable on a short or long term basis for example light levels levels of muddiness on off road tracks amounts of rain water levels and currents and iciness can all vary due to weather factors Similarly terrain is subject to short term change through the appearance of traffic and other dynamic obstacles Figure 2 Terrain can alter our ability to interact Physical terrain can require more or less mental and physical effort to traverse For example in Figure 2 the runner was foolish to attempt interaction with his phone at this point as he both needed to pay more mental attention to the complex terrain he was climbing over and also ideally required hands that were free to move in order to balance on the slippery and rocky terrain As well as affecting our movement and how much of a challenge movement is the physical environment we move in may affect interactive devices for example water or extreme
22. uspect any cold swimming game will be more similar to running games such as the popular Zombies Run series 2 A key element of cold swimming is the temperature Symptoms of hypothermia include a lack of muscle coordination slowness confusion and forgetfulness 12 all of which affect interaction As we have yet to deploy any final cold swimming systems it is currently impractical to measure exactly how this cold affects interaction directly However one element of cold exposure is the phenomenon known as the after drop 20 When a person leaves the water their core body temperature continues to drop for some time meaning that symptoms of hypothermia are as bad or worse in the first 15 minutes out of the water as during immersion This well known safety risk for swimmers is also an opportunity for our early study of this area as we can measure some effects of the cold exposure on interaction at the moment of getting out using a stop to interact application without having to build an interaction in motion application to do so In our pilot we performed tests on one experienced cold swimmer using a simple mobile phone application testing reaction time and mental arithmetic skills tested each 5 times before then repeatedly in the 15 minutes after a swim In the following section we use this data along with experience as cold swimmers to describe the challenges of swimming as they relate to the core interactio
Download Pdf Manuals
Related Search