a UI architecture prioritising HCI concerns for interactive
Contents
1. chi med making medical devices safer EPSRC Programme Grant EP G059063 1 Public Paper no 49 Buffer Automata a UI Architecture Prioritising HCI Concerns for Interactive Devices Harold Thimbleby Andy Gimblett amp Abigail Cauchi Proceedings of the 3rd ACM SIGCHI symposium on engineering interactive computer systems EICS 11 73 78 PP release date 10 October 201 1 file WP049 pdf ae p Se Swansea University Prifysgol Abertawe A J E Oy Queen Mary kod CITY UNIVERSITY EPS RC ch Cor ring and Physical Sciences uncil Buffer Automata A UI Architecture Prioritising HCI Concerns for Interactive Devices Harold Thimbleby FIT Lab Swansea University ABSTRACT We introduce an architectural software formalism buffer au tomata for the specification implementation and analysis of a particular class of discrete interactive systems and devices The approach defines a layer between the physical user inter face and the application if any and provides a clear frame work for highlighting a number of interaction design issues in particular around modes and undo Author Keywords Buffer automata modes undo interaction programming structural usability ACM Classification Keywords H 5 2 D 2 2 H 1 2 1 36 User Interfaces Theory and meth ods General Terms Design Theory 1 INTRODUCTION General purpose programming languages can implement any interactive system but such systems are not easy to2. s input function which modifies the buffer s history in response to the buffer re ceiving input events We argue that there are a small number of off the shelf components which can be combined to form all reason able input functions For example append to the history delete the history s last element reset by clearing the his tory forget by replacing the initial value with the result of A and then performing a reset Additionally functions like number date name currencyValue etc provide con sistent ways of handling application values e Hc C is the buffer s access function mapping the buffer s history to some value in C such as a floating point number This might perform a fold over the history or perhaps access its last value etc again a small number of standard functionalities could conceivably cover most real buffers e C Ho is the buffer s update function through which the application in which the buffer is embedded may modify its history and thus its value see Section 2 6 FSMs can easily be modelled using this mechanism but many other structures are possible Example Numerical buffer with direct manipulation Our radio s volume and tuning buffers might be represented as follows let C n E N 0 lt n lt 99 I C Then Vx z E C Vy FinSeq C d z y 2 4 y That is the initial value is simply overwritten with new in puts and the histo
3. actions Opt1 etc correspond to soft buttons a model using displayed names e g VTBI would have a different possibly more meaningful mode structure We recently recast this model as a buffer automaton with 10 buffers 3 are for numbers in the range 0 999 6 are inter related mode relevant switches on off running not running infusing not infusing VTBI Mode on off clear mode on off and alarm silent beeping or muted This multiplies up to 2 x 3 96 but for instance when the GP is off it cannot be infusing or alarming so in fact only 22 combinations are possible which may be further reduced to just 6 modes with distinct input behaviour mode space discovery finds exactly these modes see Figure 4 5 RELATED WORK A large number of formalisms take an automata based ap proach to modelling systems though most are more gen eral than buffer automata and do not emphasise interaction and HCI concerns as we aim to A key example is Stat echarts 8 9 see also the closely related UML state dia grams an automata based visual formalism with a variety of features enabling the representation of many complex sys tems using structured diagrams with nesting composition abstraction guarded broadcast messaging and history all of which yield a rich semantics In contrast buffer automata try to be a simple formalism that bridges certain user and de vice perspectives as cleanly as possible by refusing to rep resent so
4. analyse for their interaction properties Conversely finite state ma chines can in principle also describe any interactive system typically as enormous FSMs but for non trivial systems they are impractically large and have no clear structure to support insights into human computer interaction issues In this paper we introduce an extension to FSMs which aims to address this state explosion issue at least for some systems Consider something as simple as a handheld calculator it has a few basic modes off adding subtracting but track ing its states means also tracking say 108 possible numbers for its display 108 for its memory 108 for the working num ber and perhaps 10 for any constant when pressing just adds the current number to the constant Its full state space thus has at least 10 states this simple device is at the lim its of what can conveniently be handled by current model Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page To copy otherwise or republish to post on servers or to redistribute to lists requires prior specific permission and or a fee EICS 11 June 1316 2011 Pisa Italy Copyright 2011 ACM 978 1 4503 0670 6 1 1 06 10 00 Andy Gimblett FIT Lab Swansea U
5. automaton we have a space of four modes off MW off FM on MW on FM and some independent buffers with simple well un derstood behaviour e g for the volume level Events are cleanly distributed to the various buffers according to mode when off everything but is ignored say and there is no interaction between the buffers in this highly orthogonal device Variants are possible and explored later in the paper Tasks on many systems require a pair of activities set up one or more buffers such as the rate of infusion and make the system do the action with a buffer value such as perform an infusion Such systems are well suited to modelling as buffer automata Of course any sufficiently complex com ponent could model or even implement an entire system nothing needs to be partitioned into independent modes and buffers Our proposition is that buffer automata can pro vide clear descriptions of interactive systems of a particular class amenable to particular kinds of analysis the utility of the formalism is its exposure of certain HCI issues such as modes and undo and as an attempt to model systems closely in terms of how users think of them 2 1 Buffers and modes As described above a buffer automaton is an FSM like mode space accompanied with a set of buffers whose structure we define shortly In fact while it is possible and may be use ful to model the mode space explicitly we see modes rather as an emergent implicit
6. dis play and 7 LEDs see Figure 2 There are 69 parameters that control the sound being produced by the instrument e g Volume Arpeggiator Tempo where such a collection of parameters is called a patch the Pulse has a memory of 99 patch slots Patch parameters are edited via the matrix ar rangement on the right of the panel the 69 parameters are arranged in a 6X6 matrix with some doubling up see be low and each of the 6 knobs is dedicated to one column of this matrix Only one row of the matrix is active at once as indicated by a red LED to its left and advanced by press ing the large oval button Finally the blue button under toggles between parameters where they are dou bled up the red LED flashes when shifted For example if the second LED down is flashing and we rotate the third knob from the left we modify the OSC2 Keytrack what ever that means parameter We have simulated the Pulse s patch editing interface using a buffer automaton in Haskell connected to a web GUI in JavaScript HTMLS5 Canvas Every patch parameter has an associated buffer and each buffer responds to the events n and for some column number 1 lt n lt 6 There are also simple buffers for four values associated with modes see below We only modelled a single patch and have not considered the Pulse s memory and related functionality MOD SOURCES DESTINATIONS osci 0SC3 Semit
7. the following contributions we introduce and define buffer automata a new formalism for structuring and thinking about user interfaces we provide abstract and concrete examples of buffer automata in use we show that buffer automata provide a new perspective on several issues of interaction design and programming including modes and undo and in particular we describe how to automatically compute modes within the formalism It is our hope that this paper will direct and stimulate further discussion and exploration of some of the issues raised We present a review of potentially alternative and contrast ing approaches in Section 5 for a short paper the back ground literature is perhaps best presented after we first em phasise and examine the focus of our contribution 2 BUFFER AUTOMATA As discussed in Section 1 many interactive systems may be thought of as a relatively simple abstract mode space to gether with various data values which the user may alter Thinking of the mode space as a classic FSM or automa ton and the data values as being held in buffers where they are manipulated then leads to the idea of a buffer automaton Buffers are data values the system keeps track of such as phone number radio station CD track time now alarm time drug infusion rate Modes are significant states of the sys tem such as on off playing infusing not infusing standby Among other things modes dictate which buffers are re
8. whether shift is engaged The mode relevant buffers on the Pulse are OnOff Row Shift and Mod Select on row 4 knob 3 controls which sub row knobs 4 5 6 are focussed on all of which can be modelled as small FSMs We used mode space discovery to compute the Pulse BA s mode space see Figure 3 which is seen to be quite regular Our Pulse s focus function is hand written we plan to de vise a domain specific language DSL for describing focus from which the mode space may be computed directly via abstract interpretation rather than dynamically by discov ery the dynamic version may however remain useful as a correctness check for comparison against the statically com puted version 4 EXAMPLE ALARIS INFUSION PUMP The Cardinal Health Alaris GP volumetric infusion pump 2 is an interactive medical device with 14 buttons designed to provide patients with controlled delivery of drugs We mod elled the device thoroughly using an interactive Mathemat ica program with a realistic graphical animation that allows user testing to confirm the program is an accurate interac tion simulation The simulation has a transition graph model with 4 8 x 10 states such a model is unwieldy and com putationally costly to analyse TT N poy Figure 4 Alaris infusion pump mode space computed from the focus function of our BA model Mode names are implicit and have been chosen to reflect each mode s role Some
9. BA s buffers tells us for each input the names of buffers that should receive that input From we derive our formal notion of mode see below e A is the BA s derived input function A S gt gt S which distributes inputs to the BA s various buffers in ac cordance with A The definition is omitted here but is simple an input is distributed to every buffer which is currently in focus for that input e V is the BA s visibility function V S gt P L and de termines the names of the visible buffers and thus what is visible to the user may be computed from their values As discussed in Section 2 1 we see mode as a context de termining the interpretation of user inputs assuming rea sonably for any sane device that a given buffer s interpre tation of its input is always consistent mode is thus defined solely by the distribution of inputs to buffers which we can compute trivially given the focus function F The current mode of a BA is essentially a lookup table telling us for each buffer identified by name which inputs it can receive right now Thus we have mode S gt L gt P X 2 3 1 Radio example Let our radio have the following four buffers e OnOff an FSM with states on off actions On Off e Band an FSM with states mw fm actions MW FM e Volume with C 0 99 I v vy volume values modified with up down actions under some sensi ble interpretation of the buffe
10. ally starts an infusion before they intend to and they notice the error they wouldn t expect to hit to fix the problem they d hit Stop triggering another mode change and that would just be the first step in fixing the error prob ably couldn t restore the device to its previous state due to the physical effect of starting the infusion Conversely if the nurse makes a keying error while entering the desired in fusion rate is a perfectly reasonable approach This points to another distinction between mode relevant and non mode relevant buffers the latter can in general use their his tories to support undo whereas the former need not and of ten should not in some cases a mode change represents an action which can t easily be undone that is useful informa tion for designers This is a rather sweeping generalisation of course and there will be exceptions for example in many simple buffers such as in our radio all events have inverses so an explicit shared between buffers is pointless 2 6 Buffer automata in context Conceptually we see buffer automata as an interpretive and structure imparting layer between the UI presentation of an interactive device and its underlying application after each buffer in focus performs its transition control is passed to the application to perform access then update operations The BA is then ready for its next input event How exactly this is structured is left generic in order to a
11. ccommodate a wide range of approaches A typical setting is illustrated in Figure User input is received from the UI and passed to the BA as members of X where it is distributed to various buffers input functions as dictated by F the application 76 User level A feedback user actions A Application level P Figure 1 Buffer automata in context User actions translate to BA inputs in X distributed by 7 to each buffer s 6 input function the application reads buffer values via As and updates via s if necessary values for feedback are projected by V filtered according to V then has the opportunity to read and update buffer values via each buffer s A and functions as well as dealing with other events such as non UI I O alarms etc buffer values are then exposed subject to VY as views in the UI using some function V Hc Y from the buffer s state to the view domain V How to handle the display of buffer contents and the role of the function V the converse of A and its relationship with modes remains future work to be explored we note that visibility might in fact involve non graphical elements such as sound or vibration 3 EXAMPLE PULSE MUSIC SYNTHESIZER The Waldorf Pulse is a 1990s era electronic musical instru ment controlled remotely for performance via the MIDI pro tocol but which may also be programmed via a front panel interface consisting of 6 knobs 4 buttons a 3 character
12. ceiving user input and which are exposed as output Thus user actions give rise to input events which are distributed to buffers according to the current mode modifying buffer val ues and potentially triggering mode changes This mecha nism and the exact relationship between buffers and modes is explored in Section 2 1 In principle a user is generally aware of the current mode s of the system and the available actions to change modes conversely users do not track exact buffer values but may know in principle whether they are right wrong nearly right etc Users rely on systems to keep track of buffer de tails unless actually interacting with a particular buffer For example consider a nurse on a hospital ward with patients connected to drug infusion pumps asked is this patient get ting an infusion the nurse knows the answer is yes be cause earlier they put the device into the infusing mode when asked how fast is the patient being infused the nurse will probably check the device Here infusing is a mode but infusion rate is a buffer Another example a radio can be on or off and can be tuned in to FM or MW bands it has 100 possible volume levels and 100 possible tuning frequencies it responds to events ON OFF MW FM VoLUP VOLDN TUNUP TUNDN Mod elling this as a 40 000 state FSM 2 x 2 x 100 x 100 cer tainly obscures its structure modelled as a buffer
13. e may be discovered using model discovery 7 to dynamically explore the state space of all mode relevant buffers producing a graph whose nodes are BA state focus pairs the mode space is then obtained by collapsing together states with identical focus produc ing a graph whose nodes are distinct focus values modes and whose edges are events which change the mode Note that the set of mode relevant buffers must be specified ex plicitly otherwise a full exploration of the BA s complete state space would be required just to learn which buffers are mode relevant Alternatively given a suitable representation of F the mode space can in principle be computed statically However as the application may in general modify any buffer freely see Section 2 6 the statically computed mode space for a given system may be inaccurate and dynamic discovery is proba bly safer at the least comparing static and dynamic mode spaces tells us something about the BA and application s in teraction with each other 2 5 Undo and history Common wisdom is that user interfaces should support undo 12 but actually it is more important that they support the user achieving their goals in the face of error Undo is just one way of recovering from recognised error but often it is not the best strategy In particular we argue undo is of ten not the right strategy or not even meaningful when dealing with mode changing events If a nurse accident
14. ene Tune Shape PW inte Sync Keyteock 0x1 WOI Speed Shope LFO Speed Delay Osc 2 Ose 3 Moise MIX Anok Decoy Sustain Relesse Keyteok Trigger ENV1 2 Pitch Mod Sowrce Poriameato Mode Select Source Amour Destinetion MOD Active Ronge Tempo Clock Mede ELI Pichbend Scale GLB Cet Keytrodk Env Sens Valo Sens Resonance ILOA Yom Velo Sens Ponning LZ Cutoff Mod Source Figure 2 Screenshot from running simulation of Waldorf Pulse control panel Patch control knobs at bottom right Images reproduced from Waldorf Pulse user manual with permission row O row shift row row TON shift shift up gd0 wn wn shift A p On Shift oh ale 1p 3 QP on Hf off shit shift Shift row Ndi row lt P A h i O n aon Onw Senit shift shift on oa own own shiftrow Off off off op P3 p OpY onl ofon row OOW ow MMW ow Shito O row shift shift tow O shift on row shift ow Figure 3 Mode space of Pulse buffer automaton row and shift form a regular structure of two concentric hexagons with off in the centre Mod Select introduces six more modes to the right of the hexagons Note that this graph is nondeterministic multiple row edges at top right Turning a knob should obviously not modify all 6 or more parameters in that knob s column The Pulse s interface is thus very modeful how turning a given knob is interpreted depends at least on which row of the matrix is active and in some cases
15. es which offer undo and check our hypotheses there Currently buffer automata have no hierarchical structure thus we can t build complex UI elements from simpler ones as buffer automata or realis tically re use BAs between contexts One interesting possi ble line of enquiry here involves positioning BAs as Control components within PAC 5 triads Similarly in our current arrangement buffers cannot modify each others contents or 78 state except via the analytically opaque application layer a working theory of hierarchical buffer automata could possi bly overcome this though we also argue that this is often in fact a desirable feature Timeouts and other external events could warrant similar treatment As mentioned in Section 3 we plan for a DSL describing focus and allowing mode space to be computed statically rather than discovered dy namically Finally visibility output of buffer contents re mains a relatively unexplored issue but clearly an important one Source code for our BA simulations is available from the authors 6 1 Acknowledgements We are grateful to Michael Harrison and our reviewers for many helpful comments we were funded by EPSRC Grants EP F020031 1 and EP G059063 1 7 REFERENCES 1 J Campos and M Harrison Interaction engineering using the IVY tool In Proceedings of the Ist ACM SIGCHI symposium on Engineering interactive computing systems 2009 2 Cardinal Health Inc Alaris GP volume
16. feature arising from how inputs are 74 distributed to buffers thus the mode space is to be com puted not specified directly Let us explore this subtlety Consider the radio example above There are on off and MW FM modes combining orthogonally to form a mode space with 4 states it is precisely the states of this space that we refer to when we say mode On off and MW FM are not themselves modes like volume and tuning they are buffers data values the user can manipulate but they are special buffers only in that they introduce mode behaviour they influence how input events are interpreted This inter pretation of the word mode is well founded according to 6 a mode is a context that changes the interpretation of commands similarly 11 says for any given gesture the interface is in a particular mode if the interpretation of that gesture is constant Our notion of focus formally captures this concept see Section 2 3 So on off and MW FM are just buffers they happen to have very simple structure and we can easily model them as 2 state FSMs but in general this need not be the case buffers can have rich structure and even rich buffers may influence mode Suppose the radio also has a button another 2 state buffer which is ignored if the volume is 0O then the volume buffer modelled as a 100 state FSM perhaps cor responding to a physical slider or knob also contributes to the radio s
17. from mode changing interac tions we have described how modes emerge from the inter action of buffers via the notion of focus and thus how to compute a BA s mode space The formalism introduces a new perspective on the role of undo and when it is appropri ate or not Imagined and real world examples demonstrate that BAs are a viable tool for modelling devices of a certain kind in a straightforward and comprehensible manner Designers make decisions about UI behaviour and a BA makes some important decisions explicit In a conventional program the UI s structure doesn t relate directly to the user experience there may be an event loop and UI code sensible to a programmer but improving the code doesn t necessarily improve the UI or engage with HCI concerns In contrast a BA has explicit structures such as the 0 function which are supposed to be simple We would argue that a program mer making some 0 code simpler improves the user interface design conversely programming a Turing Machine inside a 0 which is possible defeats the object of clarity for the user By extension mode relevant buffers should probably be small and FSM like to keep the mode space comprehen sible Future work on buffer automata must explore a number of open issues The relationship we describe between undo modes and buffers is convincing but none of the examples in this paper use undo they are based on real devices but we should model some other devic
18. me controversial features Very similar comments might be made for very many related formalisms Petri nets CSP ATNs each provides a well defined generalisation that makes the formalism appropriate for some domain pro cess control say but none specifically address human fac tors or HCI concerns as is the intention for buffer automata as introduced here Modechart 10 for example is a spec ification language for real time systems built upon a real time logic specifically RTL Like BAs modecharts ex pose mode explicitly and aim to aid reasoning about mode rich systems Modechart is quite low level however and strongly concerned with timing issues Model checking 4 is a well established technique for au tomatically verifying a state space model against a logical specification We have recently seen how this can be applied to interactive systems 1 but this general approach still suf fers from the state explosion problem The BA approach to separating buffers from modes may be compared with the technique of data abstraction in model checking Composi tional model checking 3 tries to overcome state explosion by breaking up the model into smaller components simi larly a working theory of hierarchy for buffer automata see Section 6 would be useful 6 CONCLUSION In this paper we have introduced buffer automata a formal ism for describing interactive systems which explicitly sep arates data entry interactions
19. mode Every buffer we ve seen so far is easy to model as an FSM but this need not be so a text entry box is a good counterexample it can be modelled as an FSM but not naturally so our definition of buffer below is necessar ily quite general rich buffers such as text entry boxes can still influence the system s mode but perhaps with a greater risk of confusion for the user 2 2 Definition Buffers A buffer is essentially an object for some data along with several functions for manipulating and accessing that object In order to support undo see Section 2 5 we structure the object as a history rather than a single value formed from an initial value and a sequence of modifications perhaps but not necessarily corresponding to input events it is easy to ignore this history if desired as we shall show by example Let FinSeq X be the set of finite sequences of elements of X Given two sets X and Y the set of histories over X and Y Hx y is the set X x FinSeq Y That is a history has an initial value in X followed by a sequence of values in Y Notionally the Y values modify the initial X value in some sense although various interpretations are possible including where X Y and modify means replace A buffer is a tuple H 6 A with e H Hoc is the buffer s history where C is the buffer s contents alphabet and I is the buffer s input alphabet e Hcr x I Ho is the buffer
20. niversity h thimbleby swansea ac uk a m gimblett swansea ac uk 73 Abigail Cauchi FIT Lab Swansea University csabi swansea ac uk checking technology e g by exploiting its symmetries but human users obviously do not conceptualise a state space of this size Rather we suggest users do think about interac tive systems a bit like FSMs but only in terms of their mode space and the exact number values handled by the calcula tor are abstracted away in the user s mental model indeed such abstraction is much of the purpose of the device We propose that many simple but non trivial interactive systems can be thought of like this that is as a manageable space of a few explicit modes combined with a collection of abstracted data values and that this captures the struc ture users are aware of and can manage and abstracts away the structure users ignore details of There remain many complex interactive systems beyond the reach of such an approach at least as proposed here but see section 6 we are particularly concerned at this time with the many de vices from calculators to medical infusion pumps that we can handle clearly and rigorously Such simple systems still have non trivial interaction issues and the approach pro posed here exposes and makes explicit some of these issues and even where it does not resolve them it provides a frame work to support clearer arguments about design choices This paper makes
21. r s history e g perhaps with wraparound depending on the physical mechanism used e Tuning with C 0 99 I ty t similarly Sensible initial BA values might be off mw 0 50 say is On Off MW FM v v t t F is simple all events go their respective buffers unless OnOff s value is off when On goes to OnOff and all other events are ignored Unlike our earlier conception of the radio this device actu ally has only 2 modes on and off Band has no influence on interpretation of inputs in effect we have modelled an old style tuner with a single tuning control for both bands Instead let s replace Tuning with separate tuning values for MW FM better reflecting the operation of a modern radio MWTune with C 0 99 I t t FMTune with C 0 99 2 h t Note that MWTune and FMTune have identical input alpha bets we modify F so t t events are delivered to the appropriate buffer according to the value of Band which is thus now mode relevant We still don t have 4 modes how ever only 3 off on MW and on FM Perhaps that s fine or perhaps it s possible to turn the volume and tuning knobs while the device is off a variety of approaches are possi ble Even with this simple example we see that thinking concretely about focus and modes brings to light some inter esting design questions we might otherwise have ignored 2 4 Mode space discovery A buffer automaton s mode spac
22. ry is never used whatever number is se lected is entered directly overwriting any previous value Example String buffer with clear and delete Let C FinSeq a b the set of finite sequences of a s and b s I a b clear delete Let Hor x I gt Woz be Vx z E C Vy E FinSeq a b x y a d a y b x y clear x delete y z delete 8 eS N Noe yev AVG x y 6 5 x x y Where is concatenation That is a and b are appended to the history sequence whereas clear and delete modify that sequence Thus the buffer s history consists of some initial value in C the empty string say followed by a finite sequence of further values in C Then let A C x I gt C be the concatenation of the buffer s history sequence onto its history initial value This version of the buffer has no maximum length 2 3 Definitions Buffer automata focus and modes A buffer automaton is a tuple S L L so amp F A V with e Sis a tuple of buffers Labels L serve as unique names of buffers and the naming is a bijection from labels to buffers The state of the BA is the state of its buffers so is the BA s initial state e is the BA s input alphabet the union of each of its buffers input alphabets which need not be disjoint 19 e F S gt X P L is the BA s focus function which given the current state of the
23. tric pump directions for use Technical report Cardinal Health 1180 Rolle Switzerland 2006 3 E Clarke D Long and K McMillan Compositional model checking In Proceedings of the 4th Annual Symposium on Logic in computer science 1989 4 E M Clarke O Grumberg and D A Peled Model Checking MIT Press 1999 5 J Coutaz PAC an object oriented model for dialog design INTERACT S7 Proceedings of the 2nd IFIP Conference on Human Computer Interaction 1987 6 A Dix J Finlay G D Abowd and R Beale Human Computer Interaction Pearson 3rd edition 2004 7 A Gimblett and H Thimbleby User interface model discovery towards a generic approach In EICS 10 Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems 2010 8 D Harel Statecharts a visual formalism for complex systems Science of Computer Programming 8 3 1987 9 D Harel and A Naamad The statemate semantics of statecharts ACM Trans Softw Eng Methodol 5 1996 F Jahanian and A Mok Modechart A specification language for real time systems IEEE Transactions on Software Engineering 20 1994 10 11 J Raskin The Humane Interface Addison Wesley 2000 12 B Shneiderman and C Plaisant Designing the User Interface Strategies for Effective Human Computer Interaction Pearson 5th edition 2009
Download Pdf Manuals
Related Search