Home

C5 5.131 Relia s ability summ y and mary r d usab repor bility rt

image

Contents

1. 21 4 4 8 Test Case 7 Changing the homotypic synonym created in Test Case 6 into an accepted name 23 4 4 9 Test Case 8 Changing the accepted name created in Test Case 7 back to a synonym Of the Same spectesi Mame sii vs Hidi secede chante a bec eae Ades sen techeitan a E E ES 24 4 4 10 Test Case 9 Checking and editing if necessary the parsed Rank of the Genus taxon 25 4 4 11 Test Case 10 Checking and editing if necessary the parsed uninomial and specific epithets and authorship of the 5 Species taxa c ccccscssecsesscescssecsseeecssecseeeseeseescsecseeecssecseeneenessaeeas 26 4 4 12 Test Case 11 Deleting a species taxOn isccccseivessectissesedtecssdsssesstsetsnectadiaestecsatebenes 27 4 4 13 Test Case 12 Inputting the date published and title of the nomenclatural reference fot A SPECIES CARON swcsssesssoesstevesesvaassttesscdseresdivceivhaceasendiseetntedscasnd mecavsbeveeengudiaattoatesctbedssaatehstaaiteaneaaeg 28 4 4 14 Test Case 13 Inputting a nomenclatural status of Valid for a species taxon 29 4 4 15 Test Case 14 For a species taxon input a Common Name in the Descriptive Panel 31 4 5 ICCA DAC Ks seis ces egensscasnteved iepvacesetnnssns sievecesieedseatGeestioganbat ceittvacpaedens ie vadvesdceu stenyonntdevaciantaesnd tee 33 4 5 1 Section A Satisfaction Questionnaire s sessseseseeesresressrrssrrssrnssenssrnsressressrensnrnsreee 33 4 5 2 Section Br Reaction Catds sccctcsiesscvistesssccccaseusstessveneecase
2. The testers who selected the word Useful agreed that it would make taxonomic work easier and more ordered Several also expressed a wish to use the TE for their next project Testers also selected other positive words such as Relevant 3 Desirable 1 Cutting Edge 1 and Innovative 1 This indicates that the TE will fill a need for some taxonomists who indicate the willingness to move from producing their taxonomic revisions using a word processor Testers chose the word Inconsistent for a variety of reasons confusion over left and right clicking menus options displayed in some parts and not others and functions not working in the way one would expect them to These are issues that have been detailed previously and for which suggestions have been given in the relevant test cases In total testers chose positive words or statements 48 times and negative words or statements 42 times The full list of reaction cards number of testers who chose each card and the reasons they gave for their choice are given in Appendix 1 5 4 5 3 Section C Working Practices Questionnaire There were four questions in the final section of the feedback document These are given below with a summarised version of the responses The full responses of each tester are given in Appendix 1 6 Questions 1 and 2 were designed to detect important parts of the taxonomic revision process that were either not covered adequately in th
3. The Descriptive tab could then be removed completely leaving a simpler and more user friendly interface with more easily accessible data fields Alternatively ways of improving the current Descriptive interface are given below Create an Add new description field in the Descriptive tab in the same way that an Add new taxon field has been suggested for the Taxonomic Tree panel to provide prompts to show the users what they should do next Once a description tree has been created create an Add new descriptive element field which the user can click on and is then asked what descriptive element they would like to add i e they are asked to choose from the same menu that appears when using the right click gt New menu Arrange the languages in alphabetical order Additionally allow testers to type a whole name into the text field rather than just the first letter which is currently possible Create a hover over pop up box or information button in or near the language drop down box which tells the user they can customise the language drop down menu using the Preferences menu Expand the elements of the description tree after the user has successfully entered data so the results of their input are immediately visible 32 EDIT C5 131 Reliability and usability testing report Make it more obvious that the Add button relates to the Citations field for example increase the gap between the Common Name fiel
4. expand the menus in the Properties panel in that they were only visible when pointing the mouse arrow directly at it Tester was using Windows 7 4 4 11 Test Case 10 Checking and editing if necessary the parsed uninomial and specific epithets and authorship of the 5 Species taxa Out a possible 123 there were 10 deviations 8 during test case 10 The deviations were distributed fairly evenly across all 7 steps of the test case No testers consulted the manual but 8 required assistance 6 at 4 steps Thirteen testers found the test case easy and 6 did not however only 3 were not confident that they d performed the task as expected Only one stated that they would not be confident performing the test case again Following Test Case 9 all testers knew to consult the Properties panel when checking the parsed uninomial and specific epithets of the 5 species taxa However 2 testers still had difficulty finding the required fields in the Name section Testers again complained again of having to expand the section in the Properties panel for each taxon that they edited One tester pointed out that the process would be quicker if the Rank field was at the top of the list and easier to find Some testers expressed confusion over the following terms in the Properties panel uninomial editable cache and ex authorship One tester felt that the details in the Properties panel are too technical The results
5. file and can be opened closed via the File menu Consideration should be given to how this can be changed so that the users are unaware that the each taxon is a separate file For example not allowing multiple tabs to be opened within the Taxonomic Tree panel and having a single Save option rather than the additional Save All option As most testers are well experienced in using Microsoft Word trying to recreate more closely the Word interface in this way would be a good way of improving usability for all users Another way of removing the uncertainty caused by the term file and what actually is the file is to remove the term altogether For example as it is better that users are unaware of the underlying organisation of the database renaming the File menu with something that more clearly relates to the tree such as Classification to be consistent with later versions of the TE 15 EDIT C5 131 Reliability and usability testing report Have tick marks against the entries under the Show View menu that are currently open to make the function of the menu mote clear to the users i e that they can in effect turn the panels on and off this is good standard interface design specification One tester suggested that when a user clicks the File gt Exit option a message appears warning that the TE rather than an individual file will be closed The user could be given the option of do not show t
6. other panel EDIT C5 131 Reliability and usability testing report Rearrange the main toolbar menus so that the commands within can be found more intuitively from File Edit Bulk Editor Window Help to File Edit Login View Preferences Help Simplify the process of creating synonyms and allow for the creation of the two types of synonyms using the same mechanism In relation to this review the terminology of synonyms when using the ICBN compared to the ICZN The zoologists involved did not recognise the terms homotypic and heterotypic synonyms and expect to use the terms objective and subjective synonyms respectively This should be researched further by consulting zoologists and amending the terminology used throughout the interface when presenting ICZN datasets Create an easy to use back up system and provide the ability to track and reverse changes Linked with this create different levels of user rights Reconsider how the Descriptive element is dealt with in the interface The descriptive tab is difficult to find and the description tree caused usability issues Some testers questioned why elements such as Common Name ate in the descriptive tab and not the Properties Details panel It is not intuitive to the users in which panel certain types of data should be input If there is a clear definition of which data is stored and accessed in the Properties panel and the Descriptive tab then this needs
7. stated in Test Case 3 the TE now recognises and automatically inputs the taxonomic tree and parent taxon when the user accesses the New Taxon window via the File gt New menu This is a good improvement for usability The remaining issues and suggestions still apply Bugs Functional Issues One tester experienced a save fail Tester tried to add a species as a Child Taxon by right clicking on the genus name in the Freetext panel Save failed java lang NullPointerException 18 EDIT C5 131 Reliability and usability testing report Tester tried adding a species the correct way i e right clicked on genus name in Taxonomic Tree panel and selects New gt Taxon Typed in the name and clicked on Finish The Freetext panel opened with New taxon as the tab heading and the name field blank Typed in the same name into the name field Clicked on Save icon Save failed as before Name didn t appear in tab heading or under genus Not able to delete Moved to a different computer as there seemed to be no way to add the required species The user logged onto the network on both computers Installed EDIT from start Created datasource with same name Taxonomic Tree name created on first machine was present in the Tree panel but the tree was empty Attached same genus name to the tree Able to attach species to the genus but could not use the names that had been tried on the first computer even though they were not visible One
8. tester entered the authority as Miq Ex Uittien but it was saved as Uittien ex Miq 4 4 6 Test Case 5 Creating and saving a heterotypic synonym for the Genus taxon Out of a possible 90 there were 18 deviations 20 in test case 5 Testers consulted the manual on 9 occasions at 5 steps 10 and required assistance on 5 occasions at 2 steps 5 5 Deviations occurred at all steps with no one step creating the main problem Two testers deviated at all steps by finding a different but far less efficient pathway creating a new taxon as they had in the previous test cases and then using the Change Accepted Name to Synonym function in the right click menu of the Freetext panel Three testers wanted to create a synonym by creating a second genus and linking it to the first at the point of data entry Four testers deviated by attempting to add a synonym by right clicking the genus within the taxonomic tree This is because it was possible to add new taxa via this method and the testers assumed they could also add synonyms using the same method However the function is only supplied in the Freetext panel Several testers when entering the synonym name in the Freetext panel pressed return after they had finished typing the data This is an instinctive action used in many computer applications to finalise data input However in this case pressing return opens a new text box to allow the addition of a further heterotypic synonym It is no
9. to ensure that each test case was testing the usability of the software rather than the taxonomic knowledge or experience of the testers For each test case step by step details of how we would expect the tester to complete the test case the expected pathway were documented for the co ordinators The co ordinators then observed the testers and recorded any deviations from the expected pathway The following four points were recorded for each deviation 1 What did the Tester do that was different to the expected path 2 Why did the Tester follow their chosen route 3 Was it obvious to the Tester that they had deviated from the expected path 4 Was the Tester able to correct their error successfully Each instance in which the tester consulted the User Manual or required assistance from the co ordinator was also recorded The tester was questioned about each test case after they had completed the task Often steps had several alternative methods for completing that step here in referred to as multi route steps In these cases the method used was recorded to allow for analysis of the most usable or popular method to inform future development Upon completion of all 14 test cases each tester completed a feedback document This consisted of a questionnaire of 50 statements with scale answers agree undecided disagree not applicable and a list of 50 reaction cards 25 positive and 25 negative from which the tester had to choo
10. using the mouse AND the keyboard Some users will prefer to use the keyboard as much as possible but currently there are actions only accessible using the mouse for example accessing the right click menus in the Taxonomic Tree and Freetext panels Several testers did not instinctively use the right click button and did not seem comfortable using it Also it is difficult to navigate some areas using only the keyboard Having alternative ways of performing every action is considered to be a basic interface design principle e We recommend that the interface is modified in such as way that elements in the Taxonomic Tree appear to behave as if they were sections of a single document See Overall Interface Behaviour on page 7 for further discussion e Amalgamate the right click menus of the Taxonomic Tree panel and Freetext panel Testers were confused as to why some commands were in one menu and not the other and there is no intuitive way to know in which menu a desired command will be found All commands currently found in the Freetext panel menu should be added to the Taxonomic Tree panel menu as the actions of most testers show that they expect the Taxonomic Tree panel to be the main working area If there are commands that cannot be made in either the Taxonomic Tree or Freetext panel they should still appear in the menu but should be obviously inactive i e greyed out so that it will be clear to users that they need only look in the menu of the
11. 3 they d Test Deviations find the test confident Description Consulted Needed 7 completed the y Case case easy takas completing the test case again expected No No No Yes No Yes No Yes No 1 Creating a new taxonomic tree 8 10 5 7 6 8 14 4 13 6 16 2 2 Closing and reopening the taxonomic tree panel 10 13 1 1 10 13 11 8 17 2 17 2 3 Adding a Genus with authority abbreviation attached to the 12 10 5 5 6 5 11 8 15 4 17 2 taxonomic tree created in Test Case 1 4 Adding 5 Species with authority abbreviations attached to the Genus 5 4 0 0 7 5 15 4 16 3 16 3 created in Test Case 3 5 Creating and saving a heterotypic synonym for the Genus taxon 18 20 9 10 5 6 14 4 14 3 17 1 6 Adding and saving a homotypic synonym of a species taxon 14 15 2 2 2 2 13 5 16 2 16 2 7 Changing the homotypic synonym created in Test Case 6 into an 8 9 1 1 2 2 16 2 13 5 17 1 accepted name 8 Changing the accepted name created in Test Case 7 back to a 6 5 1 1 6 5 15 3 12 4 15 2 synonym of the same species name 9 Checking and editing if necessary the parsed Rank of the Genus 6 5 4 3 11 8 14 5 17 2 18 1 taxon Checking and editing if necessary the parsed uninomial and specific gt epithets and authorship of the 5 Species taxa 19 7 o Q 8 2 13 6 ai 3 te 1 11 Deleting a species taxon 0 0 0 0 1 2 18 1 19 0 19 0 12 Inputting the Date Published and Title of the nomenclatural 15 11 6 4 9 6 11 6 12 5 15 1 reference for a species taxon 13 Inputting a nomenclatural s
12. Test Case 7 Changing the homotypic synonym created in Test Case 6 into an accepted name Out a possible 90 there were 8 deviations 9 during test case 7 Deviations occurred at all 5 steps of the test case Only one tester had to consult the manual at one step 1 and two testers required assistance 2 at two steps All but two testers found the test case easy Five were not confident that they d completed the task as expected but only one was not confident that they could perform the test case again One tester first deviated by selecting the swap option in the right click menu in the Freetext panel and required assistance to correct their mistake and find the correct option One tester deviated completely by deleting the synonym and creating a new taxon with the same name due to inability to find the right command One tester had issues with the fact that even though the synonym being changed to an accepted name had a different generic name to the original accepted name after the change the name remained a child of the genus of the original accepted name One tester expressed concern over the ability of the TE to track and implement changes that should occur to taxa following changes to another taxon For example in this test case a homotypic synonym is changed to an accepted name It follows that the original accepted name therefore changes to the homotypic synonym However there is no capability in the TE to automatically track a
13. completed the task as expected and would be able to perform the test case again Several testers deviated from the initial steps by selecting a species taxon in the Freetext area rather than double clicking the species within the Taxonomic Tree panel this was a quicker and easier route than that provided in the expected pathway Three testers deviated by attempting to input a homotypic synonym in the same way as they had input a heterotypic synonym in Test Case 5 by clicking in the text box in the Freetext area and pressing return It was not obvious to these testers that they had not input a homotypic synonym One tester attempted to save the synonym by pressing return after they had input the text as also seen in Test Case 5 Please see the related suggestions made under Test Case 5 One tester created a synonym by creating a new species and changing it to a synonym They used the right click menu in the Freetext area and selected Change Accepted Name to Synonym However this created the wrong kind of synonym and the tester could not find a function to change the type of synonym Suggestions Amend the right click menu in the Freetext panel or the Taxonomic Tree panel if suggestions under Test Case 5 are implemented so that there are two options o Change Accepted Name to Homotypic Synonym o Change Accepted Name to Heterotypic Synonym This will satisfy the instinctive impression of the users that the first step in creating any kind
14. d and the Citations field Add button or make the colour of the Citations field different to the background so it is more visually joined with the Add button Change the right click menu so that New Description is removed this will be replaced by the Add new description field button suggested above This will only leave the New menu where users access descriptive elements making it more obvious what their next step should be and removing the confusion between New and New Description After clicking on New Image Gallery gallery is misspelt galery Comparison to Alpha Version 0 24 The ability to add a citation for the common name has been removed therefore the issues relating to the Add button are no longer valid All remaining issues and suggestions still apply Multi Route Steps When searching for the correct language from the drop down field 11 testers used the method of repeatedly typing the initial letter of the language in the language field until they came to the right language Six testers scrolled using the mouse and 4 used the arrow keys on the keyboard All testers clicked to select the language as opposed to pressing return on the keyboard 4 5 Feedback 4 5 1 Section A Satisfaction Questionnaire The testers expressed satisfaction with several aspects of the TE For example 13 out of 18 testers said the TE did not respond too slowly to inputs question 1 12 out of 18
15. d using the File gt New menu would create a new taxonomic tree file rather than giving them the option to add taxa to the tree One tester explained that he did not expect to find the option to add a taxon under the File menu as he did not consider the taxon to be a file Several testers did not realise that it was possible to enter the authority abbreviation in the Taxon Name field in the New Taxon window This is prompted by the name of the field Taxon Name which prompts for a taxon name only Testers found a solution by adding the authority abbreviation to the text string in the Freetext panel but this required extra steps to be taken Suggestions As with Test Case 1 provide an Add New Taxon box in the Taxonomic Tree panel which is automatically attached to the taxonomic tree 16 EDIT C5 131 Reliability and usability testing report Provide information connected to the Taxon Name field in the New Taxon window to make users aware that it is possible to add data other than the taxon name into the field As there is space in the window this could be written beneath the field so it is always visible Comparison to Alpha Version 0 24 The issue which caused most deviations at this step the fact that new taxa have to be manually attached to the taxonomic tree when using the File gt New menu has been fixed in later versions in an obvious improvement in usability The remaining issues and sugg
16. e document open represented by the tree and not the individual taxon record they were editing and could not therefore see the significance of Save All For this reason we recommend that the interface is modified to hide the underlying structure from the user so as to make the tree behave more like a single document Save All Close All etc should be removed from the top line File menu and Save and Close in this menu should operate at the tree level From this menu Save should result in all unsaved edits being saved i e the behaviour of the current Save All option should be implemented under the label Save Similarly Close from the top line File menu should close the entire tree allowing the user to open a different classification if they wish Since it will still be possible to open multiple Freetext editor panes tabs options for closing individual panes in the Freetext editor could be retained but only be accessible from the individual pane to which they apply We would also suggest adding another menu to the Freetext panel container allowing operations across multiple panels such as Save All and Close All These suggestions have been summarised in the diagram below EDIT C5 131 Reliability and usability testing report 4 3 Tester preferences at Multi Route Steps Several test cases had steps where there were several options or routes that the tester could choose when completing the step Generalised tester behaviour at th
17. e of the testing process and 3 of those only answered no for one test case Therefore the negative responses to that completion question shown in the table below are 10 EDIT C5 131 Reliability and usability testing report derived from a limited number of testers Interestingly they are not the testers who claimed to find learning new computer applications difficult One tester did not claim to find learning new computer applications easy but nevertheless claimed to have found all of the test cases easy This suggests a lack of confidence of the tester but also good usability of the TE Conversely testers who claimed to find it easy to learn new computer applications did not find the test cases easy on numerous occasions Since there appears to be an inverse relationship between level of computer experience and apparent usability of the interface it would appear that while the interface is easy to use it is perhaps too different in behaviour from other commonly used software for it to be easy for more experienced users to pick up quickly Consideration should therefore be given to where elements of the interface can be made to operate in the same way as commonly used software such as Microsoft Word and Microsoft Excel without compromising the functionality of the interface 11 Were the testers fident that Would the gt Did the testers on ere testers be Le Manual Assistance
18. e test cases or are not catered for in the TE 35 EDIT C5 131 Reliability and usability testing report Some responses from the testers concern aspects that were not included in the test cases due to some areas of the TE being avoided due to planned extensive changes For example adding wood anatomy features description geography and distribution data conservation status and adding images and specimen data were all identified as aspects not tested during this process Inputting these data types is possible indicating a gap in the coverage of the testing process rather than functional gaps in the TE In contrast some responses highlighted actual functional gaps as listed below These are either gaps because the functions were not functional in the test version or because the TE does not have the functional capacity to perform them e Transferring large groups of synonyms to another accepted name but keeping all homotypic links e Import of existing data e Lumping and splitting taxa e Taxonomic concepts e Ability to score characters e Automatic pasting of protologues e Quality control checking of names against available databases e Production of hard copy paper publications Question 3 asked the testers if they thought using the TE to produce taxonomic revisions would make the process more or less efficient Out of the 16 who answered 10 testers thought it would make the process more efficient These testers consider
19. e the default nomenclatural status Valid Reorder the Nomenclatural Status drop down menu according to frequency of usage possibly as follows 1 Valid 2 Invalid 3 Illegitimate with following entries in alphabetical order The order could be discussed with taxonomists Possibilities include ordering the menu according to the frequency each status is used in the wider taxonomic literature or by frequency of use within the local datasource Another possibility is having the most recently used items from a drop down list appear in a sub list at the top of each drop down list as is seen in other software applications Re format the Nomenclatural Status drop down menu so the entries are given in their standard abbreviations Add a full stop after valid in nom valid Provide a text field to allow for qualification justification of nomenclatural status If this is already provided create hover over pop ups to explain to the user what information needs to be entered in each box Suggestion in Test Case 2 of a re arrangement of menus on the toolbar with the Preferences menu always visible instead of embedded in the Window menu could make users more aware that they are able to customise what appears in drop down boxes A hover over or help button in appropriate areas could also explain this ability to users 30 EDIT C5 131 Reliability and usability testing report Comparison to Alpha Version 0 24 The Nomenclatura
20. ed that consistent structuring storage organisation of data would allow easier usage of the data for other purposes Some included stipulations such as ensuring that existing data can be imported and that any bugs usability issues are dealt with The remaining 6 testers said they did not think the TE would make the process of producing taxonomic revisions more efficient Three of these testers did not give explanations One tester thought that IPNI and checklist databases are easier to use although these are different tools and one thought that it was not possible to rearrange the taxonomic tree which is a misconception Question 4 asked the testers if they would change their current working practices to use the TE Of the 17 who answered 6 said they would not use the TE with no further explanation Two said they would consider if it was possible to import existing data from GBIF and three said they would when a later version or more robust product becomes available Two said they would possibly change their current working practices to use the TE and three answered Yes with no stipulations or conditions 36 EDIT C5 131 Reliability and usability testing report 5 Discussion There are several issues which should be considered when examining the results of the reliability and usability testing e Although the range of types of taxonomists was wide there were only 4 zoologist included e Although necessary to allow the extracti
21. ee steps Two testers deviated by selecting the species in the Freetext panel using the tab of the already opened taxon record rather than double clicking the species in the Taxonomic Tree panel One tester deviated by clicking initially on the New gt Homotypic synonym function but was aware of the error and able to correct it Fifteen out of 18 testers found the test case easy Four testers were not confident that they d completed the task as expected since the function Change accepted name to synonym does not give a choice of synonym type and when used creates a heterotypic synonym In this case the accepted name had previously been a homotypic synonym and the testers wanted to change it again to a homotypic synonym Because of this several testers recommended that users are given the choice of synonym type when using the Change accepted name to synonym function One tester repeated the statement given by several in Test Cases 5 amp 6 that the same set of functions should be available after right clicking in either the Taxonomic Tree panel or the Freetext area Suggestions After selecting the Change accepted name to synonym function give the user a choice of synonym type appropriate to the code they are using ICBN or ICZN As stated previously unify the menus accessed by using the right click button in the Taxonomic Tree panel and the Freetext panel into one menu This will eliminate the confusion expressed by the tes
22. en to input several taxa and minimise the number of mouse clicks and keyboard to mouse movements requires hence improving usability If possible the new taxon should default to being a member of the same higher taxon as the last entry To enable the input of multiple taxa using the keyboard only thus eliminating the need for keyboard to mouse movements it should be possible to use the tab button to select the new button and select the option by using the return button on the keyboard This assumes that the taxonomic tree and parent taxon are automatically input by the TE Currently users have to type out the genus name every time taking time effort and introducing more opportunity for data input errors This would be improved if the genus name of the Parent Taxon was automatically input into the Taxon Name field in the New Taxon window It should be easy to overwrite this name by clicking on it once the whole word is highlighted in case the user wants to add a synonym for example which has a different genus name Create a warning message at the point of data entry into the New Taxon window that all names have to be entered in full The user should have the option of Don t show this message again Alternatively to the above it could be made possible for users to enter only the species names when they are attached to a genus in the taxonomic tree as is possible in other databases Comparison to Alpha Version 0 24 As
23. enjoyed their session with the TE question 7 more testers thought that the TE would not disrupt their normal workflow than those who thought that it would question 16 and 10 out of 18 testers did not find working with the TE frustrating Also only 5 out of 18 testers thought it was easy to forget how to do things with the TE question 45 indicating a reasonably shallow learning curve However responses to other questions indicate usability issues which have on the whole already been recorded in the preceding sections e Lack of prompts There is never enough information on the screen when it s needed 7 testers agreed 4 disagreed question 18 33 EDIT C5 131 Reliability and usability testing report It is easy to see at a glance what the options are at each stage only one tester agreed question 48 e Confusion over terminology T immediately understood the function of each menu item only 4 testers agreed question 49 e Inefficient Long winded data entry The software allows the user to be economic with keystrokes only 4 testers agreed question 34 e Non intuitive organisation of the menus The organisation of the menus or information lists seems quite logical only 5 testers agreed question 33 e Unpredictability The software hasn t always done what I was expecting 16 testers agreed and only 2 testers disagreed question 41 This software occas
24. ese multi route steps is summarised in this section This information is provided as it may help direct future development of the TE Details of multi route steps specific to individual test cases are given in the test case summaries in the following sections if it is thought the additional information could contribute to future development The most common multi route step was saving changes Out of 97 recorded save actions 44 45 were implemented using the Save option in the menu accessed using the right mouse button within the Freetext panel There were 35 instances 36 where testers clicked the Save icon in the toolbar Half of these occurred during the final two test cases whereas in early test cases usage of this icon was lower This suggests that testers did not notice the icon at first but preferred the method of using it once they were aware of its existence It may be the case that testers only became aware of the icon after having to consult the manual Therefore it may be beneficial to make this icon more conspicuous Testers used the File menu gt Save 13 times 13 In the last test case no tester used this method as apparently more testers had discovered the save icon button which is a quicker method There were only 5 instances 5 where testers used the Ctrl S method Testers have several options when selecting entries from drop down lists For example when choosing a nomenclatural status 17 testers used the mouse to sc
25. estions still apply 4 4 5 Test Case 4 Adding 5 Species with authority abbreviations attached to the Genus created in Test Case 3 Out of a potential 135 there were 5 deviations 3 7 during test case 4 and testers required assistance on 7 occasions 5 No testers needed to consult the manual Reasons for the deviations vary and there is no common pattern It is reasonable to suggest that following Test Case 3 testers were aware of the main procedure of inputting taxa accounting for the lower number of deviations and possibly indicating a relatively shallow learning curve for this activity One of the deviations is the same as the most common deviation in Test Case 3 two testers used the File gt New menu and not the right mouse button to access the New Taxon window therefore encountered the problem of the greyed out Taxonomic Tree and Parent Taxon fields where by the fields were empty and required input but there were no visual clues to guide the testers As explained previously this issue has been rectified in later versions of the TE Two testers did not input the full taxon name but abbreviated the genus for example A acuminata rather than Alyxia acuminata The TE gives no warning that this format is not recognised and could cause problems during future data export analysis etc The testers had to be made aware of the issue by the co ordinators One tester again as seen in Test Case 3 was not aware that it is pos
26. from 7 out of the 19 testers suggest that the tester had to edit the parsed rank of the species although no further details are given 26 EDIT C5 131 Reliability and usability testing report Suggestions One tester suggested that upon entering new taxa in the New Taxon window the TE editor could ask the user to confirm the parsed rank as it is entered For example a pop up window asking is this a genus is this a subspecies etc This could be considered but if implemented should not disrupt the data entry process with easy confirmation possible for example pressing the return button once to select yes which is highlighted by default Users could be given the option of whether the pop up window appears or whether they check the rank in the Properties panel in their own time Consider repositioning the Rank field so it is easier and quicker to find Comparison to Alpha Version 0 24 The issues and suggestions still apply Bugs Functional Issues One tester input the species names with genus abbreviations during Test Case 4 At this stage due to parsing problems this was edited in the Freetext panel However the changes could not be saved successfully 4 4 12 Test Case 11 Deleting a species taxon No testers deviated during Test Case 11 or consulted the manual One tester required assistance Only one tester did not find the test case easy All were confident that they had completed the task as expec
27. g and clicking on the Descriptive tab at the bottom of the Freetext panel 11 deviations and accessing the right click menu to create a new description 8 deviations Six testers looked for a Common Name field in the Properties panel and one looked in the Windows gt Show View menu Four testers had no idea where to look and were given assistance from their co ordinator Once in the Descriptive tab testers were unsure how to proceed to enter a common name due to the lack of visual prompts Two testers clicked the left mouse button expecting to see a menu appear One tester could not figure out what to do next and had to consult the manual while three received assistance Two testers clicked the Display All button expecting to see a list of ctiteria descriptive elements Five testers tried to enter a common name in the Edit Description s Label text box One tester created another new description After choosing the correct route of right click gt New gt Common Name one tester deviated by clicking the Add button expecting to be able to add a common name Other testers clicked the Add button after entering the common name expecting that this would update save their changes 31 EDIT C5 131 Reliability and usability testing report Several testers had difficulty finding the correct language due to the large number of languages in the drop down menu which are not in a particular order It is not poss
28. ging relationships within the Taxonomic Tree panel Activate taxa in the Freetext and Properties areas upon a single click in the Taxonomic Tree area Reconsider synonym terminology used in datasources set up to use the International Code of Zoological Nomenclature 20 EDIT C5 131 Reliability and usability testing report Consider linkages to or import of datasets containing standardised terms such as authority abbreviations and publication journal abbreviations Consider amending the hierarchy of the taxonomic tree panel so that synonyms are listed beneath their accepted name but appear at the same level in the tree Currently synonyms are indented in the same way as child taxa which tends to give the wrong impression of the nature of the relationship being expressed If this is implemented ensure that the synonyms are easily co distinguished from the accepted child taxa by retaining the symbol Comparison to Alpha Version 0 24 The issues and suggestions still apply 4 4 7 Test Case 6 Adding and saving a homotypic synonym of a species taxon Out of a possible 90 there were 14 deviations 15 in test case 6 Testers consulted the manual twice at one step 2 and required assistance twice at two steps 2 The deviations were well distributed across the 5 steps and no particular step caused a consistent problem Thirteen testers found the test case easy where as 5 did not All but two of the testers were confident that they d
29. he option New gt Taxon rather than New gt Taxonomic Tree due to confusion over the terms used Several other testers completed the task without deviation but were still unclear over the meaning of the term taxonomic tree One tester deviated by using the New icon above the Taxonomic Tree panel finding an alternative and shorter route to those described No other testers used or mentioned this button The majority of the testers 16 19 felt that they would be confident performing the test case again which suggests that despite initial difficulties when using the database for the first time the process of creating a new taxonomic tree is easy to remember Several testers expressed confusion over the term Label in the Taxonomic Tree window The term Reference Label also caused confusion Testers were unsure what information should be input which is related to the confusion over what the Taxonomic Tree actually is Suggestions Provide some initial getting started prompts when the TE is first installed and opened for example a Create new taxonomic tree or New button in the Taxonomic Tree panel allowing users to access the New Taxonomic Tree window with one left click or a getting started help box or wizard Make the New button adjacent to the Save icon in the toolbar more conspicuous Change the Label term to Name EDIT C5 131 Reliability and usability testing rep
30. his message again once they know the function of the command Comparison to Alpha Version 0 24 The issues and suggestions above still apply 4 4 4 Test Case 3 Adding a Genus with authority abbreviation attached to the taxonomic tree created in Test Case 1 Out a possible 111 there were 12 deviations 10 in Test Case 3 Deviations occurred at 8 of the total 10 steps 5 testers consulted the manual at 3 steps 4 5 and 6 testers needed assistance at 6 different steps 5 4 Eleven testers found the test case easy while 8 did not Four were not confident that they d completed the task as expected The most deviations occurred at step 5 clicking on the Browse button of the Taxonomic Tree field in the New Taxon window Four testers deviated at this step The problems were caused by the fact that these testers accessed the New taxon window via the File gt New menu and not the right mouse button Using this method the TE did not automatically detect the taxonomic tree to which the taxon should be attached This meant that the testers had to input the taxonomic tree manually but there were no visual cues to prompt them to do this the taxonomic tree data field was greyed out as if inactive One tester deviated from the entire pathway by using the New icon on the toolbar Two testers first looked under the Edit menu for the reason explained previously in Test Case 2 they considered the taxonomic tree to be a file an
31. i EDIT K E RTO Distributed d Institute of amp C2 Taxonomy lh y Project no 018340 Project acronym EDIT Project title Toward the European Distributed Institute of Taxonomy Instrument Network of Excellence Thematic Priority Sub Priority 1 1 6 3 Global Change and Ecosystems C5 131 Reliability and usability testing summary report Due date of component Month 53 Actual submission date Month 53 Start date of project 01 03 2006 Duration 5 years Organisation name of lead contractor for this component 29 RBGE Revision final Project co funded by the European Commission within the Sixth Framework Programme 2002 2006 Dissemination Level PU Public PP Restricted to other programme participants including the Commission Services RE Restricted to a group specified by the consortium including the Commission Services cCoO Confidential only for members of the consortium including the Commission Services Contents Rw Ne FAXOCUUIVE SUMING esisin riarann anO tasudhiessddeszsssddesdeaetsncecctssceasesieeadasusedeasusnediecdensteaetensecsssss 3 NMP OAUCH OM i565 snc tink fe ivisvegredeasdeedicndesseeseystectastsbatceyasseensttesucesdensenntiesentessvdisesaeeutionde decoys sewesasdebuteedees 5 MLCT OSs ii s2cssitscessstties cesscnievsecoendeetscnsevisvetuntecsntdebatectsrtsnnstesveneadentenetiersaceewateseusdeudieasyieesesri enaiexsiceassees 5 RESUS i ssn
32. ible to press return after entering a common name into the text field this is instinctive yet it causes the text that has just been input to disappear and the user has to re enter the data It is not clear how else to save the data that has been input several testers clicked the Add button This is exacerbated by the fact that the common name tag in the description tree does not expand once the common name has been successfully entered the user has to manually expand the section in order to see it This created confusion over whether the task had been fully completed Suggestions The Descriptive tab is currently inconspicuous and was difficult for the testers to find and one tester expressed confusion over why common name data is input in the descriptive tab and not the Properties panel It is not intuitive to the users in which panel certain types of data should be input If there is a clear definition of which data is stored and accessed in the Properties panel and the Descriptive tab then this needs to be clearly explained to the user If it is not possible to clearly define the difference thought should be given to amalgamating the Properties panel and the Descriptive tab This is could be done for example by creating multiple tabs in the Properties panel each tab for a group of clearly defined and related descriptive elements Users should have the ability to remove hid descriptive elements which they do not use via the preferences menu
33. ic concepts or classifications For example some taxonomists may want to maintain a classification based on morphological data and a classification based on molecular data This would involve the same species or taxa with associated information but arranged in a different way in the taxonomic tree Although the TE allows for multiple trees to be created there is no facility to easily duplicate and transfer the taxa from one tree to another where they can then be arranged according to a different taxonomic concept or classification Related to this there is also no facility for comparison between trees of the same taxa but treated with different taxonomic concepts or classifications Failures of reliability were rare with bugs or errors encountered only 7 times throughout the process These mainly related to java null pointer errors encountered when trying to save changes Often the testers could not make the TE recover from such errors and had to either restart or reinstall the software More attention needs to be given to gracefully trapping null pointer errors 2 Introduction The Taxonomic Editor henceforth TE is part of the EDIT Cyberplatform of Taxonomy designed to create useful tools for taxonomists and make the taxonomic revision process more efficient Functionality and usability testing has been carried out to assess the performance and intuitiveness of the Taxonomic Editor This report describes the testing process and
34. ionally behaves in a way which cannot be understood only 5 disagreed question 46 Because of these issues and despite there being some positive feedback only 4 testers would recommend the software to their colleagues question 2 and 8 agreed that they prefer to stick to facilities that they know best question 20 For charts showing the testers responses to all 50 questions of the satisfaction questionnaire see Appendix 1 4 4 5 2 Section B Reaction Cards The testers were given a list of 50 words or reaction cards and were asked to choose the five words that best described their experience with the Editor These have been arranged into the word cloud in Figure 1 below Reaction cards that were not chosen are not shown 34 EDIT C5 131 Reliability and usability testing report Understandable Relevant otValuable TooTechnical Efficient vinconici TimeSaving Clear EasyToUse i Confusing Attractive Frustrating seige Inconsistent vanans _ Seful Usable Feist i Fast TimeConsuming HardToUse Consistent Complex nomprehensibie Irrelevant StraightForward Figure 1 Reaction cards chosen by testers relating to their experience of working with the Taxonomic Editor Greater Jrequency of a word being chosen is indicated by larger lettering and bolder colouring As is seen in the word cloud the two most commonly chosen words were Inconsistent and Useful each word was selected 7 out of a possible 18 times
35. ions is an accurate indication of difficulties experienced by the testers Table 1 shows that the most difficult task based on asking the testers whether they found the task easy was inputting a common name in the descriptive panel This is the only task where less testers found the task easy than those who did not 7 easy 12 not easy In all other tasks more testers claimed to find the task easy regardless of the number of deviations times the manual was consulted and times the co ordinator offered assistance After only one test case deleting a taxon were all the testers confident that they d completed the task as expected and were confident that they could perform the test case again In all but one test case the number of testers who received assistance from the co ordinator was higher than the number of testers who consulted the manual This is contrary to the request that all testers first consult the manual before being given assistance This may have been due to language difficulties in some cases but in any case may reflect a lack of enthusiasm for consulting the user manual This strengthens the various suggestions given in the test case summaries below to create more help guidance and prompts built into the interface Out of the 21 instances where testers did not feel confident that they could perform the test case again 13 of these instances came from one tester Only 7 testers answered no to this question through the whol
36. l Status menu has been amended and simplified A text field has been provided called Rule Considered to allow for qualification justification of nomenclatural status as suggested above Help may be beneficial for the Rule Considered field to explain to users what information to put in this field as it may not be obvious to some users A label of Qualification or Justification might be more easily understood than Rule Considered The remaining issues and suggestions still apply Multi Route Steps To find and select the nomenclatural status of valid from the drop down menu 14 testers used the mouse to scroll down the menu and clicked on the valid option 5 used the keyboard arrows to move up and down the list and pressed return on the keyboard and no testers typed V into the field 4 4 15 Test Case 14 For a species taxon input a Common Name in the Descriptive Panel Out a possible 286 there were 41 deviations 14 during test case 14 Testers consulted the manual on 9 instances 3 but required assistance 42 times 15 Twelve testers did not find the test easy compared to 7 who did the only test case where the number of testers who found the test case easy was lower than those who did not However only 2 were not confident that they d performed the task as expected or would not be confident performing the test case again Deviations were fairly evenly spread but were more common at two stages findin
37. ll Interface Behaviour The current interface overexposes the user to the structure of underlying data This no doubt arises from the interface being developed as an extension to the Eclipse IDE It should be borne in mind that by and large users will come from a word processing background and this will influence their expectations in respect of the behaviour of the taxonomic editor interface Specifically users coming from a word processing background will be used to organising their work into single large files probably at the genus or family level in which treatments for individual taxa are treated as subsections of a single document However the current interface works more like a directory tree with each node in the taxonomic tree acting like a folder in EDIT C5 131 Reliability and usability testing report which data about each taxon and folders representing sub taxa are stored Very few taxonomists would naturally organise their work in this way The effect of this is that when the user interacts with data in the Freetext panel and the Properties panel of the current interface they are editing a single file even though their word processor experience leads them to assume that they are actually editing subsection of a single larger document Confusion then arises when it comes to opening closing and saving objects For example users did not understand the distinction between Save and Save All options since they only considered they had a singl
38. ly one stated that they would not be confident performing the test case again All the deviations occurred either when activating the taxon in the Freetext and Properties panel or trying to find the appropriate place to input the data Several issues recorded in previous test cases resurfaced such as testers not noticing the ability to expand the sections in the Properties panel despite using them in previous test cases and looking for the function in the right click menu in the Freetext area Several testers were not aware that it was necessary to save their work and had to be prompted by their co ordinator to do so 29 EDIT C5 131 Reliability and usability testing report One tester stated that valid should be the default nomenclatural status One stated that there are too many options in the Nomenclatural Status drop down menu however there is the ability for users to customise this to suit their needs Another stated that the drop down menu could be ordered according to frequency of usage and the format changed so the menu contains the standard taxonomic abbreviations For example nomen novum nom nov According to one tester nom valid should have a full stop after valid and nom valid should not appear in the text string following the species name Two testers would like a field where they can qualify the nomenclatural status which they input Should they use the Annotations field Suggestions Mak
39. n or add a pop up window that asks the user what kind of synonym they d like to add see Test Case 6 and give them the opportunity to cancel the new synonym box This would allow users to create the different types of synonyms in the same way Alternatively make it easier for the users to immediately correct their mistake of pressing return by pressing the back space button on the keyboard which would cause the text box to be deleted Add Synonym option to the right mouse button gt New menu in the taxonomic tree area Additionally synchronising the two menus accessed via right mouse button that in the Taxonomic Tree panel and that in the Freetext panel as far as possible would improve usability as it is not intuitive as to which menu is needed when performing certain tasks Based on feedback from the testers and the necessity to use the Taxonomic Tree panel most if not all functions should be present in the Taxonomic Tree panel menu When taxa are dragged within the Taxonomic Tree panel and attached to a new parent taxon upon release of the mouse button a pop up window could appear and ask the user to confirm the relationship between the taxon and the new parent taxon The message could ask for gt with Yes No options If the user clicks Yes they are then asked about the type of synonym if using the ICBN This will create a quick and easy way gt example Is this taxon a synonym of of creating synonyms and chan
40. nd implement such changes to taxonomic relationships In a related issue there is no facility to maintain synonyms of synonyms which therefore does not allow for more complex taxonomic treatments and experimental changes and reversals in synonymy Suggestions As stated previously testers are confused by the two distinct right click menus accessed via the Freetext and the Taxonomic Tree panels This makes it more difficult to find functions than if all the functions were found in both menus Therefore these menus should be amalgamated as much as possible Allow for more sophisticated automation of taxonomic changes such as the automatic swapping of synonymy Comparison to Alpha Version 0 24 The issues and suggestions still apply Bugs Functional Issues One tester encountered an error when trying to save the changes 23 EDIT C5 131 Reliability and usability testing report When using the Swap synonym with accepted function twice with the same two taxon records the synonym name disappears from the TE when changing it back to a synonym The synonym has to be re input 4 4 9 Test Case 8 Changing the accepted name created in Test Case 7 back to a synonym of the same species name Out a possible 126 there were 6 deviations 5 during test case 8 The deviations occurred at 4 out of the 7 steps of the test case Only one tester had to consult the manual at one step 1 and 6 testers required assistance 5 at thr
41. nd which field is the correct field to input the data Two testers expressed confusion over the Title field should this be the title of the journal book or book chapter Where would you enter a journal title One tester did not understand all the values in the Reference Type field One tester would prefer the fields for the reference to be in the same order as you would find in a protologue therefore page numbers should come after volume number for example One tester would like the TE to link to a database of full reference titles and abbreviations of periodicals that can be linked to and selected from in the same way that names could be linked to IPNI This would ensure that the data is stored in the standard format and is compatible with other datasets serving the taxonomic community One tester wanted to be able to move from one field to another within the Properties panel using the arrow buttons on the keyboard as well as the by using the mouse 28 EDIT C5 131 Reliability and usability testing report Suggestions Allow navigation of the Properties panel using arrow keys on the keyboard Provide guidance as to what data should go in which fields in the Properties panel for example hover over pop up boxes with brief explanations and or consider changing the name of the more ambiguous or confusing names such as Editable cache Consider changing the order of the reference fields in the Properties panel As
42. nu and selecting the Edit option Several testers considered having to expand the menus in the Properties panel as unnecessary clicking and requested that the menus are at least partially expanded by default so it s easier to find the desired field s Three testers requested a Rank checked button or similar as although the rank may be correct there is no easy way to remove the warning The results from 8 out of the 19 testers suggest that the tester had to edit the parsed rank of the taxon although no further details are known Suggestions Consider activating taxa in the Freetext and Properties panels after only one click of the taxon in the Taxonomic Tree area Consider changing the Properties panel so that main menus are expanded by default Supply an easy method for the user to remove the check rank warning from the Freetext and Properties panels Comparison to Alpha Version 0 24 The properties panel has changed significantly and is now called the Details panel The sections are expanded or minimised upon opening of the TE according to how they were when the user last closed the TE This is an improvement to usability as some testers did not like having to expand the sections every time to check the parsed rank of their taxa However when the sections are expanded some fields are hidden due to the new arrangement of two separate panels The upper panel is hidden behind the lower panel when sections are ex
43. o testers deviated at the initial step of closing the panel by closing the entire programme using the File gt Exit menu One tester did not find the x icon and required assistance Six testers deviated whilst trying to reopen the panel by looking first in the File menu where they expected an Open command which would allow then to open the taxonomic tree panel The testers indicated that they considered the taxonomic tree to be a file similar to a file found in Microsoft Word that could be acted upon in a similar way i e via the File menu This indicates confusion over the structure of the TE and the nature of the components in the interface The remaining two testers deviated at this stage by first clicking the Taxonomic Perspective button Eleven of the 19 testers found the test case easy several commented that it was easy to close the panel but not to re open it and that it was not intuitive Several testers also questioned why there is the ability to close this panel in the first place as it is essential to use it to access all taxa files Two of the testers stated that they would not be confident performing the test case again This suggests that even after seeing the Window gt Show View menu the testers still did not feel that it was the correct location for this command 14 EDIT C5 131 Reliability and usability testing report Suggestions Several testers recommended that there is an option added to
44. of synonym is to first create a new taxon and then amend its status 21 EDIT C5 131 Reliability and usability testing report In the right click menu of the Freetext panel and or the Taxonomic Tree panel give an option to change a synonym from one synonym type to the other to allow for easy correction of mistakes Additionally see suggestions under Test Case 5 Make the icons of heterotypic and homotypic synonyms in the Taxonomic Tree panel consistent with the icons in the Freetext panel Comparison to Alpha Version 0 24 The issues and suggestions still apply Bugs Functional Issues Two testers encountered an error whilst trying to save their work Save failed Tried to create homotypic synonym for three species and save failed each time Restarted the TE choosing not to save Still didn t work so installed the TE on different computer and repeated test cases 1 6 again When unable to save added java6 bin to the path and restarted the computer Still unable to save As we needed a homotypic synonym for the next Test Case tester added a homotypic synonym to the genus and was able to save One tester encountered an error whilst clicking from the genus to the species name When changing between the genus and the species name the programme gave an error message saying an error has occurred when activating this view java lang null pointer exception 499 EDIT C5 131 Reliability and usability testing report 4 4 8
45. on of specific interface and usability issues the recording process was complicated and co ordinators did not always record deviations fully e Language difficulties may have influenced the extent to which some co ordinators and or testers could express and record their opinions e Not all test results had been received prior to the extraction of data for this report e Some aspects of the test version of the Taxonomic Editor were out of date by the time the test process was complete This was necessary to ensure that the test version was mainly free from bugs and was consistent with the testing documentation and user manual For this reason all issues and suggestions were compared a later version of the TE Alpha 0 24 237
46. ort A pop up bubble or help icon next to the terms which have caused confusion Taxonomic Tree Label and Reference to give guidance as to what kind of information should be input would help Comparison to Alpha Version 0 24 The term Taxonomic Tree relating to the Taxonomic Tree panel caused some confusion but this has now been changed to Taxon Navigator following a previous creation of a TRAC ticket Co ordinators to whom this was mentioned agreed that this is an improvement Taxonomic Tree has been changed to Taxon Classification which is also likely to alleviate some of the confusion over the terms previously used Multi Route Steps 11 testers accessed the New Taxonomic Tree window via the File gt New gt Taxonomic Tree route compared to only 5 who accessed the menu using the right mouse button which is a shorter route After inputting the data in the New Taxonomic Tree window all the testers clicked on the Finish button requiring an additional keyboard to mouse movement rather than pressing return on the keyboard which is a shorter route 4 4 3 Test Case 2 Closing and reopening the taxonomic tree panel Out of a possible 76 1 tester used the manual at 1 step 1 and 10 testers needed assistance at 4 different steps 13 There were 10 deviations 13 Most difficulties were encountered whilst trying to re open the panel via the Window gt Show View gt Taxonomic Tree menu Tw
47. panded obscuring the data fields at the bottom of the panel This should be rectified if possible by removing the split screen system and having a single panel with all sections visible at all times 5G y EDIT C5 131 Reliability and usability testing report The check rank warning still appears in the Details panel If the user edits the rank then the warning disappears This is an improvement in usability However if the data has been correctly parsed and the user has no need to edit the rank there is no quick way of the user removing the warning The user must change the rank to an incorrect rank and then change it back to the correct rank this is not intuitive and it is likely that many testers will not become aware of this Therefore another option should be given for the user to confirm that the rank is correct following which the warning disappears The remaining issues and suggestions still apply taxa do not appear in the Details panel unless the user double clicks on a taxon in the taxonomic tree panel Also if the user clicks a single time on a taxon in the tree the Details panel shows the message Current selection does not support this view The user is likely to think that there is an error or bug when in fact they only need to click again Enabling the opening of taxa in the Details panel after only clicking once would remove this usability issue Bugs Functional Issues One tester reported problems with the buttons to
48. presents summaries of the data collected Raw data can be found in the separate Appendix document It is strongly recommended that the Appendix is also consulted as along with all the data it also contains all comments received by the testers Please note that the structured testing was carried out on the test version of the TE for which development was frozen so some issues encountered may have already been addressed in subsequent versions All issues reported have therefore been checked against a later version of the TE Alpha version 0 24 to make clear whether the issues still remain 3 Methods The plan of the testing of the Taxonomic Editor was documented and distributed in advance to allow for comments and suggestions see Usability Testing of the EDIT Taxonomic Editor v2 1 0 200912300955 Test Plan Discussion Document Testing Co ordinators were appointed at various EDIT institutions across Europe see Appendix 1 1 and four training events for the co ordinators were organised and implemented by RBGE Co ordinators were charged with recruiting testing volunteers and recording the EDIT C5 131 Reliability and usability testing report results of the testing procedure Testers ideally were to have experience in carrying out taxonomic revisions The testing was divided between structured and unstructured testing The structured testing was based on test cases representing common tasks performed as part of a taxonomic revision This was
49. revisions 4 agreed strongly 8 agreed and 5 neither agreed nor disagreed Three testers disagreed with the statement that they find learning to use new computer applications easy 2 agreed strongly 10 agreed and 2 neither agreed nor disagreed Most of the testers 12 18 consider the production AND publication of a taxonomic revision NOT to be separate exercises One stated that they are currently treated as separate exercises but that ideally it should be a continuum Of the testers who consider the production and publications NOT to be separate exercises all 12 of them use Microsoft Word to perform both activities Some testers additionally use Excel 2 Adobe Photoshop 2 and Hennig86 1 Of testers who consider the production and publications to be separate exercises all 5 use Microsoft Word as the main application for both activities Additionally Delta 1 Access 1 and InDesign 1 are used Most testers 12 18 stated that they use taxonomic database as patt of their taxonomic revision activities Generally these are not tools for managing their data but external data resources for example Algaebase Cichorieae Portal EDIT Inside Wood IPNI International Plant Names Index and various checklists In addition some testers use the herbarium or specimen management systems BG BASE 2 and Brahms 1 and the character coding and key building applications Delta Description Language for Taxonomy 1 and Lucid 1 4 2 Overa
50. roll the drop down list and then clicked their choice Five testers used to keyboard arrows and pressed the return button In contrast when selecting the language of the common name the testers entered in Test Case 14 11 testers repeatedly typed the first letter of the language until their desired language was shown compared to 6 who used the mouse This is due to the very long unordered list of languages which made it very difficult to detect the desired language by scrolling Therefore there seems to be a preference to locate the desired option in a drop down list using the mouse rather than the keyboard and only when this is problematic will more testers use the keyboard When making selections from pop up windows for example when inputting a new taxon and searching and selecting the parent taxon testers rarely used the fasted method of double clicking their selection Generally they click on the selection once and the click the OK Finish button Several testers used the keyboard arrows to navigate to their selection but then used the mouse button to click OK Finish button This required an extra keyboard to mouse movement rather than the quicker method of pressing return on the keyboard Although some of these issues clearly relate to users experience levels and are not usability interface issues it may be relevant to future development of the TE in terms of ensuring that certain routes are provided or improved while not dedicating an excessi
51. s includes the three testers who pressed delete on the keyboard for these testers using the mouse to click on the OK button required an additional keyboard to mouse movement indicating that they were unaware that they could select OK using the keyboard 4 4 13 Test Case 12 Inputting the date published and title of the nomenclatural reference for a species taxon Out a possible 141 there were 15 deviations 11 during test case 12 Deviations occurred at all but one of the 9 steps Six testers consulted the manual 4 and 9 required assistance 6 Eleven testers found the test case easy but 6 did not Five were not confident that they d completed the task as expected and one would not be confident performing the test case again One tester explored several possible routes but could not figure out what to do and had to consult the manual Two testers added a reference by creating a new reference rather than adding data to the Nomenclatural Reference section within the Properties panel Two testers added the reference data directly to the name string one entered the data in the Editable Cache field the other did not specify One tester stated that they input the data into the Editable Cache field but that the data was not parsed or added to the Freetext string One tester encountered a problem after changing Protect cache field from overwriting to Yes Several testers expressed confusion over the large number of fields a
52. se 5 cards and explain their reasons for that choice Finally a short open ended questionnaire enquired about the testers specific working practices and willingness to use the TE in the future Unstructured or ad hoc testing was carried out by testing co ordinators or other interested parties This testing allowed the participants to explore and test the TE without constraint in contrast to the structured testing The majority of unstructured testing was carried on the most recent Alpha version of the TE to allow for reliability testing of a later version These results will be presented elsewhere All results were forwarded to the Royal Botanic Garden Edinburgh for assimilation and analysis 4 Results In total 8 co ordinators were appointed and 19 testers have completed the testing procedure to date 18 testers completed a Tester Profile Questionnaire and a Feedback Document Further EDIT C5 131 Reliability and usability testing report results have been delayed but are still expected Any additional issues identified by the further results will be communicated at a later date 4 1 Tester Profiles The full results of the tester profile questionnaires are given in Appendix 1 2 Of the 18 testers who returned profile questionnaires 14 are botanists and 4 are zoologists The majority of testers 14 18 perform taxonomic revisions as part of their job Only one tester disagreed that they are experienced in carrying out taxonomic
53. sible to add the authority abbreviation in the Taxon Name field in the New Taxon window and input the abbreviation into the Freetext panel instead A related suggestion is given in Test Case 3 Fifteen of the 19 testers found this test case easy and 16 were confident that they completed the test case as expected Several testers commented on the fact that entering multiple taxa is a more long winded process than necessary as after each taxon is added the New Taxon window closes and has to be re accessed Several were not happy at having to re type the genus name for every new taxon One tester found it irritating that sec appears as default after the taxon name in the Freetext panel 17 EDIT C5 131 Reliability and usability testing report One tester wants the TE to communicate with other databases such as IPNI for either selection or validation of names Suggestions Make multiple entries of taxa a more efficient process in terms of time and effort This could be achieved in several ways but perhaps the most usable would be to add an additional button to the New Taxon window alongside the current buttons of Finish and Cancel This button could be called Add Another or something similar when a user clicks this button the taxon just input is saved and added to the Taxonomic Tree panel and a clean New Taxon window opens instantaneously This would greatly reduce time tak
54. ssesksnsteens cedentsesseesseesausthutetenodnessintvetivass anesuustoawacssieaneess cea eehevea selon uesSuasesuduegsvecysus Soaks REER 6 4 1 Tester PrOMES eines sswessestacesstevenssesnsessuassamstchavtessestbensaese cuessenkseseduebatnen cosyetsbvesdeeteasteds EEE 7 4 2 Overall Tntertac Behaviour s ccsciceiscsscsideesycececsnntestiereseireenntSessnesiustessoedssunteuntewdunersenttedsaseonte s 7 4 3 Tester preferences at Multi Route Steps sesiet sdieri Eii EEE REE EEEE EEEE 9 4 4 Jest CASES 210 dessuesvucsscensceuscesdeaseoutestunes ceases kiirii ai ai apaia a aaa aiaia aa aaiae 10 4 4 1 Summary Data rcdaiseuctssiecuedsdvseueiseh evatveuentecsien suddies avfas shuevetddas seats nansatercieateasssteviassees 10 4 4 2 Test Case 1 Creating a new tAaAXONOM C tlCC cececsecssessseseeeecseecseeeseeeseesesseeeneeeeeees 13 4 4 3 Test Case 2 Closing and reopening the taxonomic tree panel ccccceseseeeeteeeees 14 4 4 4 Test Case 3 Adding a Genus with authority abbreviation attached to the taxonomic tree created in Test Case dcit vistisstceestssvesss seeeutensdvancessetessnvdvacesudveuvealeveatonvivsvesadeustasbuvseuesetestareeve 16 4 4 5 Test Case 4 Adding 5 Species with authority abbreviations attached to the Genus Created tn Test Case Josieus wei a i a Eea a aaa Eaa aaa aaus 17 4 4 6 Test Case 5 Creating and saving a heterotypic synonym for the Genus taxon 19 4 4 7 Test Case 6 Adding and saving a homotypic synonym of a species taXon
55. stated previously considering having some of the sections expanded by default suggestion from a tester that Name Author and Publication should always be in view Comparison to Alpha Version 0 24 The Nomenclatural Reference section in version 0 24 has been greatly simplified with only two fields a field to search for a reference or a text box to enter a new reference in any format The Editable cache and other fields with confusing terms no longer exist This will eliminate many of the usability issues encountered during this test case When the Details panel is activated sections are expanded or minimised depending on whether they were expanded or minimised when the tester last used the panel This allows for customisation and avoids the need to repeatedly expand a particular section when for example inputting reference details for successive taxa This is an improvement in usability The issues of navigating the panel using the arrow keys and linking to datasets of standardised reference title abbreviations etc still apply 4 4 14 Test Case 13 Inputting a nomenclatural status of Valid for a species taxon Out a possible 125 there were 6 deviations 5 during test case 13 The deviations were fairly evenly spread across the 8 steps No testers consulted the manual but 4 required assistance 3 Fourteen testers found the test case easy and 4 did not however only 1 was not confident that they d performed the task as expected On
56. t possible to remove this text box in the event of a mistake using the keyboard the user has to use the right mouse button to access Delete in the menu This forces the user into a keyboard to mouse movement Several testers did not realise that the entry was not saved automatically as until this point they had not been required to save manually Often the co ordinator had to remind testers to save their work 19 EDIT C5 131 Reliability and usability testing report One tester commented that it should be enough to click once on a taxon within the Taxonomic Tree panel to activate it in the Freetext and Properties area Fourteen of the 18 testers who completed this task found it easy and were confident that they d performed the task as expected Only one tester was not confident that they could perform the test case again The zoological taxonomists pointed out that homotypic and heterotypic in regards to synonyms are terms that are not recognised in the International Code of Zoological Nomenclature One tester requested that the authority abbreviations could be validated against a list of standard abbreviations One tester pointed out that synonyms should be in line with the parent taxon within the taxonomic tree panel rather than indented as accepted taxa are This is the standard format in other taxonomic publications Suggestions Either remove the ability to add new synonyms by pressing the return butto
57. tasvesovstuesseessecesousseaseesecesenas 34 4 5 3 Section C Working Practices Questionnaite cccccccccsessesssceesseesseeesseeeseeesseeeeees 35 DISCUSSION sira E EEEE EE EA EAE PERO 37 EDIT C5 131 Reliability and usability testing report 1 Executive Summary All testers currently use Microsoft Word for either the production or publication of their taxonomic revisions of both Ten out of 16 felt that using the Taxonomic Editor would make the revision process more efficient However only three said they would replace their current system to use the test version of the Taxonomic Editor Several high level usability issues were found repeatedly throughout the results leading to the following suggestions e Testers have shown reluctance to consult the paper User Manual so a more dynamic and accessible help system is required Include more on screen prompts and help such as hover over boxes help boxes or where space allows descriptions of expected input beneath data entry fields Define potentially confusing or ambiguous terms or explain what data should be input in the boxes which are causing difficulty in hover over boxes help boxes and or a searchable glossary added to the Help menu e Relating to the above document the standard behaviour of controls found in the interface such as drop down boxes These descriptions should be added as a new section of the User Manual e Ensure that all menus and actions are accessible and enforceable
58. tatus of Valid for a species taxon 6 5 0 0 4 3 14 4 17 1 16 1 14 Fora species taxon input a Common Name in the Descriptive Panel 41 14 9 3 42 15 7 12 17 2 15 2 Table 1 Summary data for test case scenarios included in the usability testing of the Taxonomic Editor Structured Usability Testing of the EDIT Taxonomic Editor 4 4 2 Test Case 1 Creating a new taxonomic tree Out of a possible 76 instances 19 testers and 4 steps 5 testers consulted the manual 6 5 and 6 required assistance at 3 steps 7 8 during the creation of a taxonomic tree There were 8 deviations 10 Despite the deviations only 4 testers found the test case problematic However 6 testers were not confident that they had completed the task as expected but gave no additional explanation as to why Most difficulties were in accessing the New Taxonomic Tree window and 6 testers deviated at this step Three testers deviated after identifying the Taxonomic Tree panel as the correct place to create a new taxonomic tree but they clicked the left mouse button which has no function rather than the right mouse button This suggests that it is not instinctive for all users to use the right mouse button Although this issue is likely to become less important as the user spends more time using the TE it may cause some initial confusion and frustration in the absence of other set up prompts One tester deviated by choosing t
59. ted and could perform the test case again One tester expressed that it could be a problem if another user of the same datasource deleted taxa The tester requested that it should be possible to retrieve data that has been deleted and also that deletion of names is not possible for all users One tester again asked that the right click pop up menus are the same in the Taxonomic Tree and Freetext panels One tester requested a delete function in the Freetext right click menu a delete function is in the Taxonomic Tree panel but not in the Freetext menu Where the functions are currently found does not appear to be intuitive See suggestions under previous test cases to consider the assimilation of the right click menus in the Taxonomic Tree and Freetext panels Suggestions Create different levels of user rights that can be set by the administrator Create an easy database back up system which allows simple back up of data and retrieval should any mistakes losses occur Comparison to Alpha Version 0 24 The issues and suggestions still apply ee EDIT C5 131 Reliability and usability testing report Multi Route Steps When deleting the species taxon 17 out of 19 testers selected Delete via the right click menu of the Taxonomic Tree panel 3 pressed delete on the keyboard and 1 used the Edit menu gt Delete When confirming the deletion all 19 testers used the mouse to click OK rather than pressing return on the keyboard Thi
60. ters over which menu to use for each task and allow the user to use whichever panel is preferable to them As stated previously allow users to drag and drop a taxon within the Taxonomic Tree panel and then create a pop up box prompting the user to choose the relationship type for example Ts this a synonym gt Yes gt No for ICBN users after choosing Yes they are then prompted to designate the synonym type or Ts this taxon a synonym or a child Comparison to Alpha Version 0 24 The issues and suggestions still apply 24 EDIT C5 131 Reliability and usability testing report 4 4 10 Test Case 9 Checking and editing if necessary the parsed Rank of the Genus taxon Out a possible 133 there were 6 deviations 5 during Test Case 9 The deviations occurred at 2 of the 7 steps Four testers consulted the manual at two steps 3 and 11 testers required assistance 8 at three steps Five testers did not find the test case easy As this was the testers first experience with the Properties Sheet the deviations and occasions when the testers needed assistance relate to the finding of the correct field within the Properties panel and the activation of the taxon in the Properties panel One tester had difficulty activating the taxon due to only clicking the taxon once in the Taxonomic Tree panel two testers expressed the opinion that one click should be adequate Two testers activated the taxon by using the right click me
61. the File menu to allow users to reopen the taxonomic tree panel However this may not be desirable as the taxonomic tree panel is not in fact a file One option would be to remove the ability to close the taxonomic tree panel The remaining panels could be opened and reopened using a View menu which would be more intuitively linked to such operations than the Window menu Additionally it may be better to have the Preferences menu as a header menu rather than embedded in the Window menu so users are more aware of their ability to make individual adjustments Also not all of the options in the Preferences menu are related to the window or the appearance of the TE for example keys security and updates so to have the Preferences menu embedded in the Window menu may be misleading The Bulk Editor menu could be embedded in the Edit menu for a simpler interface it is likely to cause confusion for people who want to edit but do not know which menu to use This would give change to menus on the toolbar from File Edit Bulk Editor Window Help to File Edit Login View Preferences Help In the absence of an alternative menu to place this in it is also perhaps better to have this function on the toolbar The Window menu where it is currently found is not an obvious place for it to be The fact that users can create a new taxonomic tree via the File gt New menu reinforces the view of the testers that the taxonomic tree is an actual
62. to be clearly explained to the user If it is not possible to clearly define the difference thought should be given to amalgamating the Properties panel and the Descriptive tab This is could be done for example by creating multiple tabs in the Properties panel each tab for a group of clearly defined and related descriptive elements Users should have the ability to remove hide descriptive elements which they do not use via the preferences menu Further usability issues relating to individual test cases can be found in the main body of the text Several functional gaps were identified during the taxonomic process Testers expressed concern over these gaps and stated that their omission would make them reluctant to start using the Taxonomic Editor The most important gaps are listed below Import of data from external data sets and other resources such as IPNI and GBIF Export of data to produce a hard copy publication Inability to deal with taxonomic concepts such as the maintenance of links between taxa when synonymous relationships are changed Some testers seemed to want the Editor to track any changes made to relationships and amend the nomenclature accordingly For example when changing a homotypic synonym to an accepted name the original EDIT C5 131 Reliability and usability testing report accepted name should automatically be changed to a homotypic synonym of the newly accepted name e Inability to deal with different taxonom
63. ve amount of developer time to further developing interface pathways that are rarely used EDIT C5 131 Reliability and usability testing report Generalised documentation of standard interface control behaviour would be beneficial in allowing the users to optimise the efficiency of their interaction with the interface and to ensure the consistent application of a design pattern during interface development 4 4 Test Cases Each test case is summarised in the following section Full data can be found in Appendix 1 3 Each summary describes the usability and reliability issues encountered according to the assimilated data from all deviations and comments from the testers Where bugs or functionality issues were encountered these are described Where test cases contain multi route steps where there are multiple ways of performing a step figures are given for how many testers used each step 4 4 1 Summary Data A summary table Table 1 below gives the number and percentage of deviations manual consultations and assistance given as well as summaries of the close ended completion questions This gives an easy overview of which test cases or tasks have been most difficult for the testers to complete It should be noted that deviations do not necessarily equal problems in some cases testers discovered an alternative but equally valid method of completing a step that was not included in the pathway given In general however the number of deviat

Download Pdf Manuals

image

Related Search

Related Contents

  User Manual    Preparación de muestras de ARN monocatenario TruSeq    deutsch - primo  Leatherman Serac S2 User's Manual  Console Colour Spider-Man  Wiley Digital Photography Simplified, 2nd Edition  Samsung Monitor  

Copyright © All rights reserved.
Failed to retrieve file