Usability Report ()
Contents
1. O Yes O No 7 Does your hobbies include any of the following O Photography Model making O Others Thank you for your participation We hope to get in touch with you soon 11 Appendix B3 UNIVERSITY OF WASHINGTON CONSENT FORM Usability Testing of Hitachi Camcorder Investigators William Washington graduate student of TC department 206 3529093 Chii Chang Gau graduate student of TC department 206 5272446 Regina Yap graduate students of TC department 425 6880946 We are asking you to be in a research study The purpose of this consent form is to give you the information you will need to help you decide whether or not to be in the study Please read the form carefully You may ask questions about the purpose of the research what we would ask you to do the possible risks and benefits you have been answered you can decide if you want to be in the study or not This process is called informed consent PURPOSE AND BENEFITS The purpose of this test is to assess the usability of the Hitachi camcoder You will not personally benefit Data gathered in this test will help to clarify users needs and issues and may lead to improved product design PROCEDURES Participants in these studies will be asked to do one or more of the following do a set of tasks think out loud while doing these tasks or while filling out a questionnaire fill out an opinion questionnaire and or take part in an interview Performance data like2. DISP button the month digit did not start flashing as it should according to the manual This 1s because both participants go through at least 1 cycle of pressing the DATE DISP button The first cycle is for trying out all the options available when the DATE DISP button is pressed Not until the second cycle were they willing to try how to set the date and time But the month digit only flashes in the first cycle and stops flashing during the second cycle Third both participants are having such a hard time setting the date and time Another problem is the buttons for adjusting the date and time is not intuitive enough the manufacturer uses the REW and FWD buttons to make changes to the date and time Instruction manual The participants encountered lots of problem even when they refer to the manual this is very obvious in Scenario 2 when they are requested to set the date and time for the camcorder Recommendations 1 For intuitiveness it will be better to place the VIDEO OFF CAM switch at the back half of the camcorder where the users can easily locate At the same time the top buttons should be shifted to the back or to the side If it were shifted to the right side where it is accessible by the right hand when it is wearing the strap it would allow the users to use one hand to operate the camcorder This seems like what the two participants are expecting as they commented that it is a stretch to reach those buttons with their right han
3. Disagree 1 2 3 4 5 6 7 7 I would strongly recommend my friends to buy this camcorder Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 8 Explain why you would or wouldn t recommend your friend buying the camcorder 17 Appendix C Briefer s Log Appendix C1 Log A01 REC Questions Answer 1 How did you feel holding the camera with one hand or We noticed that you were holding the camera like this tell us what that was about We noticed that you first placed your thumb tells us more about that Where did you expect to find the On Off button When you are turning OFF the camcorder did you notice that you have push the button to the VIDEO position instead of CAM or How did you find the CAM VIDEO OFF button Is it easy to operate in terms of the location and the sliding action When you first wanted to look at the image produced by the camera were you aware that there is a LCD viewer Or We noticed that you didn t use the LCD panel until tell us more about that 19 Questions Answer 6 Where did you expect to find the EJECT button 7 Did the record button work as you expected Check OFF or VIDEO 20 Appendix C2 Log A02 TRI Questions Answers 1 What did you think T and W correspond to 2 We noticed that you zoomed out before zooming in tell us more about tha
4. be awkward for the hand to turn the camcorder around for the users to select the switch Third if they are holding the camcorder in a way that the front of it is facing them again it would be inconvenient for them to turn it on and then flip to the back and place the hand in the strap unless they are turning it off b Difficulties in understanding that CAM and VIDEO will turn the camcorder on Both the participants did not have too many problems switching to CAM and VIDEO mode after they found the button Even though they do not understand the meaning of the terms used it did not seems to affect them Do not understand the specific purpose of the LCD panel and the viewfinder The participants pulled out both the LCD panel and viewfinder One commented that If they the manufacturer have an eye piece here but you are looking through your monitor why do you need your eye piece That seems a bit redundant to me This happened because once the LCD panel is opened when the camcorder is in CAM mode the image will stop appearing on the viewfinder And the participants never try to pull up the eyepiece on the viewfinder Loading the cassette tape a Misinterpretation of labels Both participants push against the deck to try to get the deck open up This is because they found the word PUSH on the cassette deck and they thought it means push to open or something to that effect For example Participant 1 found the place to load the tape but 1t
5. taken care of C Pitcher is filled A Pitcher and glass are placed on the table C The box with the Hitachi Camcorder is placed on the table DA blank cassette for the camcorder is beside the box A The materials for participants are on the table C The ink pen on the table beside the materials is usable C The test picture is on the wall table A Tripod has been set up A Timer is in your pocket Have a pad and pens for taking notes O Do Not Disturb sign is hung on the door A Make sure that you know the participant s name O Victor is ready Regina is ready When the participants arrived C Greet the participant Hi are you how are you doing My name is William I will be your test administrator today This is Victor and Regina They will be observing the test in another room C Show the participant where to sit Victor and Regina exit A Make the participant comfortable Offer water 36 Introduction Thank you for helping us with this project today We ll take a few minutes before we begin to talk about our procedures and give you a chance to ask questions Ill be reading from this booklet most of the time just to make sure explain everything the same way to each of our participants If I don t cover something you have questions about please feel free to ask First the amenities We ve scheduled a couple of breaks during our session today If you need to take a break some other time jus
6. Introduction This report presents the research design findings and recommendations for the Hitachi Camcorder Pilot Usability Testing This study was a course project for TC517 Usability Testing and was carried out by Chii Chang Gau Regina Yap and William Washington graduate students in the Technical Communication department Purpose As this product has already made available in the market the purpose of this study is to assess how easily can the inexperience users use the camcorder to perform basic tasks such as recording playing back and setting the date and time without training By asking the participants to perform some of these basic task in a laboratory settings we hope to capture the usability issues associate with this camcorder Participant Profile and Recruiting The intended audience of the Hitachi Camcorder is adult members 18 years and above of the American general public We are informed that this camcorder that we are testing is designed for the American market We have excluded children from our intended audience because they are not likely to purchase or use a camcorder As the focus of our usability study is to test the general operability of the camcorder By finding participants which exhibit little or no technical characteristics in occupation or hobbies and no experience with a camcorder we feel that we will be able to better test the functionality of the Hitachi camcorder Please refer to Appendix B for th
7. Say Thank you for your interest but we re focusing on people who has no experience with using camcorder before May I keep your name and phone number for future tests If person says Yes write keep on form Thank again and end the conversion No Go to Question 2 2 Are you a citizen or resident of America Yes ___ Goto Question 3 No ___ Say Thank you for your interest but we re focusing on people who are a citizen or a resident of America May I keep your name and phone number for future tests If person says Yes write keep on form Thank again and end the conversion 3 Are you above 18 years old Yes ___ Goto Question 4 No ___ Say Thank you for your interest but we re focusing on people who are above 18 May I keep your name and phone number for future tests If person says Yes write keep on form Thank again and end the conversion 4 Do you have any experience in film production Yes ___ Say Thank you for your interest but we re focusing on people who do not have any film production May I keep your name and phone number for future tests If person says Yes write keep on form Thank again and end the conversion No Go to Question 5 5 Do you like to fix repair things such as cars home repairs computers Yes ___ Goto Question 6 No _ Goto Question 7 6 What are they Mechanically related Yes ___ Say Thank you for your inter
8. ce the photographs calendar in the designated place in the test room Fill up the pitcher and place it the cup napkins and plates of biscuits on the table Hang the Do Not Disturb sign on the door knob 10 minutes before the test O O O O Test out the equipment make sure that you know how to use them Make sure that the time date feature is turned on on your monitor Make sure that the Data Recorder 1 log is in front of you Make sure pen is on table If all boxes are checked signal the administrator that you are ready When the participant signal that he she is ready O O Make sure that the camera is rolling Make sure that the audio equipment is recording 40 Appendix F4 Data Recorder 2 Guide C00 One day before the test O Get the Usability Testing training tape from Judy O Confirm with Judy that the TV and VCR will be in the lab O Borrow the tripod from Judy O Prepare a timer O Prepare a bell 30 minutes before the test QU aaa 000000 Hand the timer to the Briefer Place the bell on the participant desk Set up the tripod at its designated corner Put the training tape in the VCR Test play the training tape Make sure that the TV is set at the channel where you press PLAY on the VCR the image will appear on the TV Make sure that the training tape is rewind to the beginning Leave the VCR at ON position Test out the camera by recording once with someone in the room talking Ad
9. d thus they have to use their left to do it But by using their left one commented that it would make the camcorder shaky and the other mentioned that it block their view of the LCD panel Another option is to place it at the back of the camcorder so even though the users have to operate the buttons with the left hand the left hand will not be hindering users from looking at the LCD It will instead help him her to steady the camcorder by having the left hand supporting the camcorder from under and the left thumb controlling the buttons One participant did mentioned that while using the REW and FWD button to change the date setting it causes the camcorder to become shaky Users are not very certain that the CAM and VIDEO mode will turn the power on It may be better to have ON OFF buttons instead of CAM OFF VIDEO buttons if the buttons related to the respective functions do not overlap Instead of EJECT the button should be labeled OPEN for easy recognition The manufacturer can either remove the word PUSH or change it to Push to close And it would be better if both buttons are arranged on the same side and close to each other instead of diagonally apart This would make it easier for users to locate the buttons Also there should be a small icon on the deck to show where to load the tape an icon of the tape and an arrow will suffice During CAM mode users may want to have both the viewfinder and the LCD panel active It does not make
10. e Can locate Yes LCD already opened EJECT No Load tape attemps Close deck problem RECORD Push release Push hold circle one attempts Stop REC problem OFF attempts Comments 31 Appendix E2 Log C02 TRI Task PROB Remarks e D ES 5 E 2 S FA F Mount cam problem Locate switch attempts CAM mode attempts Viewfinder Yes Adjust focus No Open LCD Pull Release Can locate Yes LCD already opened EJECT No Locate In demo mode Yes No DATE DIS If consult manual further problem Yes No attempts If Yes what Set date Success Failed RECORD Push release Push hold circle one attempts Locate zoom lever attempts Use the LCD zooming display to gauge Yes No Comment 32 Appendix E3 Log C03 PLAY Task Remarks Locate switch attempts VIDEO mode Correct Discover mistake and switch to VIDEO in min sec They know what the problem is if they recover within 2 min Wrong Rewind tape problem Play tape and view problem Remove tape problem OFF Comment 33 Appendix F Administrators Scripts Appendix F1 Roles of Investigators during the test Briefer e Participants Screening o ee William Interacts with the participant e Make the participants feel welcome A series docume
11. e screening questionnaires Research Problems and Methods The broad concern of this test is Can users easily perform the basic tasks they want to without any prior training The basic tasks are recording setting date and time and playback In this study we have designed three scenarios to trap whatever usability issues that may surface when users tried to perform these tasks The possible problem areas are identified as below Holding the camcorder Turning on off the Camcorder Using the LCD panel and the viewfinder Loading cassette tape Recording Adjusting images with zoom and focus Playback Setting date and time For our research methods and test flow chart please refer to Appendix G Findings This pilot test produced a great amount of data that illustrate how users work with the camcorder to perform some basic tasks such as setting date and time stamp recording and playing back The results have been divided into the following categories 1 Getting started Both participants inspected the camcorder to look for visual cues to hint them where they should start first The feel of the camcorder Both participants mentioned that they like the size of the camcorder One commented that it was a nice size to hold and the other said that This is a lovely little size However during a debrief one participant did mentioned that she feel insecure holding the camcorder because the strap was too loose Other than this
12. e thought that that is a PAUSE or STOP button that stops the recording She did not know that that button act as both a RECORD and STOP RECORD button Also participant 1 do not know that he do not have to press and hold that button to record This is evident from his comment T like the fact that when you take your thumb of the RECORD button it pauses recording Playback The location of the buttons for playing back is very inconvenient One participant mentioned that it is hard to reach over to use the RECORD button it blocks the LCD button The other mentioned these when she was trying to use the rewind button That the REW button actually is too hard for me to reach I need two hands to get there it is a bit of a stretch and There is no point reaching it with my right hand I have to use my left Date time settings Users can locate the DATE DISP button easily But there are a few problems First one participant thought that the up and down arrows under the FOCUS VOL is for adjusting the date and time but they are not The participant proceed to look all over the camcorder twice to try to find a button that will tell him it is for adjusting time but none whatsoever As a last resort he turned to the manual for help Despite that he still encountered many problems with setting the date For the other participant she failed to change the time at all after numerous tries Second and a major one when they press the DATE
13. eading the whole manual Administrator s training The participants always had a lot of unexpected action while performing their tasks Sometimes they would be distracted from what they should do to complete the tasks and instead just do what they want to do It is important for the administrator to know how to guide the participant back to the question by the least reminder It may need a lot of practice for the administrator to guide the participant to think out loud and follow the instructions in the scenarios Unanticipated action from the participants During the test if not stated in advance participants might stand up and walk around This action will make our camera lost track of where they are and what they are doing We should state where should our participants be in our scenario and ask them to follow the instructions in the scenario Post test questionnaire While doing the post test questionnaire one of us did both the questioning and the writing This made 1t difficult to keep up with all of the comments It might be preferable if these two roles were given to different people one an interviewer and the other a note taker Time restriction It is important in our case to set a maximum time limit for each test This would help the participant to be more focus on their tasks Appendix A Scenario Descriptions Appendix A Scenario 1 You have just purchased a new Hitachi Camcorder Having a loved ones and family in a d
14. eral days before the pilot test we feel that we should have done more to be prepared for what is coming We should have invited another person to join us in the dress rehearsal so that we can have a pre pilot test test This will help us to be more prepared of what is going to happen and at the same time test out the log that we have designed Participants recruiting Because our participants are general public we recruited participants as broad as possible We chose the first two candidates who match with the criteria listed in our screening form During the test we found one of our participants heavily relied on our instructions to think out loud and to know what she should do If we started recruiting participants one or two weeks earlier we could probably have a chance to test participants ability in thinking out loud and screen out participants who are better fit our participants profile Participants may need to refer to user manual During our usability test one of our participants started her task by reading user manual One reason that she did so was that she always think high tech product is fragile if you don t first read the manual before you use it you would surely damage the product We should have anticipated this and stated in our scenario whether they can use the manual to perform the task or not If they really did need to use the manual then we should control the time or we will sit there waiting for he she to complete r
15. esponses to the usability problem we found to see if these problems really matter a lot PARTICIPANT SAMPLING 6 7 When conducting studies or usability tests there is always an issue concerning how many and what kinds of people to involve Designers always have the option of including all the people in the group i e include the entire user population Unless it is very small group this approach is usually not practical Therefore we need to sample from the population To make strong inferences the sample should be truly representative of all possible users Our problem now is that we only need to sample two participants from the general public How could these two people truly represent the whole general public As Dumas J mentioned the consensus I hear is that 1 you can learn a lot about the usability of a product with a few test participants and 2 you should avoid having to use statistical tests of significance whenever you can The only thing one must bear in mind is that the participants you choose for a usability test have to be members of the target market for the product you are testing 7 The two most commonly used types of sampling in studies and usability tests are random sampling and subjective judgmental sampling In our case we believe subjective sampling is acceptable That is we choose particular people whom we feel best represent the population to do our test USABILITY TEST 1 6 Based on the in
16. est but we re focusing on people who are inexperience with using tools May keep your name and phone number for future tests If person says Yes write keep on form Thank again and end the conversion No _ Goto Question 7 Does your hobbies include any of the following Photography If yes go to Thanks anyway at bottom of page model making If yes go to Thanks anyway at bottom of page Others WHAT If there is anything that suggests that they are inquisitive dexterity go to Thanks anyway at bottom of page Else RECRUIT Inexperience Thanks anyway Thank you for your interest but we re focusing on users who are not technically or mechanically inclined May I keep your name and phone number for future tests If person says Yes write Keep on form Thank again and end the conversation 10 Appendix B2 Screening Questionnaire Paper Electronic 502 Thank you for your interest I d like to ask a few questions to see if you meet the profile of the people we are including in this particular test Name Contact number 1 Have you used a camcorder before O Yes O No 2 Are you a citizen or resident of America O Yes O No 3 Are you above 18 years old O Yes O No 4 Do you have any experience in film production O Yes O No 5 Do you like to fix repair things such as cars home repairs computers O Yes O No 6 What are they Mechanically related
17. est Logs and Questionnaires Collected
18. formation we get from our heuristic evaluation we develop scenarios and tasks Then we use scenarios to tell participants what we want them to do during the test Scenarios describe the tasks in a way that takes some of the artificiality out of the test and contain enough information for participant to do the task 43 During the test participants will be asked to talk think aloud while performing tasks We will take notes on their comments unsuccessful attempts to perform certain tasks number of times that a document is used number of help requests and the like Questionnaires can be administered to potential participants to help determine their attitudes concerning a new product application or system These questionnaires can be given to participants prior during or after any contact that they may have with a new system Bailey R W 1992 categorized usability tests into four types formal evaluations inspection evaluations performance tests and operational evaluations 5 Our research approach is similar to his inspection evaluations PILOT TEST 1 Pilot test is nothing more than a dry run of our usability test or dress rehearsal for the usability test to follow The most important objective of a pilot test is to debug the equipment software materials and procedures you will use for the usability test A secondary objective is to give the test team a chance to practice the activities they will be using during t
19. having many local problems is likely to make the user feel that the designers were sloppy in their development effort As Jeffries and Desurvire 1992 note the best evaluation of a user interface comes from applying multiple evaluation techniques In our research we conduct usability tests and heuristic evaluations to take advantage of each method s strength 1 HEURISTIC EVALUATION 2 3 4 5 Nielsen J 1992 defined heuristic evaluation as follows Heuristic evaluation is a method for finding usability problems in a user interface design by having a small set of evaluators examine the interface and judge its compliance with recognized usability principles the heuristics Independent research has found heuristic evaluation to be extremely cost efficient confirming its value in circumstances where limited time or budgetary resources are available The goal of heuristic evaluation is the finding of usability problems in an existing design such that they can be fixed One could thus view it as a debugging method for user interfaces 2 Before our scenarios and tasks design we first define who our users are and what their goal of using this product is Based on the definition we examine our product and look for problems that violate general principles of good user interface design We note down all usability problems we encountered and design our task and scenario based on these problems Therefore we can test users r
20. he usability test 1 We follow all the procedures we will use in the usability test and collect the same data we will collect during the usability test CONCLUSION We conclude our usability testing approach as follows Define audience and basic tasks broad concerns component issues concrete questions Heuristic evaluation identify usability problems debugging gt Task scenario design Yes subjective sampling 4 N Usability test gt Participants random sampling verbal data collection observational evaluation N J questionnaire REFERENCE 1 Dumas J S Redish J C A Practical Guide to Usability Testing Intellect Ltd 1999 2 Nielsen J Finding Usability Problems through Heuristic Evaluation ACM 1992 pp 373 380 44 Jeffries R Wharton C Uyeda K M User Interface Evaluation in the Real World A Comparison of Four Techniques ACM 1991 pp 119 124 Doubleday A Ryan M Springett M Sutcliffe A A comparison of Usability Techniques for Evaluating Design ACM 1997 pp 101 110 Jeffries R Desurvire H Usability Testing vs Heuristic Evaluation Was There a Contest SIGCHI Bulletin 1992 vol 24 4 39 41 6 Bailey R W Human Performance Engineering Prentice Hall 1996 7 Dumas J How Many Participants in a Usability Test Are Enough Common ground pp 2 6 45 Appendix H T
21. ifferent part of the country you decide to make them video of your apartment and hope to send it off before the mail goes out in a few minutes Task 1 e Use any of the materials provided including the cassette tape to make a short 1 minute video documentary of your room including some of your vacation photos you have displayed Scenario 2 After recording you decide that your hand was much too shaky to see the details of the photographs you recorded This time you decide to use the tripod to record the details of the photographs In addition you decide that you would like for your family and loved ones to know the day you made the video You decide to stamp the date on the video before you finish recording Task 2 e Use the tripod in its fixed location to film the details of the photograph by framing the photograph to fill the entire display e Make today s date show up on the video before you finish recording Scenario 3 Before sending your video to your family and loved ones you want to review your recorded material Task 3 e Rewind the tape and review what you have just recorded e Take out the cassette Appendix B Forms Appendix B1 Telephone Screening Worksheet 501 Person s name Phone number Thank you for your interest Id like to ask a few questions to see if you meet the profile of the people we are including in this particular test 1 Have you used a camcorder before Yes ___
22. ike to ask you a few questions and ask you to fill up a questionnaire for us Debrief Questionnaires Thanks We ll take a break for five minutes to let us catch up with our notes and to let you relax a bit Scenario 3 playback OK for the last session turn to the next page that reads Scenario 3 and continue as before Exit When the participant press the bell to signal you ENTER 38 Thank you Before we end this session would like to ask you a few questions Debrief Questionnaires PostTest Questionnaires That s all we have for you today Thank you so much for your time We ve really appreciated your help Here is a small token for attending today s session Thank you again 39 Appendix F3 Data Recorder 1 Guide B00 On that day of the test 0000000 Prepare all the materials required in the test and separate them into different folders Prepare a pitcher cups biscuits plates and napkins Bring 6 pens and pencils Bring photographs or calendar Bring thumbtack if bringing wall calendar Bring along an extension cord just in case the battery run out of power Prepare a Do Not Disturb sign 30 minutes before the test a aa 0 Distribute the materials to the respective investigators and put one on the participant s table Make sure that Briefer Data Recorder 1 Data Recorder 2 and Participants table all have a pen And check if all the pens are usable Hang or pla
23. just the volume so that whatever recorded is audible Make sure that the image appears alright on the two monitor Make sure that you know how to adjust the focus of the camcorder Make sure that the camcorder is filming the right place Ask a person to test out by pretending to be the participant doing the test Make sure that the camera can capture the action we want 10 minutes before the test O O O O Test out the equipment make sure that you know how to use them Make sure that the time date feature is turned on on your monitor Make sure that the Data Recorder 1 log is in front of you Make sure pen is on table If all above boxes are checked signal the administrator that you are ready When the participant signal that he she is ready O O Make sure that the camera is rolling Make sure that the audio equipment is recording 41 Appendix H Test Logs and Questionnaires Appendix G USABILITY TESTING PROCEDURE OVERVIEW 1 Among all usability evaluation methods usability tests have shown advantages in identifying usability problems and global problems but they uncover fewer local problems and consume more time than other usability evaluation methods If you only conduct usability test however you run the risk of having many undetected local problems in your product While any one of these problems is probably not serious enough to keep users from completing tasks or frustrating them the combined effect of
24. nd above 2 Whatis your occupation 0000000 Student Writer Teacher Business Executive Administration staff Retiree Others 3 What kind of tasks do you do Meeting and talking to people Using the computer for typing tasks Troubleshooting hardware Performing mechanical task Others 4 What do enjoy doing during your free time 00000000 Home improvement Auto repair and maintenance Model making Repairing electronics Internet surfing Reading Strolling Others 5 You will buy a product if O O O You like the look of the product You find it easy to operate after you have tested the product There are many features on the camcorder 13 Appendix B5 Post Task Questionnaire REC P03 1 I felt that the strap was Uncomfortable Comfortable 1 2 3 4 5 6 7 2 Once found using the On Off switch was Inconvenient Convenient 1 2 3 4 5 6 7 3 Overall using the On Off switch was Difficult Not Difficult 1 2 3 4 5 6 7 4 Opening the LCD viewer was Difficult Not Difficult 1 2 3 4 5 6 7 5 Finding the EJECT button was Difficult Not Difficult 1 2 3 4 5 6 7 6 Loading the Cassette was Inconvenient Convenient 1 2 3 4 5 6 7 7 Finding the record button was Difficult Not Difficult 1 2 3 4 5 6 7 8 I felt that finding the On Off switch took Too Long Neutral Too Short 1 2 3 4 5 6 7 14 Appendix B6 Post Task Questionnaire TRI P04 1 Setting
25. nts Briefing may involve asking the participants about themselves showing them the test room informing them about the test situation answering their questions and conducting training e Debriefing participants after each scenario e Observe the participants and prepare the debriefing materials Data Recorder 1 e Get ready the material and make sure that each investigator has a copy of the Regina materials Brief the investigators how to use the materials e Act as Help Desk Operator during test sessions B series documents e Collects the data of participant s behavior and records quotes of participants during tasks using the Data Recorder 1 log Bxx Data Recorder 2 e Get ready the AV equipments o Victor Set up the test room e Operates the AV equipments during test to record the test sessions C series documents e p e Collects the data of the actions participants take on the camcorder using the Data Recorder 2 log Cxx Location of Investigators and participants during the test IG o jy Usability Lab Administrator amp Participant 35 Appendix F2 Briefer Guide A00 On that day of test O Check that the Hitachi camcorder power cord manual and all things else is in the box Remember to bring the cassette tape and make sure that it is rewind to the beginning A Make sure that the battery is fully charged 15 minutes before participant comes Make sure each of the following is
26. number of errors or time to complete a task may be collected also opinion data may be collected and the participants performance may be videotaped RISKS STRESS OR DISCOMFORT This study will not expose its participants to risk stress or discomfort beyond that normally encountered in using a camcorder being videotaped or taking a test OTHER INFORMATION Once the results of these studies have been tabulated the names of individual participants will be destroyed in order to ensure both anonymity and confidentiality No one other than the investigators named above will be informed of or have access to data on the performance of individuals The investigators will retain the data for no more than one year following the date on which the study is administered You are free to refuse to participate in the study and may withdraw at any time without penalty Signature of investigator Date This study has been explained to me volunteer to take part in this research have had a chance to ask questions If have questions later on about the research can ask one of the investigators listed above If have questions about my rights as a research participant can call the Human Subjects Division at 206 543 0098 will receive a copy of this consent form Signature of Participant Date 12 Appendix B4 Pretest Questionnaire Participant Date 1 What is your age O O O O 18 24 25 34 35 44 45 60 61 a
27. sense when the LCD is opened image goes off in the viewfinder unless the viewfinder is not design for recording but viewing That will explain why during playback users can look at both the viewfinder and the LCD for images There should be more visual cues on the camcorder One participant did mention that it would be better to have cues on the display to prompt her what to do next For the date and time setting it would be helpful to have the buttons that can change the settings close to the DATE DISP button Instead of using the REW and FWD button to adjust perhaps the up and down arrows for the FOCUS VOL can double as the adjustment buttons for date and time settings Maybe the designers can include more screen prompts or options to facilitate date and time settings Things that could be improved in our Testing Process 1 The design observers logs It is difficult for us to track what participants would do next No matter how much consideration that we put in designing our log there bound to be some exceptions which is beyond what we expected To solve this problem it is recommended to record the whole test and make sure that your camera is positioned correctly so that the important data will not be missed The recorded session allows you to go back and capture that you did not miss out any critical steps that the participant took Complete dress rehearsal A formal dress rehearsal is important Although we did a dress rehearsal sev
28. t 3 Where did you expect the Date Display button to be 21 Questions Answers 22 Appendix C3 Log A03 PLAY Questions Answers 1 We noticed that you didn t use the LCD panel Viewfinder until tell us more about that 2 What did you think CAM referred to 3 What did you think VIDEO referred to 4 When you needed to playback your recording was it clear that you needed to go to VIDEO 5 We noticed that when you turned off the camera you first went to VIDEO tell us more about that 23 Questions Answers 24 Appendix D Data Recorder 1 s Log Appendix D1 Log B01 REC Task Expressions gestures other observations Check OFF or VIDEO 26 Appendix D2 Log B02 TRI Task Expressions gestures other observations 27 Appendix D3 Log B03 PLAY Task Expressions gestures other observations 28 Appendix E Data Recorder 2 s Log Appendix El Log C01 REC Task Remarks Hold of flexing Locate switch Back Top Sides Front CAM mode CAM VIDE O Viewfinder Yes Adjust f just focus No Open LCD Pull Relea s
29. t let me know The washroom is We ve put a pitcher and glasses on the table take a drink whenever you feel like it Point out the pitcher The purpose of today s session is to study the usability of this Hitachi camcorder To do this we ll ask you to perform a number of tasks users might perform with a camcorder As you are performing the tasks Victor and Regina will be watching the test from these video cameras point out where the cameras are and take notes The test will be recorded so we can refer back to it later For added confidentiality we ll erase the tape in two months time Before we begin we have a few forms for you to fill out Show them the forms If you have any questions while you are filling out the form please feel free to ask me When you re done with the forms let me know Hand participant the forms and pen When participants are done collect the form In this test we are interested in what are thinking as you go through the tasks that we give you In order to do this we will need you to verbalize your thought by thinking out loud as you work on these tasks Simply act as 1f you are alone in the room speaking to yourself We have prepared a video tape to show you how this is done Go over to the TV and switch it on Then press start on the VCR End of video tape Switch off 37 Ok do you have any questions regarding the test Tf yes answer questions Now let me quickly go over our test The whole
30. test is broken into three sessions The folder on the table will explain to you what you should do in each of these sessions Just relax and perform the tasks describe on these scenarios Remember that throughout all these sessions we re studying the system not you If you don t like a feature or if you have problems with it chances are lots of other people will too That s what we re here to find out Go ahead and open to the page that says Scenario and read the instructions there I will be in the next room Press this bell gesture to the bell to let me know when you are ready to begin When you need help you can call this number on the table the operator will help you And when you have completed the task press the bell to signal to me I will leave you here for now Exit When the participant press the bell to signal you ENTER Thanks Before we go for a break would like to ask you a few questions and ask you to fill up a questionnaire for us Debrief Questionnaires Thank you We ll take a 5 minute break now If you need to use the washroom it s Exit Scenario 2 setting up the camcoder on the tripod OK it s about time to get started again Go ahead and open to the next page that reads Scenario 2 Read the instructions there and press the bell to let me know when you are ready to begin Exit When the participant press the bell to signal end of task ENTER Thanks Before we go for a break would l
31. they seems to get use to holding a camcorder quite fast VIDEO OFF CAM switch a Difficulties in locating the switch The participants have problem finding the power on switch Participant 1 thought that the RED button is the power switch and he commended that This power button is nice it is right under your thumb right for your thumb to find Then he discovered that it doesn t work and attempted to look at the manual for direction to look for the power button When he finally found the switch at the front of the camcorder he mentioned that I would thought they would have place it at the back where the eyepiece is Participant 2 also faces difficulties when she is trying to reach to the front and switch the mode to CAM while the camcorder is perching on a tripod After a few tries she gave up and turn the camcorder with the front facing her and move the switch to CAM position From what we have observed it seems like both the participants hold the camcorder in the filming position before they start looking for the power switch And in that position they will expect to find the power switch at the back of the camcorder perhaps somewhere near the thumb If to get ready the camcorder before turning on the power switch is what they do it would be inconvenient for them to locate the power switch in front First the hand will be in an awkward position if the users are trying to reach from behind Second if the hand is in the strap it will
32. took him a while to figure out the EJECT button means to open the deck and he commented that maybe should have said OPEN Also during the debriefing he mentioned that he noticed the PUSH on the deck and assume that is to push open the deck Participant 2 was looking at the manual on how to load the cassette tape she opened the LCD panel and then was pushing against the side of the cassette deck at the word PUSH to try to open the cassette deck From their behavior we know that to have the word PUSH on the cassette deck is very confusing for the participants if this is their first time using this camcorder b Loading the cassette tape When it comes to loading the cassette tape participant 2 had a lot of difficulties doing it She tried unsuccessfully to insert the tape from the bottom of the deck after it is opened She consulted the manual and figure that it is from the top she still has problem to correctly insert the tape into the deck For participant 1 do not have too much trouble doing it There are no indications on the camcorder that suggest how a cassette should be loaded Will users know instinctively how to load the cassette tape RECORD button Participants have a hard time realizing that that is a RECORD button because there is no label on that button One participant thought that that is a power button because RED means power in many of the remote control The other participants did not know that is a RECORD button too Sh
33. up the camera onto a tripod was Difficult 1 2 3 4 5 6 2 The location of the Viewfinder was Inconvenient 1 2 3 4 5 6 3 Focusing the image in the Viewfinder was Difficult 1 2 3 4 9 6 4 Finding the Zoom was Difficult 1 2 3 4 5 6 5 Using the Zoom was Difficult 1 2 3 4 5 6 6 Finding the Date Display was Difficult 1 2 3 4 5 6 7 I felt that setting the date took Too Long Neutral Too Short 1 2 3 4 5 6 Not Difficult 7 Convenient 7 Not Difficult 7 Not Difficult 7 Not Difficult 7 Not Difficult 7 7 15 Appendix B7 Post Task Questionnaire 3 Playback 1 Initiating PLAYBACK was Difficult 1 2 3 4 5 6 2 Opening the LCD viewer was Difficult 1 2 3 4 5 6 3 Unloading the Cassette was Difficult 1 2 3 4 5 6 Not Difficult Not Difficult Not Difficult 16 Appendix B8 PostTest Questionnaire Participant Date 1 Learning how to use the camcorder was easy Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 2 Using the camcorder was easy Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 3 The design of the camcorder is pleasing Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 4 The terms used on the camcorder are easy to understand Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 5 Recovering from errors was very easy Strongly Agree Strongly Disagree 1 2 3 4 5 6 7 6 Understanding the instructions in the manual was easy Strongly Agree Strongly
Download Pdf Manuals
Related Search