...
Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population (but don't identify your users by name). Describe how the users were briefed and what tasks they performed; if you did a demo for them as part of your briefing, justify that decision. List the usability problems you found, and discuss how you might solve them.
Because our scenarios and user tasks project mission did not change from when we did our paper prototype testing, we used the same briefing and tasks. The briefing is reproduced below:
Thank you for choosing to participate in the testing of our prototype. The goal of this test is to find deficiencies in our prototype so that we can improve the usability of our system. To be clear, you are not the subject of this test, and you should not be worried about making mistakes, as this will provide valuable feedback to us about our system. The protype you will be testing is intended to mimic the functionality of a website. The purpose of this website is to provide a quick and easy way to contact your local government officials. While not required, it would be helpful if you talk through your though process as you are carrying out the tasks that we will provide for you. Your participation in this test is entirely at your discretion, and you may stop testing at any time. |
We added two additional user tasks from the set used during the paper prototype testing. The complete set of tasks is now:
- Find the representatives for your area
- Call a representative now
- Record a voicemail for a different representative
- Schedule a call for a third representative
- Reschedule the call you just scheduled
We did not use a demo because we preferred to let the user learn the interface themselves. Our users were all MIT students who had no previous experience contacting policy makers.
Usability problems and severity
User 1
- The phone number entry field was not auto-focused on the call now page (mild)
- The user was confused about what happened after the call was placed on the call now page (moderate)
- The user did not have her phone with her, so she was not alerted by it ringing as would normally be the case
- The voicemail record button has a microphone icon while external consistency calls for a red circle (cosmetic)
- The now button on the time entry widget confused the user because she had already entered a future date, and now does not make sense in that regard (mild)
- The time entry widget is not intuitively mapped (mild)
- The header buttons are not obviously buttons (cosmetic)
User 2
User 3
Reflection
The iterative design process was useful, because it allowed us to make changes rapidly and then get feedback to double check that we had actually made an improvement with that change. If we were to do this again, we would probably investigate which technologies to use more carefully. For all of us this was our first time using css, javascript, or php, and while we got the work done, we had to spend more time learning the technology than we would have liked. However, now that we know the technology better, it would be useful to run more high fidelity tests, because we believe the feedback from testing phases that actually involved a computer (heuristic evaluation and final user testing) to be the most valuable and actionable. For example, even though our paper prototype had radio buttons on the filter tab, none of our test subjects commented on them, and this was likely due to the low fidelity of paper. Again, web prototyping is so quick, that we think it might have been more useful to test a wire-frame prototype a little earlier on.