Design
Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing).
Implementation
Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface.
Evaluation
Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population (but don't identify your users by name). Describe how the users were briefed and what tasks they performed; if you did a demo for them as part of your briefing, justify that decision. List the usability problems you found, and discuss how you might solve them.
Briefing
Thank you for participating in our user interface test. Your participation will allow us to find problems with our interface and will help us to build a more user-friendly interface.
In the following mock scenario, you will be a student studying Mandarin Chinese. Your class uses the textbook "Learning Chinese: A Foundation Course for Mandarin Chinese", and you want to practice reading sentences that use the words found in this book. To do so, you're using Reading Practice, a web application that helps members practice reading in their language of study by providing a databse of sentences in that language to read.
In order to help us understand the user’s experience, we ask that you think aloud and ask about any uncertainties that you may have. Remember, this test is entirely voluntary, and you may stop at anytime.
Scenario Tasks
1. Create an account and log in.
2. Enable all vocabulary from Chapter 1 of your textbook, "Learning Chinese: A Foundation Course for Mandarin Chinese" to be displayed in sentences.
3. Find a sentence containing 好 (hao4), read it, and find out its English translation
4. This sentence contains 忙. Find out its pronunciation and definition, and find another sentence containing it. (In the event that the displayed sentence didn't contain this word, one of the words present in the currently displayed sentence was used for this task).
5. Prevent 我 (wo3) from appearing in subsequently fetched sentences
6. Make it such that the 2 sentences currently being displayed in the "Currently Reading" tab are no longer displayed.
7. Restore one of the sentences that you just removed from display back to the "Currently Reading" tab.
8. Find another sentence containing 好. Note that this was the previous study focus.
9. Now you want to return to a general review of all words you know, and aren't focused on studying a particular word. Switch to a general review of all words, and fetch 2 sentences for review.
10. Contribute a sentence.
Observations during User Testing
User 1:
User 2:
This user grew up in a Chinese-speaking environment, and has basic reading proficiency. He was able to read the sentences that he encountered while doing the tasks for this user test. However, after the user test, when he enabled the remainder of the vocabulary, there were words that he needed to click on to discover their pronunciation and meaning. Hence, the user could benefit from using this application, and thus matches the target audience.
Task 1
The user had no difficulty going to the Create Account tab, entering his username and password (switching between the fields using tab), and having the account created and logged in after pressing enter. The user initially typed the repeat of his password in incorrectly (apparently deliberately), and was pleased to see the immediate feedback that appeared that the entered passwords were different.
Task 2
The user had no difficulty locating and using the Textbook and Chapter selection combo-boxes on the left side. However, after the chapter was selected, the user wasn't entirely sure what to do. He first clicked on the "Displayed" sorting button (which sorts the words according to whether they are allowed to be displayed or not). Then he checked the "May appear in sentences" option for some individual words. At that point he believed he had accomplished the task, and started fetching sentences. After being informed that he hadn't actually accomplished the task, the user noticed the "check all" button and clicked it (which is how we had expected the task to be done).
The user later cited that part of his source of difficulty was that the "May appear in sentences" label was cut off (displaying only "May appear") because the user's browser window was not maximized, and was at the time smaller than the minimum size we had designed for. However, the larger usability issue in this case was that the "check all" button's location did not make it discoverable, or that its label did not carry sufficient information scent.
Task 3
The user first tried pressing Ctrl-F to search for the word using the browser's built-in search. In our case, however, because we had implemented the site using Silverlight not HTML, the browser can't search the page. After noticing that he wasn't getting when searching via the browser's built-in search, he quickly switched to the search box. The user didn't have any issues locating the search box or using it. After typing in the pinyin for the word, he found the word in the search results, but initially he checked the "May appear in words" checkbox, and didn't click the "make study focus" button. Only after fetching a few sentences and noticing that the desired word was not present did he focus his attention back to the left sidebar, and clicked the "make" study focus button. He clearly noticed the change this time, as he hovered his mouse around the (now highlighted in green) combobox displaying the word in the "study focus" section and faintly mumbled "ah that's it". He then pressed the "fetch next sentence" button to fetch the sentence, as expected.
Task 4
The user clicked on the word in the sentence to display its pronunciation and translation. He clicked the "make this word the study focus" as expected to make it the study focus, and then pressed the "fetch next sentence" button to fetch the next sentence.
Task 5
The user typed the pinyin in the search box to locate the word, and unchecked the "May appear in sentences" button. He then pressed the fetch next sentence button a few times to verify that the word indeed didn't show up to convince himself that he had actually accomplished the task.
Task 6
The user had no difficulty locating the close button for the button and clicking it.
Task 7
Immediately after accomplishing the previous task and before the user was shown this task, the user switched to the Closed Sentences tab and pressed the "Remove sentence" button repeatedly until all the sentences were removed. Thus, because there were no sentences to restore, the facilitator asked the user to first fetch a new sentence and redo the previous task before proceeding. After fetching the new sentence and clicking the Close button, the user switched to the Closed Sentences tab, and pressed the Restore Sentence button. He then switched back to the Currently Reading tab, saw the sentence he had just restored, and repeated the process of closing it and restoring it from the Closed Sentences tab to verify that that its behavior indeed matched what his mental model.
Task 8
The user went to the Study Focus sidebar, clicked the combo-box, and selected selected the word in the drop-down menu as expected.
user immediately discovered how to sort sentences according to "displayed"
took user a while to figure out to click the check all button
user first attempted to use the browser's built-in search Ctrl-F to search for finding a word, but quickly discovered it didn't work and switched to the search box
user checked "may appear in sentences" to display words, instead of study focus. However after not seeing the word in the fetched sentences, he noticed the make study focus button and used it
User 3:
Reflection
Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.