...
This user grew up in a China for part of her life and moved to America when she was 12 years old. She said that she hadn't practiced reading in a long time but wanted to be able to review. Hence, the user could benefit from using this application, and thus matches the target audience.
Task 1
The user had no trouble creating an account. Although at log in, the user complained about waiting too long at the loading screen.
Task 2
The user selected the correct book and chapter. However, the user did not understand how to enable the words at first. At first she attempted to press the "Make study focus" buttons, but then realized that clicking multiple buttons undid the actions of previous button clicks. After realizing this, she then found the "Check All" button and used it.
Task 3
The user did not know how to complete this task alone. She did not know what the "Make study focus" button was supposed to do. When told what the button does, she did not know how to fetch a sentence. She complained that the panel below the 'fetch next sentence' button was blank and unintuitive what the panel was used for. She also complained about the wording used in the button, saying that it wasn't descriptive of the task of getting new sentences using checked words. She suggested teh wording "find sentences with checked words".
Task 4
The user had no problem with looking up the definition using the clickable buttons in the sentence and the translation. She focused the word by clicking the button on the popup.
Task 5
The user didn't have a problem forbidding the word from being shown.
Task 6
The user had no problem closing the sentences.
Task 7
The user had trouble locating the Closed Sentences tab and restoring the sentence. Although, she did complain about cosmetic issues. In particular, she thought there should be more horizontal spacing of sentences.
Task 8
The user had trouble noticing the study focus history button. So she searched for the desired focus word instead in order to change the focus. When asked why she didn't notice it, she said that it was too small and not obviously placed. This suggests that we need to design a way to grab the user's locus of attention better when the study focus is changed so that users will understand where the button is.
Task 9
The user had no problem with this task after having clicked the study focus history button in the previous step. However, the user suggested making the state of the system obvious. For example, she suggested using tabs to indicate that the state is in either "General Review" mode or "Study Focus" mode.
Task 10
The user had no problem with this step.
...
This user grew up in a Chinese-speaking environment, and has basic reading proficiency. He was able to read the sentences that he encountered while doing the tasks for this user test. However, after the user test, when he enabled the remainder of the vocabulary, there were words that he needed to click on to discover their pronunciation and meaning. Hence, the user could benefit from using this application, and thus matches the target audience.
Task 1
The user had no difficulty going to the Create Account tab, entering his username and password (switching between the fields using tab), and having the account created and logged in after pressing enter. The user initially typed the repeat of his password in incorrectly (apparently deliberately), and was pleased to see the immediate feedback that appeared that the entered passwords were different.
Task 2
The user had no difficulty locating and using the Textbook and Chapter selection combo-boxes on the left side. However, after the chapter was selected, the user wasn't entirely sure what to do. He first clicked on the "Displayed" sorting button (which sorts the words according to whether they are allowed to be displayed or not). Then he checked the "May appear in sentences" option for some individual words. At that point he believed he had accomplished the task, and started fetching sentences. After being informed that he hadn't actually accomplished the task, the user noticed the "check all" button and clicked it (which is how we had expected the task to be done).
The user later cited that part of his source of difficulty was that the "May appear in sentences" label was cut off (displaying only "May appear") because the user's browser window was not maximized, and was at the time smaller than the minimum size we had designed for. However, the larger usability issue in this case was that the "check all" button's location did not make it discoverable, or that its label did not carry sufficient information scent.
Task 3
The user first tried pressing Ctrl-F to search for the word using the browser's built-in search. In our case, however, because we had implemented the site using Silverlight not HTML, the browser can't search the page. After noticing that he wasn't getting when searching via the browser's built-in search, he quickly switched to the search box. The user didn't have any issues locating the search box or using it. After typing in the pinyin for the word, he found the word in the search results, but initially he checked the "May appear in words" checkbox, and didn't click the "make study focus" button. Only after fetching a few sentences and noticing that the desired word was not present did he focus his attention back to the left sidebar, and clicked the "make" study focus button. He clearly noticed the change this time, as he hovered his mouse around the (now highlighted in green) combobox displaying the word in the "study focus" section and faintly mumbled "ah that's it". He then pressed the "fetch next sentence" button to fetch the sentence, as expected.
Task 4
The user clicked on the word in the sentence to display its pronunciation and translation. He clicked the "make this word the study focus" as expected to make it the study focus, and then pressed the "fetch next sentence" button to fetch the next sentence.
Task 5
The user typed the pinyin in the search box to locate the word, and unchecked the "May appear in sentences" button. He then pressed the fetch next sentence button a few times to verify that the word indeed didn't show up to convince himself that he had actually accomplished the task.
Task 6
The user had no difficulty locating the close button for the button sentence and clicking it.
Task 7
Immediately after accomplishing the previous task and before the user was shown this task, the user switched to the Closed Sentences tab and pressed the "Remove sentence" button repeatedly until all the sentences were removed. Thus, because there were no sentences to restore, the facilitator asked the user to first fetch a new sentence and redo the previous task before proceeding. After fetching the new sentence and clicking the Close button, the user switched to the Closed Sentences tab, and pressed the Restore Sentence button. He then switched back to the Currently Reading tab, saw the sentence he had just restored, and repeated the process of closing it and restoring it from the Closed Sentences tab to verify that that its behavior indeed matched what his mental model.
Task 8
The user went to the Study Focus sidebar, clicked the combo-box, and selected selected the word in the drop-down menu as expected.
Task 9
Because the user had seen the General Review option in the Study Focus combo-box while doing the previous task, he used the combo-box to return to General Review, and was able to fetch the sentences.
Task 10
The user had no issue with this task - as expected, he went to the Contribute Sentences tab, and inputted a sentence using his computer's Chinese IME, the English translation, and submitted the sentence.
...