...
The user first tried pressing Ctrl-F to search for the word using the browser's built-in search. In our case, however, because we had implemented the site using Silverlight not HTML, the browser can't search the page. After noticing that he wasn't getting when searching via the browser's built-in search, he quickly switched to the search box. The user didn't have any issues locating the search box or using it. After typing in the pinyin for the word, he found the word in the search results, but initially he checked the "May appear in words" checkbox, and didn't click the "make study focus" button. Only after fetching a few sentences and noticing that the desired word was not present did he focus his attention back to the left sidebar, and clicked the "make " study focus" button. He clearly noticed the change this time, as he hovered his mouse around the (now highlighted in green) combobox displaying the word in the "study focus" section and faintly mumbled "ah that's it". He then pressed the "fetch next sentence" button to fetch the sentence, as expected.
...
Our solution to this problem would thus be to adapt a label which has better infromation scent, and describes what clicking the button actually does (for example, "Allow all words below to be displayed in sentences", or perhaps a less verbose version of this), rather than requiring the users to figure out that the button manipulates the checkboxes, and then read the labels for the checkboxes below, before they can determine what the button actually does. We should also remove the "Sort by" buttons for the vocabulary - the buttons conflict with our goal of simplicity on the left sidebar, and they are only useful if the user already knows what word that he's searching for is, but in this case it would be more efficient for the user to simply type the word in to locate it via the incremental search functionality.
"Make study focus" button label assumes user knows what "study focus" means
As we saw from how User 2 didn't initially press the Make Study focus button on task 3, he apparently didn't understand what "study focus" meant (namely, that the study focus must appear in fetched sentences). We could perhaps have replaced the term "study focus" in our interface with a term which better described what it meant for a word to be the study focus, for example, "word which must appear in fetched sentences".
Ctrl-F is inconsistent with usual browser search functionality
...