...
Task 7: (optional)
Start a new search
Observations.
Usability problems you discovered from the testing. Describe critical incidents encountered by the users, but don't record users' names. Record these as a series of high-level takeaways, focusing on the usability problems you saw, rather than what each participant did. For instance, you might describe how you had some learnability issues with your prototype, as evidenced by users B and C clicking all of the menus to try to find option X.
Iteration 1
User A:
The first user never seemed to realize the concept of deleting, whether it be a user or a keyword. When he messed up the OR input, he actually just decided to start from scratch and re-enter all the information. In fact, he eventually directly input the phrase from the card to get the intended output. His only variance was the addition of parenthesis. He completely skipped the task of deleting users, because he kept selecting them and couldn't figure out what to do with them. We eventually had him skip the task and continue. He was the only person who experienced any difficulty with this feature.
...
None of our users even attempted to drag and drop the tags together to create an OR. Similarly, they all had extreme difficulties with adding OR inputs.... they just didn't know what the expected behavior was. This indicates a lot of learnability issues with entering requirements.
Some of the affordances people couldn't see like which buttons are clickable (email students/see hidden students probably should not be allowed until they actually do something). Another learnability issue was figuring out how to hide or remove users. The "X" in the upper right of each user card was difficult to see, so we decided that we need to make it more visible and make it's effects more clear. The solution is to remove the "X" and add a "Hide Student" button next to the view profile information. We also are going to remove the bottom left "View Hidden Users" button in favor of moving it up to the upper right next to "Create New Search." This change will move it closer to the "Undo" message that gets shown.
...
It seems that although its getting easier for users to do composite boolean logic there is still some learnability issues with requirements. Perhaps we would benefit from a tutorial at the beginning on how to enter in requirements, or a recommended menu of things you can do (w/ composite boolean logic in the menu). Users seem to prefer personal touches to email and currently adding personal touches is very inefficient, so we may be able to improve our efficiency by having a template template that will enter in each students name individually so that the email can appear personal even though you only have to write it once. We Some users didn't even see the affordance that you can select multiple students at a time.We could also improve our efficiency by having some sort of tabs system that shows search results in one tab, hidden students in another tab, and selected students in another tab since if there are many search results on many pages it can be difficult to remember which ones are hidden and selected. We should also have some sort of count displayed for hidden students and selected students since users might not want to go over a certain limit.
...