...

• CATASTROPHIC problem:  Instructors have no idea what the red E and the blue A mean unless they have seen the admin interface.
  • Fortunately, we figured this out because one of our users were forgetful and kept thinking that A stood for "approved" rather than "absent" even though this user had seen the admin interface.
  • Possible solution:  Add a legend to the normal instructor interface indicating what these letters mean.
• MAJOR problem:  Users are unclear what the button with the white arrow (which reverts their cursor to a normal mouse cursor) does.
  • Our first user testers spent a significant amount of time trying to figure out what this button did.
  • Possible solution:  Label the button with the text "Revert cursor".  We actually implemented this change before the third user test.
• MAJOR problem:  Drawing a gray box around shifts that are pending add or pending delete does not convey the pending status to the user.
  • Our users did not even really notice the gray box; instead, they noticed the text that changes to indicate which changes are pending approval.
  • Possible solution:  Instead of or in addition to the gray box, overlay a large letter P on top of the pending shift, similar to how we indicate excused and absent with the letters E and A.  This possible solution was suggested by one of our users.
• MINOR problem:  The majority of the users during user testing expected instructions.
  • We explained to them that the interface is ideally learnable without needing to read instructions.
  • Possible solution:  Put instructions on a separate page that is accessible via the navigation bar.  This way, users who feel more comfortable with reading instructions before trying the scheduling program can do so, but nobody will be obligated to read instructions.
• MINOR problem:  It is easy to miss the "You must submit..." message at the top, since users' attentions tend to be drawn to the calendar.
  • We categorized this as a minor problem, since none of our users had any trouble remembering to press submit.
  • Possible solution:  Make the message more contrasting, perhaps with a different font color.  Also position the message so that it is slightly lower on the page and closer to the calendar.
• MINOR problem:  On the admin interface, the users did not immediately notice the radio buttons used for accepting/rejecting requests.
  • Admins would probably remember where the radio buttons are after using the interface a few times, so this is a minor learnability problem.
  • Possible solution:  Make the background of the table that holds the radio buttons a contrasting color (maybe the same light color as the background to the calendar), so that it draws attention to itself.
• MINOR problem:  The wording of the message on the popup box that appears if the user tries to navigate away from the calendar does not make it extremely clear that the user must press submit to save changes.
  • Again, this is a minor problem because our users always remembered to press submit before leaving the page.
  • Possible solution:  Change wording of the message on the popup box to "Your changes will be lost if you do not press submit.  Are you sure you want to leave this page?" or some other message that makes it explicit that users must press submit to save changes.
• COSMETIC problem:  The color scheme (yellow/blue/black to represent day/evening/night) the buttons have when users schedule shifts is not extremely intuitive until the user has many shifts scheduled.
  • Only one of our users mentioned that she did not know what the colors mean, and an instructor can probably still figure out his schedule without the colors being there.
  • Possible solution:  Incorporate these colors into the day/evening/night legend.

Reflection

Over the course of the iterative design process, we learned how important planning is.  While had we not been forced to follow an iterative design process, we likely would not have done nearly as much planning near the beginning of our interface, we found that implementing our final version was much easier because of all of the planning we had put in early on.  Furthermore, we learned the importance of user testing because there were things that we thought would be very intuitive, but that users sometimes couldn’t figure out.  Since we found out many of these problems early on in our paper prototype, we were able to fix them much more quickly than had we not found them until after our final implementation.  Finally, we learned the importance of testing on users that are representative of our user population.  We found that our paper-prototype testers (who were all MIT students) often wanted features that would make the application more efficient to use; however, when we tested on our actual user population (much less technically savvy), their primary concern was just being able to figure out the interface.  We did a good job of recognizing this and using our intuition to guide how we actually responded to user feedback; however, it would definitely be more ideal to use actual user feedback rather than having to rely on our intuition for how our testers differed from actual users.

...