GR6 - User Testing

Design

Final Design

In our final design, we divided our website into three pages: Search, Results, and Friends. These pages were navigated with a tab interface. The search page allowed users to input their search criteria. It included a widget that allowed users to efficiently select times for which they were busy, as well as a drop down menu for selecting specific departments. The results page displayed the results of their search in a gigantic table which was organized by starting-time of the class. Finally, the friends page allowed a user to search for their friends and see what classes they were taking. 

...

Search Page

Results Page

Friends Page

Design Decisions

One of our biggest design decisions was to make our interface very visual and to incorporate drag-and-drop operation. Our reasoning was that this scheme has been growing in popularity, so it would be familiar and comfortable for many reasons. Our alternatives were a more conventional interface that simply displayed all the results in a list instead of providing an elaborate graphical interface, and an interface entirely based on text commands for efficiency. The results from our paper prototyping and our heuristic evaluation made us more confident about this design decision, because everyone seemed to like the colorful, draggable interface. In addition, we found that a common use pattern was that people selected a small subset of courses that they were interested in and then wanted to compare them side-by-side to make their final course decisions. With a tradition interface that simply listed all results vertically, such as the current Course Catalog, users reported that they often opened two tabs and switched back and forth between them to accomplish this; by representing courses as manipulable objects we enabled users to physically group the courses they were interested in in just one window.

...

• We implemented the backend with php and a mySQL database. All of the class data is stored in the database. Using mySQL was very useful because it makes searching for different criteria very quick and efficient.
• We chose scripts.mit.edu to take advantage of features such as certificates login. The certificates allow us to save the sidebar data to a particular username, so that no login is needed to use the site.
• We also used php and mySQL to implement the persistency of classes in the right sidebar. Every time the user added or removed a class from a balloon, we would use an AJAX call to update a table in the database. This increased simplicity of the interface by removing the need for a "save" button. It also increased reliability of the website because the changes were always saved as soon as the user made them.
• Friends were also implemented with a mySQL database. The friends page is pre-populated with the names of everyone who has an entry stored for "classes I'm taking" in our database. This allows for fast,efficient searching of friends.
• We implemented the drag-and-drop functionality for courses using JQueryUI's Draggable and Droppable interfaces.

Evaluation

We tested our interface on four MIT students who had never seen our interface before. Three of them were asked to complete the tasks listed below, while the fourth was simply instructed to "Use this website to select your HASS classes for next semester." This allowed us to get feedback on not only the usability of all features in our interface, but also the discoverability of those features and on which features were most interesting to users.

We selected our test users from our friends and dorm-mates, trying to get a set of users that was as representative of our user population as possible. We feel that our test users were fairly representative of our target user base; they included both males and females, upperclassmen and underclassmen, students who enjoy HASS classes and students who simply want to fulfill their graduation requirements. One notable sub-group that was missing from our test users was freshmen, who are on the new HASS system. However, we felt that the difference in HASS classifications would not significantly impact usability or use patterns of our site.

We did not provide any demo as part of our user testing.

Briefing

I Can Has HASS is a schedule planning and course selection tool aimed at helping MIT students to select humanities courses each semester.

The site allows students to search for courses matching certain criteria, look at what courses their friends are taking, and keep track of courses that they are considering taking. Students can also construct a projected schedule based on their selections.

As you complete the tasks given to you, please vocalize your thought process as much as possible. This is especially important if you become confused or frustrated, because that means you have identified a problem in our design, and telling us what you are thinking will help us fix it.

Remember that you are free to quit the experiment for any reason, at any time.

Thank you for helping us to improve I Can Has HASS!

Tasks

1. Search for HASS classes of the following criteria:
   1. All HASS categories except 4 and Elective.
   2. No final exam
   3. Don't care about CI-H
   4. Not interfering with your other class, which meets 1-4pm on Wednesdays
   5. From departments STS, 21F, and 9
2. Select two classes and mark them as a classes that you are considering.
3. Compare the details of these classes side-by-side
4. Add Mark Zhang as a friend. Add one of his classes to classes you're taking.
5. Decide you no longer want to take one of the classes you were considering.
6. Decide you definitely want to take the other class you were considering.
7. Remove Mark from your list of friends.
8. Print the classes that you have selected.

Observations

Our interface received a lot of positive feedback in user testing. Some particularly noteworthy points were:

• users easily completed the tasks
• users instantly knew how to interact with non-traditional elements like our calendar and the draggable items -> good affordances
• when the search results page with the post-its first came up, several users exclaimed “Oooh!” or “Wow!” --> good user satisfaction and aesthetics

However, we also identified a number of points that our users struggled with:

• users didn’t know what all the department abbreviations stood for
• user tried to click on “I can has HASS” logo to return to Search page
• users did not immediately notice the legend at the top, as a result did not realize what colors meant
• users had trouble finding the “Compare” feature
• users were very confused that “print” link only appeared on results page
• some users had trouble closing the course details bubbles
• users were confused by the fact that when you add a friend’s class, the post-it disappears from the friend’s set of classes until page is reloaded. Asked, does he not have that class anymore?
• users weren’t sure whether friends’ classes were guaranteed to match their search criteria
• users expected results page to remain populated with results after they navigated away and then navigated back
• users were not sure what the difference was between putting a class in the “Trash” and dragging it out of the sidebar

Based on these observations, we have compiled the following table of usability issues that could be improved in the next iteration of design.

Reflection

We've learned a lot over the course of the iterative design process. Being forced to sketch out three user interfaces that were significantly different from each other showed us that there really are multiple approaches to arrange the inputs and outputs to our software. The paper prototypes showed us serious usability problems early on in the process, before we became attached to any code that we might have written. Our first paper prototype had a lot of a problems, which were easily fixed in the second and subsequent versions.

...

We took some design risks in our project. For instance, we designed a calendar interface which allows users to select and deselect blocks of time for when they are busy. The interface is completely unique - I we have not seen one like it. It's similar to Google Calendar, but simplified because we don't need all of the functionality that they provide. Our users loved it! Everyone found it extremely intuitive to use and was able to perform the necessary actions to complete our tasks.

...