GR6 - Torch User Testing

Design

Although our design maintained the general look and feel from the original prototype to our final design, many of the interface aspects changed significantly throughout the process.

This screen represents the landmark selection screen that allows users to choose the landmarks they would like to have included in their tour.  There are several features on this screen that evolved throughout the testing and implementation phases.  First, the names of the tabs changed several times.  We believe the final naming scheme represents to the user what each of the tabs is meant for; select landmarks in the first tab, view/edit them in the second.  Next, the text at the bottom was altered to ensure that users know the purpose of selecting landmarks on the map while also providing them feedback on the expected results they would find if they continued.  Third, a small popup indicating the name of the landmark selected was added so users would know what they have chosen.

This screen displays to the user what landmarks they have selected from the map view.  It allows them to remove unwanted selection as well as view a short description of each landmark.  Since users cannot select new landmarks, the name of the tab was changed from "List View" to "My selections" to indicate that this screen was for viewing current selections only.

This screen represents what the user sees when following a tour they have selected.  Paths are drawn according to from landmark to landmark in the order they appear in the tour.  Arrows were added to clarify the which direction along the path users are meant to follow.

This screen displays how users create new tours and add them to the database.  Selecting landmarks will highlight them and add them to the current tour.  Updated paths (with arrows) are drawn as users make changes to the tour.  Below the map, users can edit the current state of the tour by either removing unwanted landmarks (by pressing the X) or reordering landmarks (by dragging and dropping using the dots).  Although selecting a landmark (using the map) already in the tour removes it from the list, the X was added next to each selection to allow users a second way of removing landmarks.  Additionally, the drag and drop selection area was increased since some users were unaware of that functionality. Pressing the "Save Tour" button brings up a popup window in which the user enters a tour name and description.  This box, along with the error message displayed for insufficient or illegal information, was changed to make it more clear what the user was supposed to do.  Also, when an error is made, Torch now returns the user to the "Save Tour" popup with the current information still in tact rather than closing the popup and making them reenter information.

Implementation

We developed for Android 2.3 using the Google Maps APIs and Eclipse. We thought that this would allow us to take advantage of recent improvements to the Android OS, but still support a majority of users. The server is PHP with a MySQL database. Requests are sent from the app, and the servers responds with XML, which is parsed by the app. We tested our app on the Nexus One, Galaxy Nexus, and several other models of Android phones. The test phones included operating system 2.3, but also version 4.0, Ice Cream Sandwich.

We made several simplifications that we deemed appropriate given the scale of our project. Our app has a local front end, but the tour data is stored on our server. The landmark data is stored locally, and users cannot add new landmarks with their own pictures. Also, we simplified the database for tours by not allowing the user to chose a picture for a tour they create. These don't take away very much from the functionality of the app, at least in terms of what we think our users want/need to do. 

Users can add their own tours to the database, and in the future when they search those landmarks they will find their tour. This is not local to this user, so all users see all tours created. There is currently no way for a user to delete a tour, but this doesn't hurt functionality that much. 

Evaluation

User Population

When we were initially formulating the idea for our app, we consulted with several of our friends to determine whether they thought our idea was any good. Several of them expressed that they would use the app themselves; therefore we followed-up with these users in order to perform our user study. We believe them to be good representative users since they are people that would be likely to search for, download, and use our application. However, they are all MIT students and there are likely other biases due to the fact that we knew them and they were not selected randomly. We decided that this was an acceptable trade-off for the ease of finding users, since we did believe them to be within our target population.

Procedure

All of the users for the user study had not used our app previously. One of them was an Android user, so we provided him the application and instructed him to install it. The other two were not Android users, so we provided them with our Nexus One test device to use. We instructed all users to open the application. We then read them our briefing, unchanged from our paper prototype: "Torch is a mobile application designed to help users find and follow walking tours of notable landmarks and attractions. Users can also create and submit their own tours." We then sequentially gave them three tasks, also unchanged from our paper user study. We focused on these tasks throughout our entire design process, so it made sense to keep them the same.

1. Find a tour of MIT's campus.
2. Follow a tour around MIT's campus.
3. Make a tour of MIT's dorms that includes Baker, East Campus, and Burton-Conner.

Results

For the most part, reactions from our users were positive. Users were able to complete our tasks with very little assistance, which is strikingly different from our paper prototype user study.

Usability problems are summarized below, along with possible solutions:

• Cosmetic - Some users did not see the popup displaying a landmark's name, but they knew what it was due to their familiarity with MIT
  • This may be solved if users were not familiar with MIT, because when users searched for the name of the landmark they were able to find it.
• Minor - All three users attempted to click on a landmark and suggested that clicking on a landmark when following a tour would pop up more information about it.
  • We planned to add Yelp integration in our app, but found it outside the scope of our interface design.
• Cosmetic - Two of three users wanted information to automatically pop up when you approached a landmark.
  • Right now, the landmark is being highlighted as you approach it, so an info popup would be an easy extension.
• Minor - Two of three users could not determine how to re-order landmarks when creating a tour. All three of them first attempted to do it by dragging the landmark from its center, instead of on the left (where the draggable affordance existed). One user did see the dots and assumed that meant it was draggable ("otherwise why would there be dots?").
  • We could make the entire landmark draggable.
  • We could make the draggable affordances more obvious.
• Major - When searching for landmarks, half of the screen was taken up by the keyboard and the other half was divided into half map and half list. Both of the two users who attempted to search by name found this interface to be much too small to interact with.
  • When the keyboard is open, we should hide the list view so that a larger map view is visible.
• Minor - When you search for a landmark that is already on-screen, it is not obvious that anything happens when the map relocates slightly.
  • A popup that says "Found x results" or similar could be useful.
• Cosmetic - One user was overwhelmed with the initial map view, because there were many landmarks.
  • This could be solved by clustering landmarks together until the user zoomed in to an appropriate zoom level.

Reflections

Designing a product is highly challenging and was one of the most difficult task our group has ever faced together. We all had unique visions for our final product and merging those visions together with the other constraints of real life, such as time, was difficult and taught us all some valuable lessons.

Through Torch, we really learned how important planning is. Although we definitely pulled it together, spending more time early on would have really made a difference on our stress levels and distribution of work. Overall, this project was a lot of fun, and we all had a great time doing it and hope to potentially take it further.