Hubbub GR6 - User Testing

Design

In this section, we explain the various parts of our interface broken down by essential task. We discuss the design decisions we made, and their motivation in relation to our prototyping and analysis results.

Reading 

(Some names were blacked out for privacy.)

The goal of this part of the interface is to facilitate the reading of feed items. We picked a vertical feed format over one item per page or horizontal swiping for external consistency with other news feeds. We also chose this format because it allows users to skim many items quickly. Items are automatically marked read as the user scrolls down the feed page with similar visual feedback as Google Reader. The user can save items for later reading by hitting the item's "Save" button. Hubbub pulls live data from four sources: Gmail, Twitter, Facebook, and Imgur.

This design remained relatively constant through prototyping, with two exceptions: removing the "Share" button, and color-coding services. The paper and computer prototypes included a "Share" button on each item for sharing on their respective social media websites (e.g. retweeting for Twitter items). When we implemented the computer prototype, we realized that controls were taking up too much space on the page. Also, the behavior of "Share" was ambiguous to users. Therefore, we removed it from the final version of Hubbub.

Also, we originally planned for the interface to be mostly grayscale for simplicity. However, we decided to use color to make it easier to tell the interfaces apart. Due to screen size, the labels of the computer prototype became relatively smaller than the text and needed additional contrast. We chose colors that were externally consistent with the information sources. For example, Twitter items are color-coded light blue, and Gmail items are red. In the computer prototype, we made Imgur yellow, but on closer inspection of their design and feedback in heuristic evaluation, we realized green was more externally consistent.

Tagging/Saving

The goal of the tag menu is to help users manage their items via categorization. The goal of the save button is to give users a way to save items for later reading. Saving and tagging together comprise one of our essential tasks and provide better usability than the functionality implemented in our prototypes.

Users select tags from a list and can create new tags to add to the list. Tags are persisted in the back-end using a database. We considered an alternative approach that involved dragging tags directly onto items in the feed, but we realized that this approach did not meet our screen space constraints. Tags are selected via checkboxes so users can apply multiple tags to the same item at once.

When implementing the computer prototype, we decided to add a wide rectangular area (with a gray background) around each checkbox, and register clicks inside the area as checking the box. This combatted the "fat-finger" problem, which was not obvious during our paper prototyping user testing.

In the paper and computer prototypes, we had a "Cancel" button at the bottom next to the "OK" button. We decided to move it to the top and rename it to "Back" for two reasons: first, having two buttons next to each other was harmful for safety - users may accidentally click one button when they meant to click the other. Second, we wanted the tagging interface to remain consistent with the rest of the application. The filter interface also has a "Back" button at the top-left corner serving a similar purpose.

In GR4, the "Save" button's functionality was not implemented. When we switched to providing live data in the feed, we needed a way to prevent old items from diluting the content on the feed page. After getting feedback from Sacha, we updated the functionality of the "Save" button to keep items in the feed so the user could read them later.

Filtering

(This image has been shrunk slightly.)

Users can filter feed items to find what is currently of most interest to them. Hubbub allows users to filter by service, text, tags, or existence of hyperlinks. This interface was difficult to design for good learnability. When testing our paper and computer prototypes, the filter interface was the hardest for users to use. In particular, users were unsure how the different criteria combined (boolean "and" or "or"?) and some were confused about the function of the service icons or the text box.

The interface uses mostly standard widgets like checkboxes for external consistency. An alternative approach would have been to implement it as a drop-down menu. However, small menus are susceptible to fat-finger problems, require more clicks, and are typically not used on phones so users are less familiar with them in a mobile environment.

We made several changes to the interface to address these problems. First, we updated the icons in the Services section. Previously, we only showed faded icons for each service that became highlighted when selected. Users were unsure whether these icons were clickable, so we added checkboxes next to them (that get checked at the same time the icon is highlighted); users can select a service by tapping on the general area of the icon and checkbox.

Next, we updated the informative text in the form. We originally had a minimal amount of header text that only described the filtering category, such as "Services", "Tags", and "Text". We updated the headers to be more verbose, which gave users more guidance and helped answer the "and" vs "or" question. They appreciated the additional context and were more confident in their selections.

The paper and computer prototypes had a collapsible region called "More Options", which included filtering options such as filtering by tags and by "Has a Hyperlink". These options were hidden by default. The collapsible region used less space, but users in paper prototype testing had a harder time finding options inside the collapsible region. In addition, one heuristic evaluator reported that it was unclear whether the items inside the region were selected when it was collapsed. For these reasons, we decided to show all sections all the time and remove the collapsible behavior.

Our final changes were to the button controls. In our paper and computer prototypes, we envisioned the "Execute", "Save Filter", and "Reset", buttons to be grouped together. Heuristic evaluation showed that users would click one button while meaning to click another - this version has one button in each corner to avoid that. We also renamed "Remove Filter" as users thought "Reset" would clear the form, not remove the filter.

For reference, here's a screenshot of the filter interface from the computer prototype. The new interface updates alignment of form fields to make scanning easier.

Implementation

In this section, we provide a high-level description of Hubbub's implementation. Hubbub's interface is written Javascript using Backbone.js, jQuery, and Twitter Bootstrap. Persistence is achieved by communicating with a Ruby on Rails server application running on Heroku with a PostgreSQL database. When users authenticate with third-party services like Twitter, the server talks to Twitter directly to download recent tweets and store them in the database, which it processes and serves to the client as JSON. Users authenticate with third-party services using OAuth, so users do not have to disclose their passwords to us and may revoke our access at any time.

We used jQuery Mobile for the computer prototype, but dropped it for the final version because it had significant performance problems.

Filters are saved in the database so that the user can access them no matter where they log in, but the server doesn't use them to determine which items to send to the client. Therefore, it sends all items to the client, which will become a performance bottleneck under actual use.

We created a web-based interface so that Hubbub would be accessible to users of all types of smart phones, however it does run slower than native apps. Thankfully, we were able to overcome most interface delays with optimizations like removing background images and reducing the number of event handlers.

Evaluation

In this section, we describe how we performed our final user test analysis for Hubbub. We had 3 students (not in 6.813) user test Hubbub. All three users owned smart phones and used at least a subset of the services Hubbub supports very frequently. However, our users were not as representative of our target user group as our potential users from GR1. There was no demo, but users were briefed with the same briefing we used in GR3:

Hubbub is a mobile application for managing incoming information from Twitter, Facebook, e-mail, and so on, and reading what's important to you as efficiently as possible.

We had our users perform the following four tasks, updated slightly from GR3 in response to our interface changes:

1. Find something interesting to read.
2. Mark something to read later
3. Tag an Imgur image as "silly pic"
4. Create a filter for items from Gmail and Imgur, save it and run it.

Our observations:

• Users weren't sure how to indicate to the facilitator that they had found something interesting to read. All users asked if they were supposed to click on links, and most users tried to click on the items to follow links or open the item.
  • This is a user test issue that didn't appear in our paper prototyping.
  • There was something about testing our app on an actual phone that made users think they had to try and click on the items they wanted to read.
• As expected, most users marked items to read later by hitting the save button. However, the behavior of the save button was not clear for all users.
  • One user tried to mark an item to read later by tagging.
  • He suggested providing an optional popup explaining what happens when you hit the save button.
• All users tried hitting the load more items button, but it was not obvious whether new content was actually loaded. It was also not obvious that the button existed on the page
  • Some users scrolled past the load more items button looking for more content.
  • A possible solution is to reduce the amount of space between the button and the end of the feed page.
• Specifying filters appeared intuitive for users. However, saving and then executing filters was not obvious.
  • Two possible solutions are to not clear the user's filter selections on a save, or to auto-select the newly saved filter after clearing the form.
• Users often clicked items several times, and backtracked on operations for tasks due to slow response times in our app.
  • Users seem to be accustomed to quicker response times, which affects the learnability of our system.
  • We will need to look into optimizing our system and providing quicker visual response cues to the user

Reflection

In this section, we describe the various design decisions we've made throughout the semester, whether they were good or bad decisions, and what we could have done better.

GR1 - User And Task Analysis

Our initial writeup for GR1 did not fully cover the details of the user population, and did not distinguish between essential and non-essential tasks. After extensive feedback from Sacha, we revisited our task analysis and worked to resolve those issues, ultimately creating a great road map for Hubbub's design and implementation. We created three essential tasks: reading, filtering, and managing/saving items. The reading and filtering tasks remained consistent throughout Hubbub's entire design and implementation process. Similarly, the subtask of managing items translated into a tagging interface with minor design changes across milestones. In contrast, the process for saving items underwent extensive redesign multiple times from GR2 to GR5.

Evaluating our user population turned out to be a very important part our analysis, and helped us frame Hubbub's design goals in a way that made Hubbub considerably more useful for our target user population. If we didn't act on Sacha's feedback and update our evaluation, we would've run the risk of completely missing the point of the UID course, focusing more on the low-level details of Hubbub (like algorithms and performance) than what users would actually want out of it.

GR2 - Designs

Initially, we each had very different ideas for how we wanted Hubbub to look and behave, which translated into a lot of different interface sketches for GR2. However, by the end of GR2 we had a pretty clear idea for Hubbub's design, and some of our story boards ended up being fairly close to the behavior of Hubbub's final interface.

We explored different ways to filter feed information, as well as alternative approaches for presenting feed items to the user. Initially, we wanted to restrict the length of the feed based on the amount of free time the user had to devote to interacting with the feed. However, we abandoned this idea in favor of a more intuitive filtering interface. This made Hubbub's filtering implementation simpler for GR5, since the filtering algorithms were easier to implement. Considering how hard it was for us to make filtering intuitive, adding more complicated filtering algorithms would have probably slowed us down.

We also decided to provide a scrollable list of feed items. We felt that this was more efficient than our other designs, which required the user to swipe or tap a button every time they wanted to see a new feed item. The feed list ended up being very intuitive for our users, and none of them suggested changing this part of the interface's design.

GR3 - Paper Prototyping

The feedback we received from users interacting with our paper prototype was very helpful and we were able to make several major design improvements on the fly. For example, we switched from a tabbed interface to a main page with navigation buttons because users found the tabs counter-intuitive. Doing this for our computer prototype would have been a lot more wasteful since we would have had to throw away the work involved in creating the tabs in the first place.

We decided to focus on four sources of information for the prototype (Gmail, Twitter, Facebook, and Imgur), which lasted for the rest of the project. We actually had a long debate about whether to draw a larger or smaller paper prototype, and eventually decided to draw the interface bigger (which was the advice given by lecture and the TA's). Unfortunately, drawing the interface on a large sheet of paper obscured a space issue we had later on in terms of fitting buttons on the screen. When the interface became phone size in GR4, it was more cluttered and considerably harder to navigate. However, there was one user that told us she felt our interface seemed cluttered during paper prototyping. Since she was the only user that thought our interface was cluttered, we decided not to change it at the time. In retrospect, it would have been more beneficial for us to have users interact with a much smaller version of our paper prototype with which we could have caught our space issue much earlier.

GR4 - Computer Prototyping

This part of the project took a fair amount of work, but we were happy with the results. We were able to produce a usable interface suitable for heuristic evaluation. Starting work earlier could have made the process less stressful.

The jQuery Mobile framework was great for prototyping Hubbub's interface. However, its performance was abysmal and we had to replace it (with Twitter Bootstrap) before starting on GR5's backend.

We chose to create stub feed items for our computer prototype. However, this actually masked the issue of how to handle items the user has already read across sessions. Keeping stale items indefinitely hurts the readability of the feed, but automatically removing items may cause the user to miss out on information. Looking back, we could have simulated input streams by creating a stub generator that periodically added items to the feed.

Heuristic evaluation was very helpful and we incorporated several suggestions from our evaluators into GR5. In particular, we realized that the filtering interface was confusing and devoted more attention to its learnability in response to feedback.

GR5 - Implementation

Our final implementation reached relative feature-completeness, incorporating live data from all four of our intended information sources. We didn't have time to implement the "Share" feature, but it was a non-essential task. We could have managed our time more wisely by prioritizing tasks and completing more important tasks first. However, there was simply bad timing for our group for GR5--one group member was out of town attending CHI, and another had a paper deadline around the same time as the GR5 deadline.

Switching from jQuery Mobile to Twitter Bootstrap made our application much more responsive, and actually simplified the code!