GR6 - User Testing

Design

Overview:

The overall design of the interface did not vary significantly from the paper prototype we decided to implement. The interface has 3 main visualizations: an artist similarity graph, a friends’ concerts graph, and a calendar. Each of these pages has 2 static sidebars: one to maintain a list of curated concerts, and one to provide filtering on these concerts. A static menu bar at the top of the screen is persistent across all pages, and allows the user to access any other page in two clicks, at the most.

Design Decisions and Alternatives

One of our biggest design decisions was to allow users to access any visualization immediately. We were initially toying with the idea of guiding the user through a step-by-step tutorial, where we would present recommended concerts at the end. Instead, given the users’ varying interests, we chose to go with a more parallel design.

We attempted to make our graphs as clean and minimal as possible. Concert nodes on the graph were labelled only with the artist name, which initially confused users. Thus, we later added tooltips that provided the date and pricing information when the user moused over a node.
This way, the look of the graph remains the same, but the user can request more information with minimal interaction.

One large change made in response to heuristic testing was the price filtering. We initially chose to dynamically color the nodes based on a price slider. This was unusual and confusing, however, since it didn’t have any external consistency. Instead, we chose to bucket prices within the range of most concert prices, and colored the nodes based on the buckets, instead.

Implementation

Internals

• The backend was implemented in Ruby on Rails with a MySQL database.
• Front-end styling was built on top of Twitter Bootstrap.
• Visualizations were produced with d3.
• AJAX requests (and other general JavaScript functionality) used jQuery.
• Concert information for the next few months was gathered by hand and stored in the database.
• Concert and artist information were gathered through Echonest’s API
• Katrina’s listening information was gathered through last.fm’s API
• The artist similarity visualization was seeded on Katrina's listening habits.

Impact on Usability

The largest impact on usability came from not fully integrating various services. For example, a user flipped through the website, looking for a link to buy tickets for the concerts she had selected. This is a reasonable expectation for a concert recommendation site, but integration with ticket vendors was deemed too overwhelming to implement.

Evaluation

Briefing

Hi. In this experiment you are using the ConcertBOS site to find a concert to be inspired by.
This site allows you to visualize information about concerts in the greater Boston area, allowing you to find concerts that meet certain criteria, find friends who may be interested in the concert, and find additional details about the artist or venue.
The data we use has been pre-generated, so it isn’t currently tailored to your last.fm profile. However, all the data used is real; it’s just been tailored around one user’s musical interests and friends.
This experiment is intended to bring out flaws in the interface. We've sketched out a simple version of the interface to work through, where we play the role of the computer. Pretend you're interacting with a website. As you explore the website, we'd appreciate it if you would tell us what you're thinking, especially if you get stuck or confused. This means that there's something wrong with our design, and knowing what you're thinking will help us fix it.
Thanks for helping with our project, and keep in mind that you can stop at any time.

User Information

• Person 1
  • age/gender: 19M
  • music experience: dj-ing for 5 years at various local events in northern california, played the alto sax and piano for 10 years
  • a musician currently at college: exactly our target population
• Person 2
  • age/gender: 22F
  • music experience: highly accomplished concert pianist for 17 years, has performed with the boston symphony orchestra
  • a musician alum from MIT. A little more tech-savvy than our expected user population.
• Person 3
  • age/gender: 20F
  • music experience: pianist for 8 years, music major at MIT. Discerning music taste.
  • A music major at MIT who is very picky about concerts. She’s not very experienced with finding concerts online, and only hears about them through word of mouth. Almost exactly our target population.

Results

Our interface received a decent amount of positive feedback. Things people liked included:

• The friends visualization graph was clear and entertaining. Lots of “ooh, cool” and “I like this!” comments.
• The interface was minimal and easy to navigate
• The aesthetics of the site were well-liked. The visualizations were visually pleasing.
• The filtering bar was extremely clear. Users had no trouble understanding the price coloring and filtering concerts based on their age.

However, users also struggled with some aspects. Some of these issues (Ticketmaster integration and dynamic visualizations, for example) were difficulties stemming from implementation difficulties.

Reflection

We really enjoyed the iterative design process. Since we were trying to come up with a novel means of concert discovery and recommendation, forcing us to put different designs on paper was illuminating. We received excellent feedback at each stage of the process, both from our users and from our TA. Forcing us to iterate through several paper prototypes (we went through the 3 paper prototypes, and tested 3 versions of one prototype) really helped highlight some severe usability problems, which were easily fixed and refined in subsequent versions.

Creating an unusual interaction with concerts was both exciting and challenging to us. Dealing with these views was exactly as risky as we expected.  Since we were steeped in the world of data visualization, graphs that made sense to us needed some clarification to others. For example, we realized that the idea of an interactive graph was still a little unusual to most of our users, especially ones from our target population. Thus, at each stage, we had to ensure that our graph had excellent visible affordances. Consistency across pages was extremely useful.  Once users learned about node coloring and filtering, they could use these tools across all three visualizations.  User feedback was vital to us, since we had almost no similar interfaces to guide us. In response, we attempted to address all of our user’s concerns in each stage of the process.

Our computer prototype had essentially all the backend functionality of the implementation, which allowed us to concentrate on iterating the interface. This is something we would definitely do again, given the chance. Heuristic evaluation helped us identify several usability problems, and we had the ability to concentrate solely on fixing these issues.  We really enjoyed working with the technology stack we chose, since it allowed for rapid computer prototyping.

Our interface went through many design iterations over the course of the term, and we feel it's improved significantly since its inception.  We haven't found a good source of consolidated concert information with a good UI yet. Last.fm, Songkick,BandsInTown all have the same data, but they present the concerts as a list.  The calendar interface with coloring by price and filtering by age is a simpler, more intuitive way of navigating concerts than a list or word cloud, and the artist similarity and friend visualization provide a more interesting view.