GR6 - User Testing
Design
Overview:
The overall design of the interface did not vary significantly from the paper prototype we decided to implement. The interface has 3 main visualizations: an artist similarity graph, a friends’ concerts graph, and a calendar. Each of these pages has 2 static sidebars: one to maintain a list of curated concerts, and one to provide filtering on these concerts. A static menu bar at the top of the screen is persistent across all pages, and allows the user to access any other page in two clicks, at the most.
Artist Similarity | Friend Visualization | Calendar | Artist Page | Venue Page |
|---|---|---|---|---|
| | | | |
Design Decisions and Alternatives
One of our biggest design decisions was to allow users to access any visualization immediately. We were initially toying with the idea of guiding the user through a step-by-step tutorial, where we would present recommended concerts at the end. Instead, given the users’ varying interests, we chose to go with a more parallel design.
We attempted to make our graphs as clean and minimal as possible. Concert nodes on the graph were labelled only with the artist name, which initially confused users. Thus, we later added tooltips that provided the date and pricing information when the user moused over a node.
This way, the look of the graph remains the same, but the user can request more information with minimal interaction.
One large change made in response to heuristic testing was the price filtering. We initially chose to dynamically color the nodes based on a price slider. This was unusual and confusing, however, since it didn’t have any external consistency. Instead, we chose to bucket prices within the range of most concert prices, and colored the nodes based on the buckets, instead.
Implementation
Internals
- The backend was implemented in Ruby on Rails with a MySQL database.
- Front-end styling was built on top of Twitter Bootstrap.
- Visualizations were produced with d3.
- AJAX requests (and other general JavaScript functionality) used jQuery.
- Concert information for the next few months was gathered by hand and stored in the database.
- Concert and artist information were gathered through Echonest’s API
- Katrina’s listening information was gathered through last.fm’s API
- The artist similarity visualization was seeded on Katrina's listening habits.
Impact on Usability
The largest impact on usability came from not fully integrating various services. For example, a user flipped through the website, looking for a link to buy tickets for the concerts she had selected. This is a reasonable expectation for a concert recommendation site, but integration with ticket vendors was deemed too overwhelming to implement.
Evaluation
Briefing
Hi. In this experiment you are using the ConcertBOS site to find a concert to be inspired by.
This site allows you to visualize information about concerts in the greater Boston area, allowing you to find concerts that meet certain criteria, find friends who may be interested in the concert, and find additional details about the artist or venue.
The data we use has been pre-generated, so it isn’t currently tailored to your last.fm profile. However, all the data used is real; it’s just been tailored around one user’s musical interests and friends.
This experiment is intended to bring out flaws in the interface. We've sketched out a simple version of the interface to work through, where we play the role of the computer. Pretend you're interacting with a website. As you explore the website, we'd appreciate it if you would tell us what you're thinking, especially if you get stuck or confused. This means that there's something wrong with our design, and knowing what you're thinking will help us fix it.
Thanks for helping with our project, and keep in mind that you can stop at any time.
User Information
- Person 1
- age/gender: 19M
- music experience: dj-ing for 5 years at various local events in northern california, played the alto sax and piano for 10 years
- a musician currently at college: exactly our target population
- Person 2
- age/gender: 22F
- music experience: highly accomplished concert pianist for 17 years, has performed with the boston symphony orchestra
- a musician alum from MIT. A little more tech-savvy than our expected user population.
- Person 3
- age/gender: 20F
- music experience: pianist for 8 years, music major at MIT. Discerning music taste.
- A music major at MIT who is very picky about concerts. She’s not very experienced with finding concerts online, and only hears about them through word of mouth. Almost exactly our target population.
Results
Our interface received a decent amount of positive feedback. Things people liked included:
- The friends visualization graph was clear and entertaining. Lots of “ooh, cool” and “I like this!” comments.
- The interface was minimal and easy to navigate
- The aesthetics of the site were well-liked. The visualizations were visually pleasing.
- The filtering bar was extremely clear. Users had no trouble understanding the price coloring and filtering concerts based on their age.
However, users also struggled with some aspects. Some of these issues (Ticketmaster integration and dynamic visualizations, for example) were difficulties stemming from implementation difficulties.
Issue | Severity | Solution |
|---|---|---|
Once they’d curated artists, users couldn’t find them efficiently on other graphs. | Major | Highlight curated concerts on the graph, and possibly provide for progressive filtering. |
Visualizations did not resize gracefully at lower resolutions (less than 800px wide) | Major | The target population will use this on a computer (>800px in width), but supporting smaller devices would be good. Have smaller, less extensive versions of the visualizations meant for smaller screens, or don’t print artist names. |
User wanted a consolidated page to see summaries of all the selected concerts. | Major | Add a home page that displays a summary of all the selected concerts, with text that makes it easy to email. |
Actions on concert nodes on the calendar page are inconsistent with the other visualizations. | Major | Switch actions between the ‘+’ button and the actual concert node. |
The visualizations were not affected by interests. | Minor | Progressive filtering could be implemented. It would be difficult to dynamically update the visualizations, however. |
Users expected more artist and venue information on the concert page. They had to click through to see information about the artist or venue. | Minor | Add artist and venue details on the concert page. |
User wanted to buy tickets and was uncertain how to do so. | Minor | Add TicketMaster integration. |
Text on the similarity graph overlapped | Cosmetic | Nodes on the graph need to be spaced out more. Text should be rotated more intelligently. |
Nodes on the friends graph floated over each other | Cosmetic | Prevent nodes in a force diagram from overlapping. |
Reflection
We really enjoyed the iterative design process. Since we were trying to come up with a novel means of concert discovery and recommendation, forcing us to put different designs on paper was illuminating. We received excellent feedback at each stage of the process, both from our users and from our TA. Forcing us to iterate through several paper prototypes (we went through the 3 paper prototypes, and tested 3 versions of one prototype) really helped highlight some severe usability problems, which were easily fixed and refined in subsequent versions.
Creating an unusual interaction with concerts was both exciting and challenging to us. Dealing with these views was exactly as risky as we expected. Since we were steeped in the world of data visualization, graphs that made sense to us needed some clarification to others. For example, we realized that the idea of an interactive graph was still a little unusual to most of our users, especially ones from our target population. Thus, at each stage, we had to ensure that our graph had excellent visible affordances. Consistency across pages was extremely useful. Once users learned about node coloring and filtering, they could use these tools across all three visualizations. User feedback was vital to us, since we had almost no similar interfaces to guide us. In response, we attempted to address all of our user’s concerns in each stage of the process.
Our computer prototype had essentially all the backend functionality of the implementation, which allowed us to concentrate on iterating the interface. This is something we would definitely do again, given the chance. Heuristic evaluation helped us identify several usability problems, and we had the ability to concentrate solely on fixing these issues. We really enjoyed working with the technology stack we chose, since it allowed for rapid computer prototyping.
Our interface went through many design iterations over the course of the term, and we feel it's improved significantly since its inception. We haven't found a good source of consolidated concert information with a good UI yet. Last.fm, Songkick,BandsInTown all have the same data, but they present the concerts as a list. The calendar interface with coloring by price and filtering by age is a simpler, more intuitive way of navigating concerts than a list or word cloud, and the artist similarity and friend visualization provide a more interesting view.