GR6: User Testing
Project: Pitch Perfect
Design
Final Design |
Description |
|---|---|
|
This is an overview of the home page. Two main changes occurred from our initial concept to the final version. In our original design, we had the Director's Announcements at the top of the page above the feed. From heuristic evaluations and user testing, we learned that people were confused because they thought the home feed was meant to be a response to the Director's Announcements instead of as a general feed. Therefore, we decided to move the Director's Announcements to the left side of the page, grayed out with the Upcoming Events section, to indicate a section that cannot be edited and is not the main focus of the page. |
|
Users can write comments that show up on the home feed. The name of the person who commented is clickable and takes you to his profile. |
|
Users can select a song to practice. During paper prototyping and heuristic evaluation, we originally just had the "Choose a Song!" header by itself. People told us that they were unsure what choosing a song did so we decided to add a subtitle "Listen, Comment, Record" to make it more obvious. |
|
This is an overview of the Practice section. The Practice and Record options are on either side of the sheet music. In the paper prototype, the practice options and the record options were both on the left side under different tabs. We decided to separate them into two different columns from paper prototyping since users wanted to see comments and practice options at the same time. |
|
These are the options available when practicing the song. The play options were originally just checkboxes but paper prototyping and heuristic evaluation found the checkboxes to be confusing. We decided to use slider bars instead. |
|
Users can record songs on the right side of the Practice page. |
|
Users can see their recordings on this tab and other people's recordings on the other tab. Originally, users would have a dropdown menu that had a list of all of the recordings a user can comment on. However, user testing found that it was really inefficient to go through all the recordings to find a specific one. We decided to separate your recordings from other people's recordings and also add a search functionality to find a specific recording. It was also pointed out that comments could become very long, so we made the comments collapsible by clicking on the recording like an accordion. |
|
This is the Member Directory. In the paper prototype, we had a list of members on the left sidebar. During paper prototyping, some people thought that it was not necessary to have the left sidebar since it took up a lot of space. Therefore, we decided to have a main page that displays all of the users with their icons instead. |
|
This is a user profile. In the computer prototype, the list of available times/days used to just be a box that showed weekday availability. From heuristic evaluation, people mentioned that having an actual calendar with more than one week would be more useful, so we decided to incorporate Google calendars for users to easily integrate. |
|
Originally, we planned to have an inbox that would allow users to message each other. Due to time constraints, we decided not to implement this feature in the final version of the site because we thought that the inbox was not as crucial as other features. |
Implementation
We broke down the implementation process into three components based on the three major features of our interface: computer-generated playback of music, recordings, and comments.
Playback
We used the Vexflow API to render the sheet music in a HTML5 canvas and the Audiolet API to play the rendered sheet music using computer-generated tones. The song presented ("Twinkle Twinkle Little Star") was hardcoded; in actuality, we would need an interface for the vocal director of the musical group to input their songs, which would be another design project altogether. The playback controls are from the jQuery-UI library, and we added listeners to changes in those controls that would affect the appropriate variables in our playback code.
Recordings
We used an open-source library called Recording.js, which uses Javascript and Flash to enable recording and playback capabilities. We were unable to persist recordings due to time constraints, but we think that for the purpose of UI design, this is not a serious problem. The lack of persistence did impact the usability of our user interface because we had to make explicit that creating a new recording would overwrite the old recording to testers.
Comments
A set of comments exists for each recording for simple organization. These were implemented as collapsible elements using Bootstrap so long comment chains can be hidden for a recording that the user is not looking at. Each accordion heading displays information about the recording and allows it to be played, while the inner accordion sections held comments. Comments are not persisted because it was too difficult to persist recordings, and it only makes sense to persist comments if recordings can be persisted. We believe this is not a big issue because this class is focused on UI design rather than backend implementation.
Evaluation
Briefing
We decided not to use a demo video because we did not want to influence testers unintentionally. We used a modified version of the briefing we provided for paper prototyping:
Our project's name is Pitch Perfect and we are creating a website for a cappella singers. It can be hard for an cappella singer to perfect his individual parts when singing in a musical group. Pitch Perfect will allow users to practice a cappella songs online, with an emphasis on social interactions among the group. The site allows users to practice their parts with the other voice parts in the background to simulate singing with the whole a cappella group. Members can listen to other people's recordings and make comments.
For the purpose of this test, you are an a cappella singer named Anna who wants to practice her solo for an upcoming performance. There will be 4 tasks total. Don't worry if you have no previous music experience- the tasks use very minimal music terminology. This is a relatively new site, so any problems you encounter are the computer system's fault.
Two of our group members will be observing you throughout the tasks. Your responses will be kept completely confidential and if you would like to stop at any point, just let me know. Any questions before we begin?
Tasks
- Practice singing the song "The Sign". For the play options on that page, please set the Volume to 85%, the Tempo to 130 beats/min, and include the Metronome. You may practice for as long as you would like.
- Now, make a recording of the song "The Sign". Feel free to change the play options and sing for however long you would like. After recording your singing, please save your recording as "Super Solo".
- Listen to Ben's recording of "The Sign" called "Tenor Recording". Make a comment on his recording. You can write whatever message you like.
- Check Ben's profile to see when he is available to practice.
Users
- The A Cappella Veteran (same person that we interviewed for GR1!)
- The A Cappella Beginner
- The Musical Theater Veteran
Usability Problems
We consolidated the feedback we received into the following usability problems.
- Major: Entering into the site for the first time is confusing. Our users took longer than we would have liked to realize they should head to the "Songs" tab to accomplish the first task. In other words, we need more affordances on our home page that direct the users towards the "Songs" page; this should also improve the efficiency of the site as a whole since the "Songs" page is the site's major feature. Possible solutions include a large button at the top, an overlay with an arrow that points to the "Songs" tab, or some very visible guiding text that briefly explains what a user should do to get started.
- Cosmetic: The guiding text on the "Choose a song" page seems like an affordance for clicking. One of our users tried to click on one of the words in the guiding text ("Listen - Record - Comment") since he thought it was a link to where he wanted to go. Possible solutions include rewriting the guiding text into complete sentences or changing the font style (into italicized) and color (from blue) to remove the clicking affordance.
- Catastrophic: There is no way to stop playing saved recordings. Our users noticed a bug in our interface - once they start playing one of the recordings they've made, there is no way to stop that playback. A solution for this is to simply add a stop button for the recordings or add states to the existing play button so that it can be toggled.
- Major: The computer-generated playback doesn't stop automatically when the user is done recording. Some users tried to record themselves singing with the computer-generated playback, which is fine, but it was very annoying for them to have to manually stop the playback of the music after they stopped recording. A solution for this is to add an event listener that stops the playback of the computer-generated music when the "Stop Recording" button is pressed.
- Minor: "Currently Recording" mode is not visible enough. The only indication that a user was currently recording was the updating of the timestamp next to record button. Possible solutions here include adding an icon or text to indicate the mode either near the record buttons or at the top of the page.
- Major: There is no way to change the volume of recording playback. The solution here is to add volume control for playing back the recordings.
- Cosmetic: It was not clear that the site does not allow tempo to be changed once playback of a song has begun; one user thought it was a bug. The solution here is to add guiding text for the playback controls.
- Minor: The metronome was very faint. One of our users thought the metronome was too faint for his needs, so it would be worthwhile to add a volume control for the metronome as well.
- Major: There is no way to tell what measure the computer is currently playing when playing back the song. The solution here is to highlight the measure (or note) that is currently being played and would involve changing our code slightly so that playing a note triggers an event that causes highlighting in our HTML5 canvas.
Reflection
The iterative design process allowed us to address our website's main features and usability issues in stages. We believe all of the prototyping techniques used in this class were useful in different ways.
By analyzing our user population, we first came up with the most important features, including playing and recording music, leaving comments, and communicating with others in an a cappella group. These features drove our priorities for implementation and usability fixes.
The low-risk paper prototype with user testing pointed out the most obvious usability problems so they could be fixed before coding even began. We even made changes to our paper designs between testing rounds, as they were very low fidelity and could be thrown out. We incorporated changes that solved both issues testers directly told us about and issues we simply observed with multiple testers.
Heuristic evaluations of the computer prototype outlined the major fixes necessary for our final implementation. Since changing the computer prototype was more risky and time-consuming, we focused our time on fixing issues directly related to our main musical features. For example, we chose to work on playing and recording the music correctly rather than implementing a search bar on the People page. We wanted the user to focus on testing what makes our website specialized.
Finally, testing with our user population showed us whether our website really helped our targeted audience.