KaJaMi - GR6 - User Testing

Video of the final implementation:http://www.youtube.com/watch?v=R0NJUoRmfBk
(Use 1080p)

Design Description:

Back to the top

Implementation Description:

Implementation Details

The interface was implemented on top of Django, with user-side UI code making use of jQuery and Bootstrap, as well as a number of UI-related libraries such as LESS to manage CSS styles, X-Editable for in-place editing, and Tablesorter for sorting tables.

Django enforces a model-view-controller-like design strategy, which was mirrored to an extent in the Javascript code. Each conceptual function (listing uploads, editing uploads, and reporting plays) was divided into separate JavaScript code and HTML templates for ease of maintenance, with server-side manipulation using separate controller functions ("views" in Django). Django models such as UploadGroups, Albums, and Tracks were mirrored in the JavaScript code with corresponding classes and objects used to help maintain UI consistency by posting events relating to those objects to the window (to allow list entries to dynamically update when the contents of in-place editors are saved, for example), and limited model-specific functionality (such as parsing metadata out of music files) was implemented as model functions.

While most HTML content is either produced server-side at load time or client-side at edit time, the reporting interface is unique in that it uses partial HTML to update tables when changing the release date filter. Changing the genre filter and sorting the tables are client-side functions which use the data in the HTML table itself to selectively hide and re-order rows, improving responsiveness lost in the release date filter.

Javascript functions were largely written as flat functions that strongly depended on the HTML5 event model to fire them and appropriately chain them to meaningful events to perform relevant actions both in the UI and on the server (e.g. drag & drop, in-place editing of server-side data, and XMLHttpRequest upload events).

Implementation Decisions

Perhaps the most complex code was that which permitted drag-and-drop uploads in the first place. Although the HTML5 drag-and-drop and file APIs did not actually restrict what was possible, practically all drag and drop code (including selection, dragging, and dropping of MP3 files in separate upload groups) had to be written from scratch from the underlying drag events to not only resemble behavior on image-hosting site imgur, but also to permit reorganization of unzipped MP3 files being uploaded much like dragging and dropping into different folders. Furthermore, different drag and drop event ordering in Firefox and Chrome required careful management of "dragleave" and "dragover" events to correctly manage the fading of the "Upload" overlay. The file upload API additionally offered in-line upload progress bar tracking which would not otherwise be possible.

Furthermore, in the reporting section, as mentioned above, we improved performance and responsiveness of filtering of the upload lists and reporting tables by doing client-side search of the various rows on each page, rather than relying on partial updates over the network. In practice, however, it may be necessary to only load uploads partially, leading to an XMLHttpRequest-based partial update mechanism for upload list searching, much like that used in release date filtering on the Reporting page.

Implementation Limitations

Because the code was initially written to map to the existing WMBR music database schema, a number of limitations arose from requirements imposed by Django (such as the requirement of exactly one primary key column, and an inability to use Django's custom column types like FileFields). Some of these issues were resolved by simply migrating to an appropriate backing table, while others were left only partially resolved pending a better resolution (e.g. genres are stored as comma-delimited strings rather than as separate models using a join table, which meant that JavaScript code currently splits a string for handling as separate tag buttons).

With respect to uploads, limitations in HTML5 and HTTP prevented the "pause" button dictated by our earlier feedback from behaving as expected (pausing the upload). At the moment, it is currently a placebo that does not actually affect the upload, as the only other alternative is to cancel the upload entirely and restart it when unpaused, which seems equally undesirable. Given the original use-case justifying the need for the "pause" functionality (i.e. pause would be clicked before "Delete Upload" which should cancel the upload in any case, it seems that a placebo effect would not have significant downsides and would nevertheless improve learnability by "having a button in an expected place" with some understandable behavior.)

CSS limitations in the implementation of X-Editable prevented the use of the "pop-up" design of album-level editable details with track-level details. As a result, consistency was broken between editing album-level details (which presented popup editors) and track-level details (which were in-line). Users generally did not seem to have much of an issue switching between the two, but several users expressed some general concerns about when data was saved regardless of the editor which might be exacerbated by the in-line editors. The benefits of the pop-up balloon for learnability, however, far outweigh the costs of inconsistency in not being able to use pop-up balloons for track-level ordering. In practice, however, pop-up editors should be used for all editors.

Similarly, limitations in X-Editable prevent using a date selector along-side a text-editor for date, without which, release dates may take longer to edit due to the need to search a calendrical display. A ComboDate editor was evaluated, but due to the ever-changing bounds of release dates (e.g. the system may need to support dates for many years in the future as well as many years in the past, making the year combobox difficult to scan), this may not be a sufficient replacement. As such, we stuck with the calendar editor, although it is not likely to be ideal.

Finally, space limitations from the standard HTML table layout algorithm resulted in the cramping of the "Play Count" columns, which may impede readability and understanding of the durations over which play counts are aggregated.

Back to the top

Evaluation

User Population

All users are Music Directors at WMBR and fall directly into our target user population. We connected with these users through a group member who is a DJ at the station, but not a Music Director (hence satisfying the project requirement that none of the group members fall into the target user group).

The 4 users are within the age range of 20s - 60s, with varying levels of technical proficiency. Three are female and one is male. All had a need for a music importing system such as KaJaM! Some use the uploading feature more while others are more concerned with the reporting function. Each Music Director is typically in charge of one single music genre. Some music genres deal with more digital content than others, hence usage of the KaJaM! application is expected to differ across users. 

Conducting User Tests

User testing was conducted at the WMBR Radio Station located on MIT campus, lasting on average 30 minutes in length. We briefed the users by explaining the scope of our application (limited to importing music, not playing or managing the library), and our hypothetical scenario. The briefing is kept consistent across users by showing them the same instruction sheet. We also explained that we are interested in how users naturally interact with the interface, and assured them that there is no right or wrong way to use the application.

All users interacted with a Macbook laptop owned by one of our group members and used the attached Touchpad + Keyboard to navigate the KaJaM! interface. One group member was acting as the facilitator and was in charge of walking the user through the scenario tasks, providing assistance only when necessary. The remaining two members sat alongside the user and noted down critical incidents + usability issues. 

We decided that a demo is not necessary due to the intuitive interface of the application. The scenario tasks presented to all users can be found as follows:

*Assume Lana Googles the correct album information, so facilitator provides all the correct information to the user on a separate sheet of paper:

Usability Problems Observed

Music continues to play even after the uploaded album is deleted. (Catastrophic)

• This is a major problem because if a user plays a track while editing the details and decides to delete the album (upon discovering that the album has already been uploaded previously), the music continues to play despite the removal of the entire album.

Potential solutions:

• This needs a bug fix that checks for currently playing music when a track or an entire album is deleted. If the track or album playing is deleted, stop the music right away.

"Add Genres" button is not conspicuous, it is easy to miss the fact that an album has not been assigned a genre. (Major)

• Currently, unlike other album details such as "Album Name" or "Release Date", it is not obvious when an album has not been assigned a genre. It is easy for a music director to approve an album without a genre right now, since the "Add Genres" button is quite small in size.

Potential solutions:

• Insert a blank field labeled in red "Missing Genre" in order to warn the music director that no genres have been added to the album.
• Increasing the size of the "Add Genres" button and make it more conspicuous by adjusting the color to be brighter/more saturate.

Filtering by genre is not intuitive on the reporting interface. (Major)

• Currently, users can filter the reporting interface by genre through adding and remove genre tags to the "Current Displaying:" field of the interface.
• Many found adding genre tags to be intuitive, but removal non-intuitive. 
• If more than one genre is displayed, users will tend to click on one genre tag to filter by that genre only.

Potential solutions:

• Make the "X" beside each genre tag more conspicuous (perhaps changing its color to red?). This will prompt the user to notice the removal feature of the genre tag.
• Clicking on an already displaying genre tag will filter the reporting interface by that genre only.

Clicking on "Add Genres" button after typing in custom tag does not actually add the custom tag to the album (only pressing "Enter" works). (Minor)

• One user noted that after typing in the custom tag, it is more intuitive for her to click on the "Add Genres" button to save and add the new tag to the album. However, this does not work as the application currently requires the user to press "Enter" to add the new custom tag.

Potential solutions:

• Add functionality to the "Add Genres" button, so that if the custom tag field is populated upon clicking the tag field gets saved.
• Add a prompt for the user so that after typing in the custom tag, a small tooltip pops up to remind the user to press "Enter".

Specific track details missing from the "album details" view. (Minor)

• Music directors would like to see more information about each track in the album, such as duration of the track, bit rate quality, as well as size of the file etc. Currently there is no way to view such information.

Potential solutions:

• Adding commonly used track details (duration, size) beside the track name.
• Complete track details could be viewed by right clicking on the track and displaying a "Get Info" option to invoke a separate pop-up window with full track details.

"FCC Clean" toggle is not comprehensive enough for the music director to identify where exact obscene words are found in a music track. (Minor)

• Music directors currently have to record manually, on each album cover, which tracks contain inappropriate words and what/when words appear. 
• "FCC Clean" toggle on our application allows directors to mark tracks with obscene words, but does not give more details beyond that.

Potential solutions:

• Allow a "FCC" comment field to be associated with each track, so that music directors can type in information regarding what and when obscene words appear in the track.
• Display a track progress bar (representing the duration of the track), that allows music directors to drag the cursor and place specific markers along this progress bar indicating when and what inappropriate words are in the song.

"FCC Clean" toggle is not conspicuous. (Cosmetic)

• Users noted that the "FCC Clean" toggle functionality is not immediately obvious.
• Users tried to append "FCC Dirty" to each track name instead of using the "FCC Clean" toggle button.

Potential solutions:

• Use the familiar FCC icon with the "poison" sign, this maintains external consistency.
• Use a bigger icon that is simply labeled "FCC" instead of any graphic icons.

What Users Liked

1. The "Add Genres" function allows any number of tags to be added to the album, so a single album can be tagged with multiple genres.
2. The "Add Genres" function allows custom tagging, so music directors can enter their own genre tag (if it does not already appear on the list).
3. Date picker for "Release Date" field is very useful - user noted "that's so nice!"
4. Entire interface is very clean and intuitive. No messy cluttering of information, controls, or menu items.

Back to the top

Reflection

One thing we realized early on is that exploration contributes to a better final product. When brainstorming for project ideas we pitched more than a dozen in total, pursuing some to the extent of writing up short interviews we had with owners of various stores (bookstores, video game stores, music stores) such as this one which never saw the light of day. All of the brainstorming culminated in a final direction that we were all satisfied with, and later we would see the payoff of exploration again by using multiple paper prototypes instead of a single one.

Overall, we learned that prototyping early and going through multiple rounds of user testing is very helpful in increasing the learnability of the final prototype. Finding usability problems early in the paper prototyping stage allowed us more flexibility when designing the computer prototype. Due to the fact that we had conducted multiple rounds of user tests (totaling over 7 users) with the paper prototypes, we were quite successful with user testing for the computer prototype - no major implementation changes were required for the computer prototype. Users specifically noted that we had taken their earlier suggestions during computer prototype user testing.

In addition, having multiple paper prototypes enabled us to request feedback on each one, combine the benefits of each, and form an ultimately more user-friendly application interface. Some users noticed that we combined the best of all paper prototypes in designing the computer interface. On the other hand, paper prototypes failed to indicate the complexities of implementation issues - many features we drew up quickly during the paper prototyping stage in fact takes much longer to implement during the computer prototyping stage. For instance, the overlay feature of "Drag and Drop" was easily prototyped using transparencies and highlighters, but required major time commitment to code in Javascript/CSS. On the other hand, this fact also was informative in so far as it reminded us that a good design is not simply a matter of putting together existing UI components. Thought and care must be taken during the design process to allot time to develop complex, but ultimately easier to use interfaces.

If we were to do this project again, we would like to conduct earlier computer prototype user testing, especially with the uploading interface. During GR6 user testing we discovered that there were minor deficiencies in the current interface that only music directors could spot. This is partly because fellow classmates are not familiar with the needs of WMBR music directors (such as marking tracks as containing obscene words with the FCC flag). Furthermore, we could have done better task prioritization, focusing on key components of the system earlier (uploading files and editing details) as they are central to the KaJaM! application.

Finally, we discovered that user testing can be challenging because users often feel pressure to "do the right thing", rather than identify usability issues. There were instances where users became less communicative as they became frustrated with completing a scenario task. It was an important cue for us to be patient and re-assuring them that the problems lie in our design, and not their competence. We learned to accept criticism objectively and recognize that "the user is always right".

Back to the top