Anchor | ||||
---|---|---|---|---|
|
Note: We created two separate designs for user testing, one which took a more user-friendly approach and another which focused exclusively on efficiency. After the first round of testing (which consisted of three music directors and an "elf" from the radio station) we dropped the design which focused on efficiency and merged the best design concepts from both into a single design. Because we ran each user in the first round on both designs (we counterbalanced so that an equal number of users started with either design) there was a significant learning and preference bias (the tasks remained the same across the two designs, therefore users were more comfortable by the time they tried the second design). This may have been an influence on the users' unanimous preference for the second design they were presented with. Nonetheless, we found it a useful exercise to use multiple designs in our first round of testing, since many details from both made it into the final design.
Quick Links |
---|
Prototype Photos
Anchor | ||||
---|---|---|---|---|
|
| v1.0 of user-friendly implementation |
| v1.0 of user-friendly implementation |
| v1.0 of high-efficiency implementation |
| v1.0 of high-efficiency implementation |
| v1.0 of reporting interface (both aforementioned v1.0 implementations use the same reporting interface) |
| One of the user testing sessions at the WMBR studio. |
| Some of the interface widgets and task cards used. |
| v2.0 of our merged implementation |
| A view of the testing session on Monday March 18th. |
| v2.0 of reporting interface |
Briefing
Anchor | ||||
---|---|---|---|---|
|
What follows is the briefing presenting to participants in the paper prototype tests:
Panel | ||
---|---|---|
| ||
Thank you for taking the time to help us test the prototype of our system. You will be acting as the “user”, our goal is not to test you, but to test our system and find flaws or inconsistencies that will improve the final implementation. One person from our group will act as a “computer” who will control the experiment and provide tasks that we would like you to perform using the interface. The other group members will act as “observers”, and will be taking notes on the experiment. We will be asking you to perform tasks that relate to your current music director duties, but using our interface instead of what you are normally used to. At a high level these tasks will include importing digital music, previewing music and getting CMJ reporting data. We ask that you do not ask interface related questions during the test itself (as this will better help us find faults in our interface) but we can answer anything after the test is complete. Thanks and have fun! |
Scenario Tasks
Anchor | ||||
---|---|---|---|---|
|
Scenario | Task |
---|---|
Imagine you are a music director, Lana, and you wish to upload some albums to the digital library via the KaJaM! interface | Open KaJaM! and log into the system |
You have downloaded 2 albums onto your computer: Starmarker.zip and KaJaM.zip and wish to import them to the library | Import Starmarker.zip and KaJaM.zip into the digital library |
While KaJaM.zip is importing, you wish to preview one of the tracks | Preview (play) a track in the KaJaM album |
The track from the KaJaM album sounds familiar, and you quickly realize you have already imported the album and don't need it again | Delete the currently importing KaJaM album |
The Starmarker.zip file finishes importing, and you wish to make sure all the track details are correct | Inspect track details* |
You find incorrect and incomplete track details you want to fix | Edit track details to reflect the correct information** |
You wish to approve the album to officially file it in the digital library | Approve the album |
Before you leave, you wish to complete College Media Journal reporting for the month | Take a look at the reporting data in the digital library |
You notice the reporting data is displaying _the Blue Dot _genre, but you want to look at "Electronica/RPM" | Filter the reporting data to show only "Electronica/RPM" |
Now all reporting data is on "Electronica/RPM" only, but it is not sorted by play count in the last month | Sort the reporting data by play count in the last month, in descending order |
Everything looks good, but you still want to make adjustments to the reports based on physical CD play count before submitting them | Export the reporting data to Excel file format for further editing |
Now that you are all done, you wish to go home | Done! |
*The album contains tracks with pre-tagged information, but some are incorrect or incomplete:
Track # | Track Name | Artist | Album | Label | Release Date | Genre |
---|---|---|---|---|---|---|
1 | While I'm Dead | Starmarker | Unknown | Polyvinyl |
|
|
2 | Sand Beauty | Starmarker | Unknown | Polyvinyl | ||
3 | Silvre Lining | Starmarker | Unknown | Polyvinyl |
**The facilitator assumes Lana Googles the album information, so facilitator provides all the correct information to the user on a separate sheet of paper:
Track # | Track Name | Artist | Album | Label | Release Date | Genre |
---|---|---|---|---|---|---|
1 | While I'm Dead | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
2 | Sand Beauty | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
3 | Silver Lining | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
Observations
Importing tasks
Anchor | ||||
---|---|---|---|---|
|
Legend (relevant for prototype iteration 1 only, note that reporting interfaces were identical in both designs for this iteration):
(U) - This point specifically references the user-friendly design
(E) - This point specifically references the high-efficiency design
Prototype Iteration 1
Prototype Photos
...
...
...
...
...
...
...
One of the user testing sessions at the WMBR studio.
...
...
Some of the interface widgets and task cards used.
...
...
...
A view of the testing session on Monday March 18th.
...
...
Briefing
KaJaMi Participant Briefing
Thank you for taking the time to help us test the prototype of our system.
You will be acting as the “user”, our goal is not to test you, but to test our system and find flaws or inconsistencies that will improve the final implementation.
One person from our group will act as a “computer” who will control the experiment and provide tasks that we would like you to perform using the interface.
The other group members will act as “observers”, and will be taking notes on the experiment.
We will be asking you to perform tasks that relate to your current music director duties, but using our interface instead of what you are normally used to. At a high level these tasks will include importing digital music, previewing music and getting CMJ reporting data. We ask that you do not ask interface related questions during the test itself (as this will better help us find faults in our interface) but we can answer anything after the test is complete.
Thanks and have fun!
Scenario Tasks
Scenario | Task |
---|---|
Imagine you are a music director, Lana, and you wish to upload some albums to the digital library via the KaJaM! interface | Open KaJaM! and log into the system |
You have downloaded 2 albums onto your computer: Starmarker.zip and KaJaM.zip and wish to import them to the library | Import Starmarker.zip and KaJaM.zip into the digital library |
While KaJaM.zip is importing, you wish to preview one of the tracks | Preview (play) a track in the KaJaM album |
The track from the KaJaM album sounds familiar, and you quickly realize you have already imported the album and don't need it again | Delete the currently importing KaJaM album |
The Starmarker.zip file finishes importing, and you wish to make sure all the track details are correct | Inspect track details* |
You find incorrect and incomplete track details you want to fix | Edit track details to reflect the correct information** |
You wish to approve the album to officially file it in the digital library | Approve the album |
Before you leave, you wish to complete College Media Journal reporting for the month | Take a look at the reporting data in the digital library |
You notice the reporting data is displaying _the Blue Dot _genre, but you want to look at "Electronica/RPM" | Filter the reporting data to show only "Electronica/RPM" |
Now all reporting data is on "Electronica/RPM" only, but it is not sorted by play count in the last month | Sort the reporting data by play count in the last month, in descending order |
Everything looks good, but you still want to make adjustments to the reports based on physical CD play count before submitting them | Export the reporting data to Excel file format for further editing |
Now that you are all done, you wish to go home | Done! |
*The album contains tracks with pre-tagged information, but some are incorrect or incomplete:
...
Track #
...
Track Name
...
Artist
...
Album
...
Label
...
Release Date
...
Genre
...
1
...
While I'm Dead
...
Starmarker
...
Unknown
...
Polyvinyl
...
...
...
2
...
Sand Beauty
...
Starmarker
...
Unknown
...
Polyvinyl
...
3
...
Silvre Lining
...
Starmarker
...
Unknown
...
Polyvinyl
**The facilitator assumes Lana Googles the album information, so facilitator provides all the correct information to the user on a separate sheet of paper:
Track # | Track Name | Artist | Album | Label | Release Date | Genre |
---|---|---|---|---|---|---|
1 | While I'm Dead | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
2 | Sand Beauty | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
3 | Silver Lining | Starmarker | Killer Kilometer | Polyvinyl | 20/02/2013 | Electronica/RPM |
Observations (Importing Tasks)
Prototype Iteration 1 | |||||
---|---|---|---|---|---|
User | Import Files into the Library | Preview Song Tracks | Delete Importing Album | Inspect and Edit Album/Track Details | Approve Album |
Music Director #1 |
| Learnability
|
Efficiency
- User's single click on the play button starts playing the track right away
Safety - If user clicked on play accidentally, a pause button appears in place of the play button to stop track | Learnability
- On the user-friendly interface, the user had some trouble locating the trash can because it was placed far away from the user's line of vision
- On the high-efficiency interface, User quickly found the "X" delete button beside the uploading album
Efficiency: - High-efficiency interface requires very little mouse movement to reach the "X" button, whereas the user has to move the cursor all the way across the screen in the user-friendly version of the interface.
Safety: - User prefers that the user-friendly interface gave a confirmation warning before deleting album | Learnability
- User found the both interfaces to be easy to understand in terms of what information is being displayed
- Both interfaces lack affordances in terms of what is editable and what isn't, user had to experiment by single clicking on the fields
Efficiency - User liked how both interfaces showed all the album and track information on one screen
- User noted the directly editable fields are very convenient
Safety - User noted a mistake in adding the genre can be easily fixed by clicking the "x" beside the genre tag | * User was confused about how to "approve" the album on both interfaces because he did not associate approval with the "checkmark" button (learnability issue)
- Again, a tradeoff was made between efficiency and safety: single click on the "checkmark" button will approve the album immediately (efficiency). User prefers a confirmation message (safety). |
Music Director #2
Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Music Director #3
Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:"Elf"
Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:Learnability:
Efficiency:
Safety:
...
Prototype Iteration 2
Advanced Tables - Table Plus | ||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||
|
|
...
Student User #1
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Student User #2
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Student User #3
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
...
Learnability:
Efficiency:
Safety:
Observations (Reporting Tasks)
Prototype Iteration 1 | ||||
---|---|---|---|---|
User | Switch to Reporting Mode | Filter View by Genre | Sort View by Play Count | Export Reporting Data to Excel File |
Music Director #1 |
| Learnability
| Learnability
|
|
Music Director #2 | Learnability: | Learnability: | Learnability: | Learnability: |
Music Director #3 | Learnability: | Learnability: | Learnability: | Learnability: |
"Elf" | Learnability: | Learnability: | Learnability: | Learnability: |
Prototype Iteration 2
|
Reporting Tasks
Anchor | ||||
---|---|---|---|---|
|
Prototype Iteration 1
User | Switch to Reporting Mode | Filter View by Genre | Sort View by Play Count | Export Reporting Data to Excel File |
---|---|---|---|---|
Music Director #1 |
|
|
|
|
Music Director #2 |
|
|
|
|
Music Director #3 |
|
|
|
|
"Elf" |
|
|
|
|
Prototype Iteration 2
User | Switch to Reporting Mode | Filter View by Genre | Sort View by Play Count | Export Reporting Data to Excel File |
---|---|---|---|---|
Student User #1 |
|
|
|
|
Student User #2 |
|
|
|
|
Student User #3 |
|
|
|
|
Prototype Iterations
Anchor | ||||
---|---|---|---|---|
|
Design | First Iteration | Second Iteration |
---|---|---|
Importing | The efficient design featured a text box and button, while the "friendly" design featured neither. | A text box added to the "friendly" design to provide affordances for pasting or typing URLs |
Managing Imports | The efficient design had a more visible "stop" button, next to the progress bar, while the "friendly" design placed its "delete" button far from the user's focus | The "friendly" design added a 'pause upload' button near the progress bar and changed the "No (do not approve)" button to a visible, red "Delete Upload" button |
Organization | The efficient design used a button to switch between upload and reporting modes and featured recent uploads on the same "upload" page, while the "friendly" design separated current from recent uploads, and used tabs for navigation | The "friendly" design reworded its tabs to be more clear as to their purpose |
Navigation | The efficient design assumed that scroll-bars would be used normally to scroll through recent uploads, while the "friendly" design used a button to scroll through the details while keeping the approve buttons static. | A scroll-bar replaced the "scroll-button" for scrolling through details in the "friendly" design. |
Reporting | The interface featured a single-genre drop-down box which required clicking on a "filter" button to filter. The release dates included and date ranges of playback counting were unclear. | Genre was converted to a Piazza-like tagging design much like that of the "friendly" design and removed the requirement to click on a "filter" button; release date ranges were made explicit and columns were added to clarify over which date ranges counts were collected |
Miscellaneous Features | N/A | Added a hover-based mechanism to view per-track artists for compilation albums; tooltips were added to clarify what each text field represented in the editor pane |
Prototype Iteration 2 | ||||
---|---|---|---|---|
User | Switch to Reporting Mode | Filter View by Genre | Sort View by Play Count | Export Reporting Data to Excel File |
Student User #1 | Learnability: | Learnability: | Learnability: | Learnability: |
Student User #2 | Learnability: | Learnability: | Learnability: | Learnability: |
Student User #3 | Learnability: | Learnability: | Learnability: | Learnability: |
Prototype Iterations
...
Design
...
First Iteration
...
Second Iteration
...
...
...
...
...
...
...
...
...