Design
The final design of our interface comprises four tabs and one pop-up. Our interface diverges dramatically from our original paper prototypes. We made many design decisions that we felt would contribute to learnability, simplicity, and efficiency.
...
We followed the following set of steps with each test user . We made sure that the test environment was as standardized as possible.
- The facilitator reads read the briefing (see Briefing section) to the test useruser, then gave the user as much time as necessary to familiarize himself with the interface.
- The facilitator reads read each task one by one (see Tasks section). After each task is read, the user attempts sequentially. The facilitator would wait for the user to complete the task while vocalizing their feedback. Where appropriate, the facilitator encourages the user to provide feedbackbefore proceeding, providing guidance as requested.
- The observer records the feedback given recorded any vocal feedback provided by the test user.
Roles
- Susie Fu facilitated the test.
- Emily Zhao observed the users.
- Ashutosh Singhal was on call to perform last minute bug fixes.
Briefing
Our application is Wishdex.com, a site that helps you keep wishlists. You can keep track of items you find online, show your friends and family gifts you want for Christmas or your birthday, and see what your friends are interested in or check out popular items on the site. We're doing this test to get some feedback on how well we've designed the user interface. There are probably a lot of problems with the design, and we need help finding them. Keep in mind that we're testing the computer system, not you. Also, the results of this test will be kept completely confidential, and you can stop and leave the test at any time. My name is Susie, and I'll be reading out the tasks we want you to perform. This is Emily, and she'll be taking some notes to help us remember the problems we find.
Note that we did not use a demo as part of our briefing. The motivation for this was that we wanted our site to be usable without a demo to the test users we were able to find. We designed Wishdex.com to be a site that someone who falls under our target test definition can go to and instantly be able to use.
Tasks
- Log in
- Managing
- Create a new wishdex called "Birthday Wishlist"
- Add new item from Anthropologie.com to this wishdex
- After adding the item, change the item description
- Create another wishdex "Christmas" and move this item to the 2nd wishdex
- Share "Christmas" with your mom
- After some thought, you decided to buy this item on Anthropologie.com. You still want to keep the item on your wishdex, though, but you want to mark that you've acquired it.
- Delete an item
- Exploring
- Browse popular wishdexes on the Popular page
- Check out the recent activity
- Friends
- Find Emily's wishdex, "Cool Patterns"
- Like one of the dresses in her Wishdex
- You want to buy this item for her, so claim the item and view the item on the Anthropologie.com website
- Copy an item that you like into your own "Birthday Wishlist" wishdex
Usability Problems Found
...
Severity
...
Description of Problem
...
Possible Solutions
...
Cosmetic
...
Finding Wishdex.com - One user went to wishdecks.com instead of wishdex.com.
...
Buy both domains
...
Minor
...
Log In - One had deactivated her Facebook account. She had also let a friend log into Facebook on her computer, so she had to log that friend out first.
...
Allow users to create an account and then later possibly link to Facebook.
...
Major
...
Log In - Two users were reluctant to sign on using Facebook Connect. We had to reassure them that we were only pulling their full name and their profile picture.
...
Put a tooltip near the login with a disclaimer that we will not interfere with their Facebook lives or take any personal information.
...
Catastrophic
...
Add Item - One of our users accessed the Anthropologie UK site. Although the item information scraping code works for the US site, it failed for the UK site.
...
We need to improve the scraping system immensely. This was a warning that unexpected things will break. We should look into APIs and smarter ways to get item data.
...
Cosmetic
...
Edit Item - Every time the user edited the item description, an extra space automatically appeared at the beginning of the item description.
...
This is clearly a bug. We need to look at the code and if necessary strip the space on the frontend.
...
Minor
...
Share Wishdex - With an item box open, it was difficult to find the "Share Wishdex" button. Many first tried "Item Link."
...
Ideally, users should be able to share individual items as well as entire Wishdexes. The backend functionality is already built in, we just need to add a button.
...
Minor
...
Share Wishdex - Users clicked on the Share link a few times. They expected more to happen.
...
We should implement automatic selection and copy pasting, with a tooltip that says "copied."
...
Catastrophic
...
Viewing - A user tried to click on the user's icon next to their comment. However, clicking on their name/image currently does not do anything. This can be potentially confusing or inconvenient for the user.
...
We can make the user's icon link to that user's Wishdex profile page.
...
Major
...
Move Item - The user clicked on an item and tried to drag it into another Wishdex on the left navigation.
...
We should implement drag and drop to move items into other Wishdexes.
...
Cosmetic
...
Copy item - Copy item is what we label the button used for the user to copy an item into one of their Wishdexes. The user found "copy" to be a misleading word. She said it implied copy and pasting, such as with a link.
...
We thought of changing the word "Copy" into the word "Add" with an icon that matched the "Add Item" icon, so maybe users will associate it with adding an item to their own Wishdex.
...
Major
...
Like Item - Two users wanted to see who else had liked an item.
...
We should implement this.
...
Minor
...
Copy Item - After copying an item from another user's Wishdex into his own Wishdex, the user was redirected to his Wishdex. He said it would be nice to stay on the friend's page.
...
We are not convinced that all users will feel this way. We would probably run a slightly bigger test to see what people prefer, or maybe implement some way of giving users a choice.
...
Cosmetic
...
Move Item - A user tried to add an item from a different Wishdex by clicking on "Add Item" in the new wishdex.
...
We felt that drag and drop would probably clear up the confusing around moving items between Wishdexes immensely. We want to avoid implementing this too many times.
...
Cosmetic
...
Explore - One user avoided hovering over the items on the Explore page. Instead, she pressed the arrows on the sides and did not discover the hover function.
...
One suggestion is to have a tool tip over the items the first time a user visits, telling them to hover over the items. Once a user tries this once, they often remember it.
...
Cosmetic
...
Claim - When trying to go to the item's webpage, one user first thought of clicking the item name. She was hesitant to click "Buy" because she wasn't sure if it would immediately go to the buying page.
...
Reflection
Our interface was designed for the brothers of ZBT, a very specific audience that is more or less an expert user in the realm of trading midnights. Yet we found ourselves constantly criticizing the design for being too specific in terms of language. Fundamentally, we were trying to merge the midnight trading concept with that of financial trading, which turned out to be more of a challenge than we had initially anticipated.
Some questions we found asking often was “Would all brothers know what a bid/ask is?”, or “Does buy market and buy limit sound intuitive for brothers with no financial exposure?” As a compromise, we ended up keeping some trading terminology while removing others. Even then, some brothers who did have trading experience confused the notion of buying and selling labor rather than midnights.
...
- Timothy Peng was the facilitator.
- Stephanie Chan and Yang Yang were observers.
Briefing
Hello there! Thanks for agreeing to user test our interface. Our interface attempts to provide stock market-esque functionality to the midnight process. Please go to this URL. Ready? stepcie.scripts.mit.edu/index.php. You will see that, after logging in with your certificate, you'll be able to see what midnights you've been assigned, as well as the schedule, the current market, as well as a watch list. Please note that these are the two definitions that we'll be using in our website. A bid means that you'd be willing for someone to do the labor for you: so, if you had Monday waitings and couldn't make it, you'd want to put a bid out. An ask is how much you'd be willing to sell your labor for; so, if you didn't have anything to do on Thursday and enjoy waitings, you could put an ask out on it. Now, we're going to give you a few tasks to do. Take some time now and familiarize yourself with the interface, you can click and do whatever, then let me know when you're ready. Also, any issues you may have with the interface are not your fault, but the interface's. As you do the tasks, please be as vocal as possible. So speak like you're talking to yourself out loud and we aren't here to hear it.
Demo
There was no demo. We just gave the user as much time as necessary to familiarize himself with the interface. We thought this would be a good idea since it was the most likely thing to happen in the deployment of the website.
Tasks
- See what midnights you've been assigned for the week.
- Put a bid on the market for one of the midnights you've been assigned.
- Put an ask on the market for one of the midnights you've not been assigned.
- Verify that the bid and ask have successfully been entered, and if so, check to see if you have the current best bid and ask.
- Add all Thursday midnights to your watch list, then remove some.
Usability Problems Found
Severity | Description of Problem | Possible Solutions |
|---|---|---|
Major | Batch Removal of Midnights from Watch List. The user found that there was no support for removing more than one midnight at a time from the watch list at a time. | Use checkboxes instead of 'x's. Then have a "clear all" button at the bottom. |
Major | No Log Out. The user wished to, but couldn't figure out how to, log out. | Include a logout button that will log the user out. |
Minor | Quotes Page Aesthetics. The user felt that the quotes page was very information-dense, and suggested color coding bids and asks. | Color code bids and asks. |
Good | *Very Nice. *The user felt that the interface was very nice. | None. |
Cosmetic | Edit Item - Every time the user edited the item description, an extra space automatically appeared at the beginning of the item description. | This is clearly a bug. We need to look at the code and if necessary strip the space on the frontend. |
Minor | Share Wishdex - With an item box open, it was difficult to find the "Share Wishdex" button. Many first tried "Item Link." | Ideally, users should be able to share individual items as well as entire Wishdexes. The backend functionality is already built in, we just need to add a button. |
Minor | Share Wishdex - Users clicked on the Share link a few times. They expected more to happen. | We should implement automatic selection and copy pasting, with a tooltip that says "copied." |
Catastrophic | Viewing - A user tried to click on the user's icon next to their comment. However, clicking on their name/image currently does not do anything. This can be potentially confusing or inconvenient for the user. | We can make the user's icon link to that user's Wishdex profile page. |
Major | Move Item - The user clicked on an item and tried to drag it into another Wishdex on the left navigation. | We should implement drag and drop to move items into other Wishdexes. |
Cosmetic | Copy item - Copy item is what we label the button used for the user to copy an item into one of their Wishdexes. The user found "copy" to be a misleading word. She said it implied copy and pasting, such as with a link. | We thought of changing the word "Copy" into the word "Add" with an icon that matched the "Add Item" icon, so maybe users will associate it with adding an item to their own Wishdex. |
Major | Like Item - Two users wanted to see who else had liked an item. | We should implement this. |
Minor | Copy Item - After copying an item from another user's Wishdex into his own Wishdex, the user was redirected to his Wishdex. He said it would be nice to stay on the friend's page. | We are not convinced that all users will feel this way. We would probably run a slightly bigger test to see what people prefer, or maybe implement some way of giving users a choice. |
Cosmetic | Move Item - A user tried to add an item from a different Wishdex by clicking on "Add Item" in the new wishdex. | We felt that drag and drop would probably clear up the confusing around moving items between Wishdexes immensely. We want to avoid implementing this too many times. |
Cosmetic | Explore - One user avoided hovering over the items on the Explore page. Instead, she pressed the arrows on the sides and did not discover the hover function. | One suggestion is to have a tool tip over the items the first time a user visits, telling them to hover over the items. Once a user tries this once, they often remember it. |
Cosmetic | Claim - When trying to go to the item's webpage, one user first thought of clicking the item name. She was hesitant to click "Buy" because she wasn't sure if it would immediately go to the buying page. | We are considering changing the word "Buy" into something more informative, or having the tooltip be visible by default. |