...
Finally, we allow the user to trade for midnights on their watch list from the watch list page because that is why the watch list page exists.
...
Implementation
There were a few important design decisions we made on the implementation level.
Design Decision I: Store All Bid/Asks, But Display Only the Best
We mentioned this earlier. Our original design showed all the bids, along with the usernames of the people who posted them. Now, we chose to display only the best. This made our design easier to implement and better from a usability standpoint. The constant content size allowed for the reuse of the table interface that we'd use for the schedule page. Furthermore, the 'click-to-offer' pop-up interface was much simplified by having a fixed-length table. The usability aspect was discussed earlier; the only extra point we'll make is that anonymity is important on a fair market.
Design Decision II: Display Prices in Tables Similar to How Midnights are Displayed
Although we changed the way MidnightExchange ran (mentioned above) to more accurately reflect a stock exchange, we chose to display the information in a more user-friendly manner than the stock exchange (i.e. in the same way the schedule page displayed assigned midnights). Again, this was easier to implement and had positive usability repercussions. For the majority of users, the display bears more resemblance to the real world, and is thus easier to learn.
Design Decision III: No Ajax
We couldn't figure out how to use Ajax in time, so we didn't. Thus, while the page refreshes when the user himself puts a bid on the market, it doesn't refresh when other users put bids on the market. This is a potentially catastrophic usability issue, since using old market data can very easily lead to inconsistent pricing and misinformed decisions. Clearly, there was no incentive for us to avoid Ajax except that we couldn't figure it out.
Design Decision IV: Yes, PHP and mySQL
We could figure out how to use PHP and mySQL in time, so we did. This meant that we could maintain user data on-line. This is obviously very important if the interface is actually used in real life, since a midnight exchange whose transactions are ephemeral is useless. The implementation was a little tricky, but definitely worth it.
...
Evaluation
We conducted the user test at 8:00 PM on 11 May, 2011 at the fraternity house for which the interface is built. The target user population, brothers of the fraternity, was the same as the that of the environment in which the user tests took place. Our tasks involved putting a bid on the market for an assigned midnight; thus, only users whose usernames appeared on the schedule could be used. We added every user on the schedule to a list (no repetition), and chose three randomly. We asked them if they'd like to be part of the user test, and had a one-hundred percent success rate. The first user was a senior double majoring in math and management. We deem him an expert user, as he had extensive work in finance (internships at top-tier banks). The second user was a senior majoring in political science, and the third a sophomore majoring in 6-1. Neither had financial background, and so we deem them average users. The first user had the least trouble understanding and remembering the definitions of bid/ask.
Process
We followed the following set of steps with each test user . We made sure that the test environment was as standardized as possible.
- The facilitator reads read the briefing (see Briefing section) to the test userto the user, then gave the user as much time as necessary to familiarize himself with the interface.
- The facilitator reads read each task one by one (see Tasks section). After each task is read, the user attempts sequentially. The facilitator would wait for the user to complete the task while vocalizing their feedback. Where appropriate, the facilitator encourages the user to provide feedbackbefore proceeding, providing guidance as requested.
- The observer records the feedback given recorded any vocal feedback provided by the test user.
Roles
- Susie Fu facilitated the test.
- Emily Zhao observed the users.
- Ashutosh Singhal was on call to perform last minute bug fixes.
Briefing
Our application is Wishdex.com, a site that helps you keep wishlists. You can keep track of items you find online, show your friends and family gifts you want for Christmas or your birthday, and see what your friends are interested in or check out popular items on the site. We're doing this test to get some feedback on how well we've designed the user interface. There are probably a lot of problems with the design, and we need help finding them. Keep in mind that we're testing the computer system, not you. Also, the results of this test will be kept completely confidential, and you can stop and leave the test at any time. My name is Susie, and I'll be reading out the tasks we want you to perform. This is Emily, and she'll be taking some notes to help us remember the problems we find.
Note that we did not use a demo as part of our briefing. The motivation for this was that we wanted our site to be usable without a demo to the test users we were able to find. We designed Wishdex.com to be a site that someone who falls under our target test definition can go to and instantly be able to use.
Tasks
- Timothy Peng was the facilitator.
- Stephanie Chan and Yang Yang were observers.
Briefing
Hello there! Thanks for agreeing to user test our interface. Our interface attempts to provide stock market-esque functionality to the midnight process. Please go to this URL. Ready? stepcie.scripts.mit.edu/index.php. You will see that, after logging in with your certificate, you'll be able to see what midnights you've been assigned, as well as the schedule, the current market, as well as a watch list. Please note that these are the two definitions that we'll be using in our website. A bid means that you'd be willing for someone to do the labor for you: so, if you had Monday waitings and couldn't make it, you'd want to put a bid out. An ask is how much you'd be willing to sell your labor for; so, if you didn't have anything to do on Thursday and enjoy waitings, you could put an ask out on it. Now, we're going to give you a few tasks to do. Take some time now and familiarize yourself with the interface, you can click and do whatever, then let me know when you're ready. Also, any issues you may have with the interface are not your fault, but the interface's. As you do the tasks, please be as vocal as possible. So speak like you're talking to yourself out loud and we aren't here to hear it.
Demo
There was no demo. We just gave the user as much time as necessary to familiarize himself with the interface. We thought this would be a good idea since it was the most likely thing to happen in the deployment of the website.
Tasks
- See what midnights you've been assigned for the week.
- Put a bid on the market for one of the midnights you've been assigned.
- Put an ask on the market for one of the midnights you've not been assigned.
- Verify that the bid and ask have successfully been entered, and if so, check to see if you have the current best bid and ask.
- Add all Thursday midnights to your watch list, then remove some.
- Log in
- Managing
- Create a new wishdex called "Birthday Wishlist"
- Add new item from Anthropologie.com to this wishdex
- After adding the item, change the item description
- Create another wishdex "Christmas" and move this item to the 2nd wishdex
- Share "Christmas" with your mom
- After some thought, you decided to buy this item on Anthropologie.com. You still want to keep the item on your wishdex, though, but you want to mark that you've acquired it.
- Delete an item
- Exploring
- Browse popular wishdexes on the Popular page
- Check out the recent activity
- Friends
- Find Emily's wishdex, "Cool Patterns"
- Like one of the dresses in her Wishdex
- You want to buy this item for her, so claim the item and view the item on the Anthropologie.com website Copy an item that you like into your own "Birthday Wishlist" wishdex
Usability Problems Found
Severity | Description of Problem | Possible Solutions |
|---|---|---|
Cosmetic | Finding Wishdex.com - One user went to wishdecks.com instead of wishdex.com. | Buy both domains |
Minor | Log In - One had deactivated her Facebook account. She had also let a friend log into Facebook on her computer, so she had to log that friend out first. | Allow users to create an account and then later possibly link to Facebook. |
Major | Log In - Two users were reluctant to sign on using Facebook Connect. We had to reassure them that we were only pulling their full name and their profile picture. | Put a tooltip near the login with a disclaimer that we will not interfere with their Facebook lives or take any personal information. |
Catastrophic | Add Item - One of our users accessed the Anthropologie UK site. Although the item information scraping code works for the US site, it failed for the UK site. | We need to improve the scraping system immensely. This was a warning that unexpected things will break. We should look into APIs and smarter ways to get item data. |
Cosmetic | Edit Item - Every time the user edited the item description, an extra space automatically appeared at the beginning of the item description. | This is clearly a bug. We need to look at the code and if necessary strip the space on the frontend. |
Minor | Share Wishdex - With an item box open, it was difficult to find the "Share Wishdex" button. Many first tried "Item Link." | Ideally, users should be able to share individual items as well as entire Wishdexes. The backend functionality is already built in, we just need to add a button. |
Minor | Share Wishdex - Users clicked on the Share link a few times. They expected more to happen. | We should implement automatic selection and copy pasting, with a tooltip that says "copied." |
Catastrophic | Viewing - A user tried to click on the user's icon next to their comment. However, clicking on their name/image currently does not do anything. This can be potentially confusing or inconvenient for the user. | We can make the user's icon link to that user's Wishdex profile page. |
Major | Move Item - The user clicked on an item and tried to drag it into another Wishdex on the left navigation. | We should implement drag and drop to move items into other Wishdexes. |
Cosmetic | Copy item - Copy item is what we label the button used for the user to copy an item into one of their Wishdexes. The user found "copy" to be a misleading word. She said it implied copy and pasting, such as with a link. | We thought of changing the word "Copy" into the word "Add" with an icon that matched the "Add Item" icon, so maybe users will associate it with adding an item to their own Wishdex. |
Major | Like Item - Two users wanted to see who else had liked an item. | We should implement this. |
Minor | Copy Item - After copying an item from another user's Wishdex into his own Wishdex, the user was redirected to his Wishdex. He said it would be nice to stay on the friend's page. | We are not convinced that all users will feel this way. We would probably run a slightly bigger test to see what people prefer, or maybe implement some way of giving users a choice. |
Cosmetic | Move Item - A user tried to add an item from a different Wishdex by clicking on "Add Item" in the new wishdex. | We felt that drag and drop would probably clear up the confusing around moving items between Wishdexes immensely. We want to avoid implementing this too many times. |
Cosmetic | Explore - One user avoided hovering over the items on the Explore page. Instead, she pressed the arrows on the sides and did not discover the hover function. | One suggestion is to have a tool tip over the items the first time a user visits, telling them to hover over the items. Once a user tries this once, they often remember it. | Cosmetic | Claim - When trying to go to the item's webpage, one user first thought of clicking the item name. She was hesitant to click "Buy" because she wasn't sure if it would immediately go to the buying page. | We are considering changing the word "Buy" into something more informative, or having the tooltip be visible by default.
Major | Batch Removal of Midnights from Watch List. The user found that there was no support for removing more than one midnight at a time from the watch list at a time. | Use checkboxes instead of 'x's. Then have a "clear all" button at the bottom. |
Major | No Log Out. The user wished to, but couldn't figure out how to, log out. | Include a logout button that will log the user out. |
Minor | Quotes Page Aesthetics. The user felt that the quotes page was very information-dense, and suggested color coding bids and asks. | Color code bids and asks. |
Good | Very Nice. The user felt that the interface was very nice. | None. |
Major | Confusion Over Buy/Sell. Despite verbal guidance, the user was still confused by the bid/ask terminology. This happened to two users. | This is a problem we have been trying to deal with ever since the beginning. We don't have a solution yest, other than some sort of help section. It should be a one-time problem for any user, though. |
Major | Unintuitive Enter Press. The user pressed the enter key to complete a bid, but instead, the bid was cancelled. We were able to replicate this event, though not with a 100% success rate. | Something fishy going on in the back-end. Need to explicitly say what happens when the enter key is pressed. |
Major | Button Relocation. Sometimes the buttons on the bid ticket pop-up would appear outside of the boundaries of the pop-up. The user was able to realize this and complete the task, but still remarked upon it. | The way we chose to change the content of the pop-up led to this error. While the buttons could move down, they didn't move back up. We need to fix the back-end to compensate. |
Minor | Text Field Relocation. In certain windows, the text box for entering prices on the bid ticket pop-up would appear on the next line, confusing the user. | Increase the bid ticket size. |
Minor | Watch List Ordering. The user noticed that after deleting an item from the watch list, the order of the items in the watch list would change. | We can't figure out the reason for this, but a possible solution is to re-sort the list after deletion of an item. |
Good | Table Formatting. The user liked the table formatting, and also that his username was bolded. | None |
Cosmetic | Table Formatting. The user thought it'd be nice to change the color of the column corresponding to the current day of the week. | Change the back-end to do this, though we feel it is unnecessary. |
Major | No FAQ. The user looked for a FAQ while exploring the website and didn't find one. | Take advantage of the fact that the user might actually want a FAQ and put one in. This can help alleviate the bid/ask distinguishment problem. |
...
Reflection
Our interface was designed for the brothers of ZBT, a very specific audience that is more or less an experienced user in the realm of trading midnights. Yet we found ourselves constantly criticizing the design for being too specific in terms of language. Fundamentally, we were trying to merge the midnight trading concept with that of financial trading, which turned out to be more of a challenge than we had initially anticipated.
Some questions we found asking often was “Would all brothers know what a bid/ask is?”, or “Does buy market and buy limit sound intuitive for brothers with no financial exposure?” As a compromise, we ended up keeping some trading terminology while removing others. Even then, some brothers who did have trading experience confused the notion of buying and selling labor rather than midnights.
Another issue that our group realized was that designing for the UI often affects how the backend is implemented. During GR4, we cut corners in the code by relying on some visibility hiding tricks to give the illusion that we in fact swapped pages. In turns out that while this was extremely fast to prototype, it limited the capabilities of the back button in the browser. We had to rethink some of our strategies during GR5 in order to get the backend to work, given our coding decisions in previous iterations.
All in all, we learned a lot through the process. We were somewhat surprised by the number of times that we scrapped our design and started from the beginning, but we feel that this was necessary, since our final design was quite different from the original, and we couldn't see how it could have evolved as such. What we found the most useful was asking other people what they thought. We sought input from a variety of people: people not in the class, people not in course 6, graduates of the class, etc. It's amazing how drastically different people's views can be.
We took a few design risks in this project. As has been mentioned time and time again, we avoided explicitly stating what a bid/ask was. This clearly came back to bite us, and if we could redo the project, we'd just throw in a FAQ page to take care of it, regardless of whether or not such things are poor taste. If we could redo the project, however, we'd also do a few other things differently. We'd definitely keep seeking more input from not just the target audience, but everyone. While there are certainly aspects that input from the target audience is essential for, the general population is also quite useful for input. We'd also have spent more time in the beginning coming up with, and nurturing, different ideas (as mentioned earlier, we started from zero a lot).
We tested our interface by appealing to our friends to test it for us. This definitely was a good choice, since they were very diverse and provided useful feedback. Keeping an open mind was one of the greatest contributors to this interface's success.
...
Acknowledgments
Special thanks to Sean Chang and Mason Tang for their inputEvaluation
We conducted the user test at 8:00 PM on 11 May, 2011 at the fraternity house for which the interface is built. The target user population, brothers of the fraternity, was the same as the that of the environment in which the user tests took place. Our tasks involved putting a bid on the market for an assigned midnight; thus, only users whose usernames appeared on the schedule could be used. We added every user on the schedule to a list (no repetition), and chose three randomly. We asked them if they'd like to be part of the user test, and had a one-hundred percent success rate. The first user was a senior double majoring in math and management. We deem him an expert user, as he had extensive work in finance (internships at top-tier banks). The second user was a senior majoring in political science, and the third a sophomore majoring in 6-1. Neither had financial background, and so we deem them average users. The first user had the least trouble understanding and remembering the definitions of bid/ask.