Midnight Exchange - User Testing

Design

The final design of our interface comprises four tabs and one pop-up.  Our interface diverges dramatically from our original paper prototypes.  We made many design decisions that we felt would contribute to learnability, simplicity, and efficiency.

Design Decision I: Combined Tabs and Moved to Top 

Our paper prototypes had six tabs arranged in a somewhat arbitrary order on the left-hand side, while our final design has four tabs arranged horizontally above the content.  This change dramatically increased efficiency and learnability.The original tab names were CurrentAssignedMidnights, CurrentOpenTrades, Contracts, CurrentOpenSwaps, Watchlist, and LogOut.  While these names and their associated pages allowed for better specialization, this specialization proved to be unnecessary given the nature of the underlying environment, size of the user population, and scope of the content; indeed, we found it more efficient and intuitive to combine the information from certain tabs.

The new tab names are Home, Schedule, Quotes, and WatchList.  Home includes content from CurrentAssignedMidnights, CurrentOpenTrades, and Contracts.  A user is initially assigned to no more than three midnights a week; this is due to the fact that the user population is approximately 30, and there are 77 open spots.  Given that each midnight takes, on average, half an hour to complete, we believed that a rational upper bound on the number of midnights a user was assigned to, or traded for, was ten; this is quite a small number, and a list of all ten would not be large enough to merit its own page.  Similar logic dictated that the typical user would be involved in only a handful of trades, and have completed a handful of contracts.  Thus, the efficiency of the website could be much improved if all of this data were displayed on the same page.

We chose to put this information into the home page because it is the information any user (with the exception of a user whose goal is pure speculation, which we do not believe will happen) would wish to see before doing any actions: what midnights he is assigned, what midnights he is trying to trade for, and what midnights he has successfully traded.  By pulling it all together and presenting it in a compact manner, the new design allows for two distinct and common tasks: a user at home can analyze his current situation and decide to take action from there, all without navigating to a different page; or, a user on-the-go can quickly glean updates.  The old interface would require a considerable amount of navigation to achieve the same tasks.

We changed the arrangement of the tabs from vertical and on the left to horizontal and at the top.  We were able to do this because we had a smaller number of tabs, and so only one row would be required.  This proved to help the aesthetics of our interface, and also saved valuable screen real-estate.  Our design is, overall, very horizontally centered.  Left-aligned tabs would have forced the tables in Schedule, Quotes, and WatchList to shrink in size (thus reducing visibility), and would have made the home page much less visually pleasing.

Design Decision II: Nested Menus Replaced by Tables

Tables are ubiquitous in our new design: we opted away from nested menus and lists in the name of efficiency, visibility, and simplicity.

We originally had the market values for midnights presented as a set of nested menus (in the tab CurrentOpenTrades).  We replaced it with the table in the Quotes page.  There were several reasons for this change.

First of all, we changed the way MidnightExchange ran in order to more accurately reflect a typical exchange.  Instead of showing all the bid/asks for a midnight, their values, and their proposing parties, we chose to display only the best bid/asks (thus, the highest bid and lowest asks) and to preserve anonymity.  This was an important and correct decision.  The typical rational user need not see all the bids/asks on a midnight if he is seeking to either buy the labor for it, or sell his labor for it; rather, he only wants to see the best deal that he can get.  Anything extra information would detract from both efficiency (more reading/scrolling) and visibility (harder to distinguish the important information from the information that the server only needs to know).

This means that, since only the best bid/ask is being displayed, there is a constant amount of information; furthermore, we limited the maximum bid/ask values to $99.99 USD.  It was only natural, then, to use the table: the rigidness of the boundaries would not create a problem, since the content is always fixed in length.  Also, since the schedule itself is a table, we took advantage of consistency to positively affect learnability.  The new layout is more efficient, learnable, and aesthetically more pleasing than the older drop-down menu ridden design.  It is also more fault tolerant: a malicious user could have rendered the old interface unusable by flooding the system with unrealistic bids and asks: these would be displayed, but nobody would ever accept them.  Now, our design will not show such offers, though the back-end maintains them.

Design Decision III: Bid/Ask Table

An issue that came up along every step of the development of the UI was how to concisely communicate exactly what constituted a "bid" or an "ask."  Our official definitions are as follows.  A "bid" is a bid for labor: that is, were I to bid $20 for Monday Pots, I'd be willing to pay up to $20 for someone's labor for the midnight.  It follows that an "ask" is an ask for labor: that is, were I to ask $25 for Monday Pots, I'd be willing to accept no less than $25 for my labor for that midnight.

The confusion arises from the fact that a user who buys labor for a midnight is no longer assigned to it; thus, superficially, it seems that the user has lost something that belongs to him.  The rationale is that chores are un-wanted; thus, you have to pay someone to give them your chore.

Our final design does not explicitly give these definitions anywhere.  Instead, it only indicates bids and asks on the quotes table.  We assumed that, given the small user population, it would be possible to communicate this some other way.  We chose these definitions because it makes the most sense.  The opposite definition would imply that a user was paying money to do more work, which cannot seem rational to most intended users.  While we could have easily added a FAQ or README tab, we believed that this would be poor taste and decided against it.

Design Decision IV: Ability to Trade from Most Pages

We promoted user control and efficiency by allowing users to initiate trades from any page.  Users can initiate trades on midnights they are assigned to from their home page; they can initiate trades on any midnight from the schedule and quotes page; finally, they can initiate trades on midnights on their watchlist from the watchlist page.  We chose to offer such functionality because it was a good balance between simplicity and user control.

The user accesses the home page for one of two reasons: he has just navigated to MidnightExchange in his browser, or he wishes to see only the midnights and trades pertaining to himself.  Although the user could have the most control if we allowed him to trade for any midnight, doing so would detract from visibility and efficiency.  Users on the home page will likely care most about trading away their assigned midnights, if at all; were we to allow them to trade for any midnight on the home page, we'd have to implement some way for the user to choose from the very large selection of midnights.  The home page does not have a table interface like the other pages; thus, any implementation would take up extra room and require the user to do more work to find the midnights he is interested in.  The schedule and quotes pages are only one click away and offer full functionality.

We allow users to trade for any midnight on the schedule and quotes pages because the tables therein lend themselves very well to such a task.  Organization has already been explicitly defined by the table structure.  The only work necessary is the visual affordance for clickability.  

Finally, we allow the user to trade for midnights on their watch list from the watch list page because that is why the watch list page exists.

Implementation

There were a few important design decisions we made on the implementation level.

Design Decision I: Store All Bid/Asks, But Display Only the Best

We mentioned this earlier.  Our original design showed all the bids, along with the usernames of the people who posted them.  Now, we chose to display only the best.  This made our design easier to implement and better from a usability standpoint.  The constant content size allowed for the reuse of the table interface that we'd use for the schedule page.  Furthermore, the 'click-to-offer' pop-up interface was much simplified by having a fixed-length table.  The usability aspect was discussed earlier; the only extra point we'll make is that anonymity is important on a fair market.

Design Decision II: Display Prices in Tables Similar to How Midnights are Displayed

Although we changed the way MidnightExchange ran (mentioned above) to more accurately reflect a stock exchange, we chose to display the information in a more user-friendly manner than the stock exchange (i.e. in the same way the schedule page displayed assigned midnights).  Again, this was easier to implement and had positive usability repercussions.  For the majority of users, the display bears more resemblance to the real world, and is thus easier to learn.

Design Decision III: No Ajax

We couldn't figure out how to use Ajax in time, so we didn't.  Thus, while the page refreshes when the user himself puts a bid on the market, it doesn't refresh when other users put bids on the market.  This is a potentially catastrophic usability issue, since using old market data can very easily lead to inconsistent pricing and misinformed decisions.  Clearly, there was no incentive for us to avoid Ajax except that we couldn't figure it out.

Design Decision IV: Yes, PHP and mySQL

We could figure out how to use PHP and mySQL in time, so we did.  This meant that we could maintain user data on-line.  This is obviously very important if the interface is actually used in real life, since a midnight exchange whose transactions are ephemeral is useless.  The implementation was a little tricky, but definitely worth it.

Evaluation

We conducted the user test at 8:00 PM on 11 May, 2011 at the fraternity house for which the interface is built.  The target user population, brothers of the fraternity, was the same as the that of  the environment in which the user tests took place.  Our tasks involved putting a bid on the market for an assigned midnight; thus, only users whose usernames appeared on the schedule could be used.  We added every user on the schedule to a list (no repetition), and chose three randomly.  We asked them if they'd like to be part of the user test, and had a one-hundred percent success rate.  The first user was a senior double majoring in math and management.  We deem him an expert user, as he had extensive work in finance (internships at top-tier banks).  The second user was a senior majoring in political science, and the third a sophomore majoring in 6-1.  Neither had financial background, and so we deem them average users.  The first user had the least trouble understanding and remembering the definitions of bid/ask.

Process

We followed the following set of steps with each user test.

1. The facilitator read the briefing to the user, then gave the user as much time as necessary to familiarize himself with the interface.
2. The facilitator read each task sequentially.  The facilitator would wait for the user to complete the task before proceeding, providing guidance as requested.
3. The observer recorded any vocal feedback provided by the user.

Roles

• Timothy Peng was the facilitator.
• Stephanie Chan and Yang Yang were observers.

Briefing

Hello there!  Thanks for agreeing to user test our interface.  Our interface attempts to provide stock market-esque functionality to the midnight process.  Please go to this URL.  Ready? stepcie.scripts.mit.edu/index.php. You will see that, after logging in with your certificate, you'll be able to see what midnights you've been assigned, as well as the schedule, the current market, as well as a watch list.  Please note that these are the two definitions that we'll be using in our website.  A bid means that you'd be willing for someone to do the labor for you: so, if you had Monday waitings and couldn't make it, you'd want to put a bid out.  An ask is how much you'd be willing to sell your labor for; so, if you didn't have anything to do on Thursday and enjoy waitings, you could put an ask out on it.  Now, we're going to give you a few tasks to do.  Take some time now and familiarize yourself with the interface, you can click and do whatever, then let me know when you're ready.  Also, any issues you may have with the interface are not your fault, but the interface's.  As you do the tasks, please be as vocal as possible.  So speak like you're talking to yourself out loud and we aren't here to hear it.

Demo

There was no demo.  We just gave the user as much time as necessary to familiarize himself with the interface.  We thought this would be a good idea since it was the most likely thing to happen in the deployment of the website.

Tasks

1. See what midnights you've been assigned for the week.
2. Put a bid on the market for one of the midnights you've been assigned.
3. Put an ask on the market for one of the midnights you've not been assigned.
4. Verify that the bid and ask have successfully been entered, and if so, check to see if you have the current best bid and ask.
5. Add all Thursday midnights to your watch list, then remove some.

Usability Problems Found