...
We first found our users by going to the Beehive Cooperative to find people in startups to test our product. Our user test consisted of several tasks, separated into two sections. The first section required the user to create and use their own account, while the second section had users test on a sample account of ours. Testing on a sample account allowed users to experience having many contacts, and not have to create many on their own to save time. We did not use a demo, but in retrospect, it would have been useful to create a demo video.
We briefed the users by presenting our problem statement from previous GR’s, and presented the users with the following tasks:
...
- Learnability
- All 3 users weren't able to figure out what to do after the signup page (should’ve authenticated Twitter account, 1 and 3 attempted to change password). This is a catastrophic error, and we would correct it by directing them directly to a page to only do authorizations, and then direct them to the grid.
- User 1 attempted to use comma delimited tags, while user 2 tried space delimited tags. This is a major error, and could be corrected by a demo video.
- Users 1 and 3 didn’t realize you have to save tags using the “Save contact” button. This is a major error, and could be corrected with an alert as the user left the page to indicate they needed to save, as well as a demo video
- User 1 didn’t find resizing intuitiveUser 1 didn’t notice the tag search as separate from the name search. This is a major error, and could be corrected by a demo video
- Users 2 and 3 weren't sure what to do about “add contact” or “grid”. This is a major error, and could be corrected by a demo video
- User 2 found “add contact” to be out of place next to grid, thought it should be next to filters. This is a minor error, and could be moved to place it on the same hierarchy as the filters on the grid page.
- User 3 thought history was tied to interaction medium. This is a major error, and could also be corrected by a demo video
- User 3 thought you could drag around contacts on the grid page. This is a major error, and could be implemented
- Safety
- User 1 found that confirming leaving a left the details page without saving was necessary. As previously mentioned, this could be corrected with an alert dialog, mentioning they haven't saved
- Users 1 and 2 needed to authenticate, but didn’t and ran into errors when trying to engage with customers. This is a catastrophic error, and could be corrected by requiring authorization immediately
- Efficiency
- User 1 was interested in last contacted date search. This is a minor error, and could be corrected by being implemented, perhaps as an "advanced search"
- User 1 wanted resizing in place. This is an error that is something we will sacrifice, in order to maintain the grid.
- User 2 wanted to search by content, such as emails. This is a minor error, and could be corrected by being implemented, perhaps as an "advanced search"
- User 3 wanted to read whole tweets in the history. This is a minor error, and could be corrected by using appropriate resizing, or perhaps by allowing overflow.
- User 3 expressed concerns that tweets were not automatically linked. This is a major error, and could be implemented.
- User 3 wanted to relate size of card on grid to priority, as well as metadata (ie tags). This is a minor error, and could be corrected by being implemented, perhaps as an "advanced search"
For the most part, the users were able to very easily pick up the features of the application, once they actually saw an action happen once. For example, once the user found the resizing affordance, they were easily able to understand how to resize contacts. Thus, we feel like a demo video, as mentioned previously, would be extremely useful to helping the user overcome these issues.
Reflection
One of the major lessons learned through implementing ContactLens was the balance of breadth and depth. Because of the vast number of APIs we were accessing, we quickly realized that we would not be able to implement a fully functional product in the timeline given. One of the difficult design decisions was to leave several of the APIs unimplemented. However, there was a large sunk engineering cost in the first week of the project setting up the product to be ready for use with all of the APIs.
...