Conducting the User Test

The users were given a set of tasks to do which were similar to those from GR3, which covered the main use-cases and tasks which we expect our target population of users would do. One member of our team served as one of the test user's friends in order to provide the end-to-end communication between each other. We provided a sample device for the user to use, and gave them the option to download the application onto their personal devices if they chose to. We presented the user with a set of tasks to perform in order to observe them using Discover.Me in action.

Task List

  1. Launch the app and run through the tutorial.
  2. Given a chance to explore around the interface to acquant with the device.
  3. The user was told to search for a pre-specified person (Phil Mercer) in the user directory and to add him as a friend. 
  4. The user was told to inform us when he was aware when his Friend Request had been approved.
  5. The user was told to create an event and invite Phil to it, at a location he felt would be close enough for both himself and Phil. The scheduled time would be 30 minutes from the time at which the task was being performed.
  6. The user was told to inform us when he was satisfied and assured that Phil would be coming to an event.
  7. Phil (Our team member) would send an event to the user. The user was then told to propose a change to the event, and particularly to schedule it at a time 1hour away from the original time.
  8. The user was then told to inform us when he was satisfied that his proposed change had been accepted by the originator, Phil.

User Population

Users were picked randomly among friends and acquaintances who had not used the system before. We re-affirmed that they matched our target population by posing several questions with regards to how they went about organizing impromptu meetings. As an example, our facilitator posed the following questions:

  1. Have you ever been in a situation where you wanted to organize a quick, impromptu meeting amongst friends whom you knew were located relatively close to you?
  2. Describe, briefly, how would you go about doing so currently?
  3. How would you keep yourself informed of everyone's responses?
  4. How would you keep yourself informed of all proposed changes made by all your participants
  5. How would you enable all participants to each know the responses of every other participant
  6. How would you go about informing all participants of any proposed changed to the event.

Task Briefing

Users were given a quick overview of Discover.Me, which included the motivation for improving the creation of impromptu meetings, locating participant whereabouts  and being able to easily view the RSVPs of each participant. The users were also shown the video which we used for the Madness Demonstration, as it included an overall picture of Discover.Me, as well as a voice-over.

Usability Problems

Overview of Usability Issues Observed & Proposed Solutions

Presented with the Tutorial Screen

Finding a Friend in your Friend List or on the Map

Adding an Individual into your Friend List

Creating and Inviting Friends to an Event

Responding to an Invitation by Proposing a Change In Time

General Comments

  • The tutorial screen was welcomed by all our users. The only issue was with the visibility of the "Next" button.
     
    Solution: Instead of having both an arrow AND a next button, we could make the arrow itself a button and change its text to say "Click here for next tutorial". This will help serve double-duty, acting as both an instruction as well as providing the affordance of a button.
  • The user wanted a way to locate a friend on the map easier.

    Solution: Provide the ability to go to a friend's location via the Profile page, i.e. with a button stating "View Current Location".                                                                                                                                                                                                                                                                                                                                                                     
  • The on screen message confused some users as it said "Friend has been Added", when it actually meant that the request had been sent.

    Solution: Reword the on-screen message to say that the friend's request has been sent.
  • The users automatically knew that the notification received about the friend accepting the request was actionable, but there was a case where the user was then tempted to press "Delete Friend", as it was the button that stood out the most upon reaching the Profile page.

    Solution: Help safety by placing the delete functionality into the options menu (Activated by pressing the device's physical menu button) which would only then appear on the screen. We could also highlight the choice a different color like "Red" to bring the user's attention to that as an important choice, (i.e. externally consistent with what the Facebook mobile application does to indicate that a user wants to log out). We can also maintain safety by bringing up a confirmation dialog prior to actually deleting the friend.
  • Upon successfully adding a new friend, some users expressed an interest to know where the user was on screen. However, the newly added friend merely gets added onto the map without much feedback.

    Upon adding a friend, we can actually make the notification actually pan to where the newly added friend is. Also, we can do something like what other applications do and provide a sort of "dropping pin/dropping down" animation which would help bring attention to where the newly added friend is located.
  • The user wanted to be able to select a place by address or name rather than by selecting the location on the map.Solution: Provide a search bar to type in the desired location and place a marker on the map based on the input. * The user didn't notice the Done button when selecting a location on the map and instead hit the Back button.Solution: Make the done button more prominent.* The user wanted to be able to add notations about the event, such as specific room numbers.Solution: Provide a field to allow users to add messages/comments to the event.
  • The user was not aware of the Propose Change button and went to RSVP expecting Propose Change to be an option.

    Solution: Make the Propose Change button more prominent or include this functionality within the RSVP option.
  • The user wanted to give a reason behind the proposed change.

    Solution: Provide a field to allow users to comment why the change to the event was suggested.
  • The user wanted to differentiate between accepted events and those with a pending proposed change.

    Solution: Incorporate an indicator to inform the user that a proposed change is pending.                                                                                                                                                                                                                                                                                                                                                                  
  • The user wanted to block proposing changes within a certain time before the event.

    Solution: Allow the user to input a time after which they will not entertain changes.  If the current time is passed that then the Propose Changes button can be deactivated for the other users.                                                                                                                                        

Detailed Observations

User

Presented with the Tutorial Screen

Finding a friend in your Friend List or on the Map

Adding an individual into your Friend List

Creating and Inviting Friends to an Event

Responding to an Invitation by Proposing a Change in Time

User's General Comments

1

  • The user had no problems with the tutorial.
  • User exhibited no difficulties identifying the correct icon in the action bar, thanks to both the the tutorial and the correct affordance provided by the button.
  • The on-screen message said "Friend has been Added". The user was then surprised to find his friends list still empty.
    (Confusing the User)
  • When the notification bar displayed the counter, the user then understood that he had actually sent a request and had just received confirmation.
    (Good Visual Variable)
  • Expressed an interest in being able to specify a place by name, which would then pan the map to the location.
    (Efficiency)
  • The user hit the Back button instead of the Done button, losing his selected location.
    (Safety)
  •  Did not exhibit difficulty in performing the task -- found it intuitive to follow based on the application's general workflow.
    (Internal Consistency)
  •  Generally found the overall interface useful and intuitive.

2

  • The user was tapping on the arrow image pointing to the next button, as opposed to the next button itself, in trying to proceed to the next step in the tutorial. 
    (Confusing the User, Unclear Affordance)
  • The user wanted to have a way to search for a friend, either through the friend's list or a search bar. This would improve the efficiency of locating someone as opposed to manually tapping each marker.
    (Efficiency)
  • Upon receiving the notification that his friend had added him, the user proceeded to tap on the notification, as he saw the ">" arrow.
    (Good Affordance)
  • The user was tempted to press the "Delete Friend" button, and he said that it was the first thing that came to his mind when he saw the profile page.
    (Safety)
  • Expressed a wish to have free text area to add a some comments to the event invitation, much like most applicaitons that send invitations to others.
    (Matching the Real World)
  • Hoped for a way to be more specific in selecting a location (i.e. Room Number or a particular floor).
    (Matching the Real World, Efficiency, Visibility)
  • Expressed a wish to have free text area to add more information as to why a change was being proposed.
    (Matching the Real World)
  • Wanted to view events which he/she had proposed a change for in the Events List with some sort of indicator that they were pending.
    (Visibility)

  • Expressed a wish to be able to prevent users from proposing a change too close to an event.
  • Was generally interested in conveying more information with regards to his action to his friends.

3

  • The user had no problems with the tutorial.
  • The user expressed concern that in the event that many of his friends were clustered close to one another, that it would be hard to locate a particular friend. He later discovered that the map's zoom levels implemented direct manipulation for panning and zooming, and expressed satisfaction.
    (Learning by Exploring)
  • The user wanted to know where his newly added friend was after adding him. However, the friend took a while to locate where his friend was.
    (Efficiency)
  • Expressed an interest in being able to specify a place by name, which would then pan the map to the location.
    (Efficiency)
  • Did not not realize that the propose change button was there. Proceeded to RSVP "Yes" without making the change, thinking that the part where a suggested change came after the RSVP.
    (Visibility, User Model of System)
  • Initially thought ">" in the notifications list meant that the notification was new.
    (Visibility)
  • When asked about how he/she was able to distinguishing whether notifications were read/unread, user realized that the beige tint in the background was already triggering that knowledge unconciously.
  • No labels