Design

Implementation

Evaluation

User Testing

How it was conducted

We conducted our user tests via Skype and asked the users to share their screen with us, so that we could see where they were clicking.  We ran our website on one of our computers for the duration of the test and gave the users a URL to go to.

For our first two user tests, we had the users perform tasks both as an admin and as an instructor.  However, users in reality would only either be an admin or an instructor.  Our third user test involved the user only performing the instructor’s tasks in order to see what usability problems might arise from not having knowledge of the admin interface.

Our users

Since one of our group members (Julia) is an instructor at the small midwestern ski area for which we made our user interface, we were able to contact other instructors and have them be our users for this round of testing.  Therefore our users for testing are actual users in the user population.  They are also very representative of the user population in that their proficiency with computers vary.

Briefing

Thank you for volunteering to test our ski school scheduler.

Today we will ask you to perform tasks both as an normal instructor of the ski area and as an admin.  We’ll give you separate logins for each type of user and instruct you which logins to use at any given time.  The program requires admins to approve entire schedules at the beginning of the year.  After that, if an instructor wants to make a change to their schedule, each individual change must be approved by an admin.  Normal instructors are only permitted to view/make change requests their own schedules.  Admins can view and change all instructors’ schedules.

Do you have any questions?

[pause for questions]

You will greatly help us improve our interface if you think aloud as you go through the tasks we will give you.  We welcome any and all feedback, so please say anything that comes to mind!

Tasks

1. Log in as the instructor Sam (username and password sseaborn):## Create your initial schedule before the season begins. ## Submit your schedule for approval.## Log out when you’re done.
2. Log in as the admin Darryl (username and password dsmith):## Approve Sam’s schedule. ## Log out when you’re done.
3. Log in as Sam## Request to remove yourself from a shift.## Request to add yourself to a different shift.## Oops! You requested to add the wrong shift.  Delete that shift and add a different one.
   1. Log out when you’re done.
4. Log in as Darryl:## Approve one of Sam’s pending changes.## Reject one of Sam’s pending changes.## Mark Sam as being excused from one of his shifts. 
   1. Mark Sam as having been absent from one of his shifts.
   2. Log out when you’re done.

Usability Problems

Reflection