Design
Implementation
The biggest design decision we made for our implementation was to use the Django framework. We chose to use a framework in general, because it would allow us to make our application much more robust with much less code. We chose to use a database, because it provides an easy and persistent way to store data. We specifically chose Django, because it provides an excellent admin interface, which we could use without having to actually write it and which makes it possible to edit the database without being a technical person. This was important for our project because it makes it so that the administrators at the ski school could actually directly edit the database if need be even though they do not have any coding abilities. Django also provides authentication methods, which made implementing login very easy.
Additionally, Django models enables us to easily follow an MVC pattern, since the models make it easy to separate the model from the controller. Furthermore, we choose to keep all of our CSS in a separate file to separate out the view from the controller. By following MVC, it was much easier for us to make changes to the view as we got feedback from HW2, because many of the changes, namely the cosmetic ones, only required editing the CSS, but did not affect any of the logic.
We also used Django templates. This allowed us to inherit the heading and navigation bar and also allowed us to use the same views for the admin and instructor interfaces simply by adding in if-statements. All of these features of templating allowed us to write significantly less code than we otherwise would have had to write, which allowed us to implement more features and reduce the number of bugs that could be introduced.
We used javascript to implement much of the logic. We chose to draw much of the calendar with javascript, rather than doing it directly in the html. This makes our application unusable for users without javascript enabled browsers, which could be a usability problem, but since pretty much all modern browsers have javascript enabled, we don’t expect it to actually be a problem. We chose to implement the calendar in the javascript because doing so made it very easy to keep a copy of the working schedule in memory on the client side without editing the database. This made it very easy for users to cancel all changes in a session simply by going to another page (we do warn users before they do this), but also limited the ability of the application to recover any unsaved changes if the browser crashed.
The implementation decisions that we made allowed us to use many of the Django features, thus enabling us to focus on the most interesting user interface portions of our application design, while still providing a fully functional application.
Evaluation
User Testing
How it was conducted
We conducted our user tests via Skype and asked the users to share their screen with us, so that we could see where they were clicking. We ran our website on one of our computers for the duration of the test and gave the users a URL to go to.
For our first two user tests, we had the users perform tasks both as an admin and as an instructor. However, users in reality would only either be an admin or an instructor. Our third user test involved the user only performing the instructor’s tasks in order to see what usability problems might arise from not having knowledge of the admin interface.
Our users
Since one of our group members (Julia) is an instructor at the small midwestern ski area for which we made our user interface, we were able to contact other instructors and have them be our users for this round of testing. Therefore our users for testing are actual users in the user population. They are also very representative of the user population in that their proficiency with computers vary.
Briefing
We did not do a demo as part of our briefing.
For User Tests #1 and #2
Thank you for volunteering to test our ski school scheduler.
Today we will ask you to perform tasks both as a normal instructor of the ski area and as an admin. We’ll give you separate login names and passwords for each type of user and instruct you which logins to use at any given time. The program requires admins to approve entire schedules at the beginning of the year. After that, if an instructor wants to make a change to their schedule, each individual change must be approved by an admin. Normal instructors are only permitted to view/make change requests their own schedules. Admins can view and change all instructors’ schedules.
Do you have any questions?
[pause for questions]
You will greatly help us improve our interface if you think aloud as you go through the tasks we will give you. We welcome any and all feedback, so please say anything that comes to mind!
For User Test #3
Thank you for volunteering to test our ski school scheduler.
Today we will ask you to perform tasks as a normal instructor of the ski area. We’ll give you the login name and password for the instructor. The program requires admins to approve entire schedules at the beginning of the year. After that, if an instructor wants to make a change to their schedule, each individual change must be approved by an admin. Normal instructors are only permitted to view/make change requests their own schedules. Admins can view and change all instructors’ schedules.
Do you have any questions?
[pause for questions]
You will greatly help us improve our interface if you think aloud as you go through the tasks we will give you. We welcome any and all feedback, so please say anything that comes to mind!
Tasks
TASKS GO HERE
(For the third user test, we asked the users to perform the instructor's tasks 1 and 3, and one of us did the admin's tasks 2 and 4.)
Usability Problems
- CATASTROPHIC problem: Instructors have no idea what the red E and the blue A mean unless they have seen the admin interface.
- MAJOR problem: Users are unclear what the button with the white arrow (which reverts their cursor to a normal mouse cursor) does.
- MAJOR problem: Drawing a gray box around shifts that are pending add or pending delete does not convey the pending status to the user.
- MINOR problem: The majority of the users during user testing expected instructions.
- MINOR problem: It is easy to miss the "You must submit..." message at the top, since users' attentions tend to be drawn to the calendar.
- MINOR problem: On the admin interface, the users did not immediately notice the radio buttons used for accepting/rejecting requests.
- MINOR problem: The wording of the message on the popup box that appears if the user tries to navigate away from the calendar does not make it extremely clear that the user must press submit to save changes.
- COSMETIC problem: The color scheme (yellow/blue/black to represent day/evening/night) the buttons have when users schedule shifts is not extremely intuitive until the user has many shifts scheduled.
Reflection
Over the course of the iterative design process, we learned how important planning is. While had we not been forced to follow an iterative design process, we likely would not have done nearly as much planning near the beginning of our interface, we found that implementing our final version was much easier because of all of the planning we had put in early on. Furthermore, we learned the importance of user testing because there were things that we thought would be very intuitive, but that users sometimes couldn’t figure out. Since we found out many of these problems early on in our paper prototype, we were able to fix them much more quickly than had we not found them until after our final implementation. Finally, we learned the importance of testing on users that are representative of our user population. We found that our paper-prototype testers (who were all MIT students) often wanted features that would make the application more efficient to use; however, when we tested on our actual user population (much less technically savvy), their primary concern was just being able to figure out the interface. We did a good job of recognizing this and using our intuition to guide how we actually responded to user feedback; however, it would definitely be more ideal to use actual user feedback rather than having to rely on our intuition for how our testers differed from actual users.
If we were to repeat the process, it would be ideal to do the paper prototype testing on users who were more representative of our actual population. We also should have done more testing where users only played the role of either admin or instructor, rather than having users switch back and forth between different types of users. We discovered in our final testing, that we didn’t have a legend for some of our symbols in the instructor interface but that the symbols were explained in button labels in the admin interface. The only reason that we discovered this was because one of our testers forgot what she had seen in the admin interface when acting as the instructor. Had she not forgotten this, we might have not noticed a very important learnability problem. To prevent this problem in the future, we would be better off having each tester only test as only one type of user so that things that they learn in part of the test would not unduly influence how they act in other parts of the test.
Overall, the iterative design process that we followed was very helpful to designing an interface that our testers loved at the end of the process, while also enabling us to implement the user interface in a timely manner.