...
During implementation, we tested and retested our site, checking for usability problems and bugs, following the iterative pattern of design. Overall, we felt that using Rails helped us abstract our design into separate parts, helping us focus on different individual parts with ease.
Evaluation
The user tests were conducted by giving the uesrs a brief overview of the system by explaining to them its intended goals. We then asked the users to navigate the interface in order to familiarize themselves with it before we assigned them specific tasks. The tasks we assigned were very simlilar to the ones we used in our paper prototyping. Since our interface has a limited number of use cases, we wanted to optimize the design for these common scenarios instead of having an overly-complicated UI.
We selected three different users from our fraternity to test the system. Since our system is designed specifically for the brothers of PKS, it didn't make sense to test it on anyone else. We did, however, decided to pick users carefully based on their computer expertise and experience with managing house jobs using other methods. We chose two users that had never managed house jobs before and one who had previously done all management by hand. One of the users who had never managed house jobs before was a programmer while the other didn't use his computer for much other than email. The user who previously managed house jobs was a power users, but not a programmer. We thought picking users of varying skill levels and domain experience would help us identify different types of problems.
We briefed each user by describing the basic fucntions of the interface. Since the users already know what house jobs are and how room reservations work, we didn't need to do any type of demonstration. Instead, we hoped that the interface would be intuitive enough for them to figure out on their own. In fact, this is a necessary requirement for our UI, since it would be infeasible to hold a training session or demonstration each year as new brothers need to use the system.
We first had them login as regular users with predefined usernames and passwords. We asked them to find their house jobs, complete them, and tells us what their past house jobs were. From there we asked them to identify other users jobs and had them switch to the room reservation tab. In this tab, we gave them a predefined event time to schedule for, had them manage an event time conflict, and then schedule an event of their own. Once completed, we asked the users to manage their account to change their password and logout. We had the user take a 30-second break while we changed their admin status in the database. Upon completion, we had them log back in with their new password. We told them that they were now administrators and asked them to manage several settings in the admin panel. Since we hoped that most things would be intuitive, we didn't give them any specific guidance; instead, we observed what they tried to do. After letting them explore the panel on their own, we asked them to use the filters and search (if they hadn't done so already). Once we were satisfied that they had explored all the interfaces features, we had them log out and give us their overall impression.
Usability Issues
-
Reflection
Over the course of the iterative design process, we were able to focus on improving the overall design of the site, by focusing on the individual components. At the start of the project, we had a much larger project in mind, but we decided to tone it down to be able to work on making the most integral components of the site as good as possible. Prototyping our design in several different ways helped us to form a better picture of the site in our heads before we even sat down to code, which significantly cut down on the actual coding time. The computer prototype in particular allowed us to be able to focus a lot of our attention on the back-end of the site, and to make several small iterative changes to the prototype to improve the final product. User testing and heuristic evaluation really helped us focus on improving particular parts of the user interface, also cutting down time working on the front-end. Furthermore, because of the time saved, we were able to mock up, design, and completely implement a brand new feature to the site (the Admin Panel).
...