MITJobs - GR6 - User Testing

Design

When a user first comes to the website, he is showed the log in page. The user can log in either with MIT certificates, or with a previously registered username and password or by signing up for a new account. For efficiency reasons, we decided to have the sign up form on the first page itself, rather than having a “Sign up” button/link leading to a different page.

Figure 1: Log in page

After signing in or signing up, the user enters the website (Figure 2). At the top of the webpage we added the message “Welcome” followed by the name of the user. This message is centered on the page for clear visibility, and to give the user a hint that he/she has started a session. For MIT students the default tab opened is the Browse tab, as browsing is one of the most critical tasks of the system. For non-MIT users (recruiters), the browse tab is hidden so we the default tab opened is "My Posts", as for these users, viewing and editing their own posts is key. We decided to use tabs as we had only 3 main views and tabs were suited for navigational visibility; we made this decision while paper prototyping.

Figure 2: Browse tab.

Users can filter posts on the same page. As soon as a user changes the value of a filter (e.g. course), posts are updated for instant feedback. In our initial design scenarious (before paper prototype), we had a separate page to enter filters before displaying posts. We decided to not to do this as it reduces efficiency of the interface.

Each course has its own job groups. For example, under course 6, we have Hardware, Software, Electrical Engineering, Consulting and Investment Banking. There are some job groups, namely Consulting and Investment Banking, are common through all majors. On selecting a job group, it is displayed with an “X”, in case the user wishes to to delete the filter. An example of filtering is showed in Figure 3 with the filters on the left of the page, along with the resultant job posts.

Figure 3: Filtering job posts

Users can follow a post in the follow tab by clicking the "Follow" button next to the post. Initially, when a user clicked on "Follow", our interface initially only had the text on the follow button changing to “Unfollow”. Our TA, Vijay, in his evaluation of our computer prototype brought to our attention that this feedback does not pass the squint test. As a result, we decided to also change the color of the button to red to ensure the feedback is clearly visible. Figure 4 illustrates this feedback

Figure 4: Following job posts

To create a post, the user can click on the "Create Post" button. We decided to put a button to “Create Post”, which is visible throughout every tab, as opposed to being able to create posts only through the “My Posts” tab. We did this to increase the learnability, efficiency and visibility of a critical task of the system: users being able to create posts. On clicking this button, an overlay form is displayed for users to create posts (Figure 5). Another issue that our TA brought up in his evaluation of the computer prototype was that in the "create post" view, the "Create" button is blocked from view (you need to scroll to see it). We addressed this issue, and ensured that this button was visible when the pop-up form was displayed.

When creating a post the user can change the Title, the Job Type, Course(s), the Job Group, Skills, Location, Date and Application Instructions. If the user decides, that he/she doesn’t want to create a post, he/she can simply close the overlaid box. If the user presses the "Create" button, the post will be saved, and the form closed. The user can see the updated changes in the "My Posts" tab and in the "Browse" tab.

The location form has an auto-completion property, which prevents the user from committing mistakes when filling this form. The date form when clicked shows a box with a calendar where the user can choose the date easily. These two features were added for error handling and for efficiency.

Figure 5: Creating a post

In the "My Posts" tab, the user can see the posts he/she follows, and also the posts he/she has created. He can unfollow posts, and also message creators of the posts he is following.  For recruiters (non-MIT affiliates), this tab only shows the upper section titled "My Posts" and not "Posts I'm Following". This tab is illustrated in Figure 6.

Figure 6: My Posts tab

Clicking on the job post in this tab or in the browse tab opens the description of the post as an overlaid form (Figure 7).

Figure 7: Post description overlaid.

Users can delete the posts that they have created by clicking the "X" button next to the post. If the user deletes a post, a message appears at the top stating that a post was deleted and the user can undo the deleted post (Figure 8). This function was introduced after our paper prototype to facilitate error handling.

Figure 8: Deleting post

The user can see all the messages in the "Messages" tab (Figure 9), which is divided between three different parts: Inbox, Sent, Updates. "Inbox" includes messages received, "Sent" includes messages sent, and "Updates" includes updates about posts received (e.g. change in deadline). The total number of messages/updates that are unread is shown in red on the tab next to "Messages", and in Inbox and Updates, the number of unread messages/updates is also shown. These numbers were added during paper prototyping for state visibility.

At the left of each message there is a dot showing if the message is read or unread. If the message is unread the dot is red and is filled, and if the message is read, then the dot is blue and not filled. This feature was added for visibility. These features were added after paper prototype evaluation to increase visibility.

The currently open view (Inbox/Sent/Updates) is highlighted in light blue for navigational visibility.

Figure 9: Messages tab

The user can choose to compose a message either by clicking the "Compose" button the messages tab or by clicking the "Message" link on the "My Posts" tab. In the compose view (Figure 10), three fields appear at the right of the webpage: To, Subject and the content of the message. If the user gets to the page by clicking one of the "Message" links in the "My Posts" tab, the "To" field is already prepopulated with the appropriate recipient. If the user instead has got here using the "Compose" button, the "To" has an autocomplete drop-down with names of posts a user has created (if user wants to message followers of posts), and with names of posts he/she is following (if user wants to message the creators of a post he/she is following).

The user, then, fills all the information necessary to send a message and he/she can send the message. Also, a user can discard a message, if he/she decides not to write a message. We enabled this after the paper prototype to increase user control and freedom. If the user discards a message he/she is redirected to the “Inbox” folder in order to be consistent with applications such as Gmail.

Figure 11: Composing Messages

Implementation

In our final implementation, we decided to focus on developing the front-end using HTML, CSS and JavaScript, as opposed to implementing a persistent backend using PHP and SQL. Using JavaScript, we implemented a single session for a single client. The only place where we used PHP was in logging in through certificates, as we felt it was important to implement this for user testing.

Our implementation was mostly based on our domain analysis. The main entities in our system include a user session, user, post and messages. The session maintains the current user of the session, the list of messages, posts, users, posts the user has created and posts the user is following.

Each time a user logs in, a session is created for him using some preloaded posts, and messages. The model is consistent between tabs; for example, if a user follows a post through the browse tab, it is reflected in the “Posts I’m Following” on the “My Posts” tab. Similarly, if he/she creates a post, it is reflected both on the browse page and on the “My Posts” page.

There were several implementation constraints which could possibly affect the usability of the interface.

1. Our implementation placed the posts that a user decides to follow on a different tab from the browsing tab. As such, when a user decided to follow a post they could not tell that we had a list of “followed posts” in a separate tab that was dynamically changing based on which posts they decide to follow or unfollow. We tried to alleviate this lack of information by offering some affordance by changing the color of the button from blue to red, and the text inside the button from “follow” to “unfollow”.
2. In our implementation, some users can only post jobs but not browse or follow posts (company recruiters). Other users can both browse and post jobs/starups/UROP etc (MIT affiliates). Separation of these two categories could be easy if all MIT affiliates could use certificates to sign in because we could associate a certificate sign-in with some sort of master login that allows both posting and browsing, while non certificate login can be associated with only browsing. This would just be a hack because we still should ideally allow an MIT affiliate to sign in with a username-password combination if they feel reluctant to use certificates.

When we were testing our system, we still had some bugs we were trying to fix, and the test facilitator was prepared to explain these if a user encountered them. However, these issues are minor and do not hinder users from accomplishing the critical task we were trying to test, these did not come up in the test. For example in the “My Posts” tab, the total number of posts that were created by the user is shown, along with a "Next" link to possibly show different posts. This text should not be displayed when all posts already fit on the first page (e.g. the text says "Previous 1 of 3 of 3 Next". The previous and next link should not be displayed at all in this case. This same feature worsk fine in the "Browse" tab

Evaluation

For our user tests, we had 4 representative users to test the system: 2 MIT student browsing opportunities, a course 6 TA looking for UROP’s and a company recruiter for a company looking for course 6 students. We found these users mainly through our own contacts; these users are representative of each user groups of our target population (2 students browsing opportunities and 2 people advertising opportunities). We decided not to do a demo for the users as we felt if we did, we would not able to judge the learnability of the system properly.

Briefing

MIT Jobs is a website that will replace Anne Hunter's mailing list. A recruiter will be able to create an account an directly advertise internship or full-time positions on the site. An MIT student with a start-up idea can find students to work with by advertising his/her idea on the site. A professor or grad student seeking a UROP or RA can advertise their available position on the site. On the other hand, an MIT affiliate can browse all posts on the site by different categories if interested in finding something to do that meets some criterion. With this in mind, we will give you some tasks, try to accomplish them, giving us as much feedback as possible in the process.

Tasks

User Group 1: MIT Students

1. Log into the website with your certificate.
2. You are looking for a summer internship in course 6 and specifically in hardware
3. Follow all posts in that category.
4. Send a message to the recruiter of one of the job posts that you are now following.
5. Check your inbox for new messages and read all new messages.

User Group 2: Recruiters looking advertising opportunities

1. Sign up for an account on the website.
2. Create a new post for a software internship in Course 6, in Cambridge, MA, and deadline May 20, 2011.
3. Edit the post and change the deadline to May 27, 2011.
4. Message all people following that post.
5. Delete the post.

Usability Problems and Possible Solutions

First user:

1. Our first user did not have problems doing most of the tasks, but he gave some tips on how we could improve our website. The first tip he gave was that when he does some task and change from one tab to another tab, or from one view to another, he stated that it would be good if the user could go back to the previous view/tab by only using the backspace button on PCs or the delete button on Macs.
2. Also he noticed that the visibility for seen if a message was read or not read is not that good. There is only one dot, that is red and filled if the message is unread, and that is blue and not filled when the message is read. The user noted that it would be better if the whole color of the button was different if it was in a different state.
3. He also noted that the feature of undoing the deletion of a post should be enabled for messages as it had been enabled for post, or at least there should be a mailbox with the most recent deleted messages, which would be called “Trash”.

Second user:

1. Our second user mentioned that he would like to be able to search for a company based on their name. To implement this, we would display a search bar above the filters on the “Browse” tab and put in some searching functionality on the back-end.
2. Currently, all posts are sorted based on application due date (i.e. deadline). The user mentioned that he would like to be able to sort on other fields, such as post title. To fix this, we would have to implement sorting functions on the back-end. Clicking on a field title would sort based on that field to be consistent with other applications that enable such sorting.

Third user (MIT TA):

1. He said he had a more wholesome experience on MITJobs when he thought of himself as a job seeker than when he thought of himself as somebody who is offering a job post. When he created a job post, the application deadline he entered was earlier than the other posts in the system, and hence his post got appended to the end (as we had sorted posts on Date). To quote him: “For some reason, the browsing scheme seems a lot smoother than the posting scheme. When I post something, it does not show up at the front page, it goes to the end of the list of posted things. Despite the fact that the ‘Create Post’ button stands conspicuously at the top of the page, the user is equally served if he can see his post immediately after creating it -- he shouldn’t have to go to the last post to see it. On the other hand, the filtering scheme for browsing is the big shot feature over here, I believe. It’s what makes browsing smooth. Maybe increase the filtering categories to allow filtering by company too.”  To address his main concern, we would add some feedback for post creation such as a message at the top of the screen "Post has been created".
2. He proposed displaying the “My Posts” tab only for the recruiters, and moving the “Posts I am Following” from the “My Posts” tab to the “Browse” tab. As for recruiters (non-MIT affiliates, this tab would only have "My Posts" and not "Posts I am Following", we had not envisioned that it would be a problem for a user who could potentially be a browser and a recruiter. The "My Posts" tab thus merges the 2 views together for such a user, and these parts should be separated. He said the “My Posts” tab was just confusing when he thought of himself as a job seeker, both in its name and its content. Both suggestions were good.

This user raised some issues that we found to expose some aspects of our implementation that in essence limit the usability of the interface.

Fourth user:

Our fourth user really liked the website overall, and said it would be a much better interface than CareerBridge if we could actually implement it. He had one suggestion:

1. He mentioned he would like to be able to check the users who were following his job post to see his/her profile. This issue was also brought up by our TA, Vijay, in the project demo. As this issue had not come up during the user/task analysis or in any of the prototype evaluations, we did not implement it. However, we realized it is easy to fix as in the JavaScript back-end, we already keep track of the followers for each job post, and we realized we could just have a separate button next to a post the user has created in the “My Posts” tab to view all followers of a post.

Reflection

In the course of the iterative process of designing and implementing our interface, we principally learned the skill of self-critique. At all stages, of the design process, we had to analyze our own designs and find room for improvement.

The very first task of coming up with several potential designs for our implementation proved to be the most difficult but turned out to be the task that influenced our design the most. Having to come up with 4 different designs made us consider many different approaches by which we could lay out the MITJobs interface. We initially made the mistake of making 2 out of 3 designs be conceptually similar, and our TA, Vijay, pointed this mistake. He encouraged us to develop a design with an approach that was conceptually very different; we brainstormed a design that was focused on locations of jobs and internships and identified ways of displaying information centered around location (using maps).

When we started paper prototyping, we realized that coming up with designs beforehand was particularly instrumental because when we selected one of the designs to work with, we realized that most of the improvements we thought of could be attained by incorporating various parts of the alternate designs that we had come up with.

Through the 3 phases of testing (paper prototype testing, heuristic evaluation and final user testing), we learned that actual user reviews can expose completely unforeseen potential problems in mostly the layout of a user interface. We, as designers, at every stage would like to believe we have designed an interface, which is learnable, efficient, visible, error-preventive and yet simple, and that we have considered all alternative choices. However, the reviews we received from our evaluators helped us realize ways in which we could improve our interface. These reviews were broad but starkly incisive; ranging from layout issues such as buttons being too big and inter-tab consistency mishaps, to some suggestions of tasks they would like the interface to have. Refining our interface at every step thus helped us learn the user centered design process, and also develop a usable interface.

One thing we had not expected was that we got much higher-level feedback (e.g. separating "Posts I'm Following" and "My Posts" from the "My Posts" tab as opposed to layouts and buttons) from user testing at the end than from testing our 2 prototypes (paper and computer). We believe this may be because the final users are testing the final product and can put themselves in context better.

The process of building the feedback and the user interactions in different parts of the website was the most difficult, and decisions about the interface during the paper prototyping phase influenced and changed a lot how the different parts of the website interacted with each other. Hence it is clear for us that the first stages of development of a product play a big role on what a product may become and, therefore, should be considered as a very important part of software development. We focused on consistency between tabs a great deal to ensure a smooth experience for users. We also tried to maintain the balance of visibility and simplicity as well as of error-prevention and user control.

Since both user reviews and paper prototyping are really important for software development, we also learned that the development of a software is not a single stage phase that has a determined time to start and to end. Software development is complex and requires to be constantly updated so it can satisfy the needs of the users. Had we followed the waterfall model in implementing this interface, it surely would not have turned out as well as it did with the user-centered design process.