...
- The front end of our website features a simple and usable design that allows users to enter information pertaining to a particular event. Our website creates a QR code that gives potential attendees a simple way to RSVP to these events by scanning the QR code (hence the name RScanVP). The user is presented with another page of development that allows them to recolor the QR code and also re-size it within visible constraints.
- QR codes are regenerated each time a new size or color is set. This was simpler than recoloring the image in JavaScript and thus we chose this way of doing it. However, based on user testing, this was a bad idea because users get confused when the QR code keeps changing.
Implementation Fixes
Based on feedback from testing we fixed several issues with RScanVP
- We redesigned the homepage to better fit on narrower devices.
- We added a drop down for times to the event details page.
- We added a poster to the event details page to communicate the poster metaphor.
- We decided to take away the "Download" button altogether. This makes it a little harder for users to acquire the QR code that has been generated. We made this decision so that the user was not redirected away from the site at any time because we could not figure out any other way to do it.
- We decided to add red text indicating an error in text fields so that users would not know why their submissions were being rejected. Our user testing indicated that this was a good idea but would be more helpful if it were more visible and possibly located next to the field that had the error.
- QR codes are regenerated each time a new size or color is set. This was simpler than recoloring the image in JavaScript and thus we chose this way of doing it. However, based on user testing, this was a bad idea because users get confused when the QR code keeps changing.
Evaluation
Getting the users
...
- After getting the users who would participate in the test, we told them that we created a website to help with the tracking of results for advertised events.
- Poster Maker - You are in charge of publicizing for an event. Your first task is to use our site to create a QR code that prospective attendees can scan to send an RSVP for the event. Later on, you will be tasked with viewing the statistics for your event.
- Prospective Attendee - Your task is simply to scan a QR code, RSVP 'yes', and choose to get an email reminder/add it to your calender.
Usability Problems and Non-Implemented Solutions
- User did not know what a QR code was - Major - We could add more information about what a QR code is to the homepage.
- Download Obscurity - Catastrophic - Users were unsure of how to actually download the QR code. There was a note saying to right-click and download the image, but users got stuck because the term "download" was not consistent with the supplied option of "Save Image As...". In order to fix this, we could simply change the line of text telling the user how to download and make the syntax consistent. Further, we could simply email the image to the user when we email them the Admin link so that they do not have to worry about downloading it. However, this means that supplying an email would not longer be optional.
...
- Recolor Confusion - Major - Users were a little confused that every time they clicked "Set Color" a new QR code appeared. As mentioned in the Implementation Section, we did this on purpose. To avoid this, we could made re-size and re-color happen on the QR images themselves rather than trying to create a new QR image each time the button was clicked. As a simpler, but not desired, solution we could have written a note to the user explaining that the codes would continually change but still work, so they should not worry that the images keep changing.
- Not Download Error - Major - Users felt that downloading the QR image was important enough that there should be a safety precaution in place to check if the user actually downloaded the image. When a user clicks done on the Design page, they should be prompted with a pop-up box asking if they downloaded the image. This solves itself and would help users not to miss the line of text telling them to grab the image.
- Redundant Color - Cosmetic - One user thought that after choosing a color, he needed to also click the "Set Color" button to confirm the color of the QR code. He found this slightly unnecessary and thus redundant. Of course, this was not actually necessary, but he did not know that. We could solve this by letting the user know that the image displayed on the page is actually the QR image that they will download. This will help clarify that changing the color is a permanent action and no further actions are required to set the color.
- AM/PM Confusion - Major - The radio button on the phone is confusing because two of the users (possible attendees) were unsure of whether or not they were going to the event. They had trouble deciphering whether or not "Yes" was the currently selected option. We could fix this by simply providing a better radio button that makes it much more clear which option is being chosen.
- Error Text - Minor - As noted by the TA, error text was far from the fields that had the error. This was not mentioned in the user testing because the screen that we used for user testing happened to be larger than the one used for GR5 demoing. Thus, it was clear to users when the text appeared that it had appeared.
- User did not know if they had a QR code scanner - Catastrophic - We could ask poster makers to insert instructions on how to download a QR scanner. We could ask poster makers to also provide a text URL for them to enter.
- Color took a while to change - Minor - We could indicate that the page was processing while the new QR code was loading.
- Didn't hit set size - Minor - The user typed in a size, but did not hit "Set size". We could automatically set the size when the box loses focus.
- The user did not want to give their email - Minor - We could add a message saying their email will only be used once.
- Network slow - Out of Scope - The user was confused when the internet network was slow; little you can do about this.
- User surprised could see other RSVPs - Out of Scope - Security was not a requirement for this project.
Reflection
- The main thing that we learned over the course of the semester is that we are not our user. That is, even if we think something is simple, obvious, or clear, this does not mean that the user will feel the same way. We have a lot of preconceived notions about what things should do because we actually created them, the users of our system will not be in that position and are thus much more likely to be confused.
- There is no one right method of testing. We caught different things that needed to fixed when doing heuristic evaluation vs paper prototyping for example. Heuristic evaluation was much better for finding things that needed to be visually fixed. It was very helpful to get someone else's opinions on what the website was missing in terms of display and aesthetics. Where as, paper-prototyping was better at providing us with functionality problems. User testing was very helpful in drawing a line between what we thought vs. what the user thought and how we perceived the site as a whole. For us, the most useful tests were those where users tried to play with the system and commented on wherever they got stuck or confused (i.e. paper-prototyping and user testing).
- If we were doing this project over again, we would focus on providing more options for how things are done in paper-prototyping. For instance, we would present three options for QR image re-sizing. One a text box, one a drop down box, and one with a slide-rule. Basically, we would have preferred to add a step into the iterative design process where we did something like AB Testing to determine which designs were optimal for the most users.
- Doing this project again, we would focus a lot more on testing with a lot of users (as opposed to less than 5). With this many users it was hard to get a general feel for how they all felt together, but rather their individual concerns, which may or may not have been necessary or realistic.
- Finally we would have liked to do some smaller rounds of user testing, not just on the final implementation, but on some of the modular sections as they were completed. While the overall experience for the user cannot be tested without an almost fully complete implementations, several individual sections of the application are significant enough to stand on their own and warrant user testing. In this way, we could catch usability problems earlier on in each of the parts and iterate on them, before pulling all of the tasks together into a test of the overall final design and experience.