...
Evaluation of the Issues discovered during the User Tests:
...
Issue |
...
Severity |
...
Possible Solutions |
...
Didn’t notice that the Fix Mistake Menu had a tutorial |
...
Minor |
...
Despite not using the tutorial, she was still able to navigate the menu and complete her tasks. This is just a matter of changing the wording at the end of our first tutorial to make things clearer. |
...
User wanted to try the tutorial but accidentally clicked “Skip” instead of “Continue” |
...
Cosmetic |
...
We think this is because the “Skip” button has a brighter color (red). This should be an issue of just changing the button colors to be more intuitive. |
...
Expected to see a ballot-like UI, which listed candidates in order. Commented that this felt slower. (Current approach is based on a pie-menu) |
...
Minor |
...
This is a new UI, but we put a lot of thought into how to make this most efficient. After a bit of practice, we think this would be faster. This implies our learnability could be higher but the tradeoff is for long-term efficiency. |
...
The order of candidates on the page seemed random |
...
Minor |
...
Again, we considered efficiency and placed the candidates based on party and frequency with which that party is selected. Once again this is probably a trade-off we consciously make between learnability and efficiency. |
...
No place to write in the name of a write-in candidate |
...
Minor |
...
We were told in our first interview that usually the write-in ballots are put to the side and not recorded, so we are handling the needs of many of our users. |
...
User didn’t see the “cancel” button to return from the fix mistake menu |
...
Minor |
...
She just went back to the previous element so it was not too inefficient. |
...
“This would be so much better than the hand counts” |
...
Good |
...
She thought this system as a whole was a success and an improvement on the existing methods. |
...
Restarting from a mistake somewhere on this ballot or the previous ballot rather than allowing a fix of just one mistake was a good decision for elderly users |
...
Good |
...
She thought it was very helpful that the elderly users won’t need to jump around pages but can just back track a little bit to be sure of accuracy. |
...
Wanted a “fix one mistake” option for ballots more than 2 ago. Currently we assume that means the audit is off and requires a reset. |
...
Minor |
...
Mistakes that long ago were not a major concern of hers, and she said this makes sense too. |
...
Reflection
We originally had considered two user classes: the auditor and the warden. In our subsequent GRs, we focused on the auditor's needs, but we haven't addressed the warden. The warden acts as an administrator and would have the ability to create/assign/view results. Without the warden account, our website just focuses on the input side of the election, not really the output, and isn't really meaningful in the context of an election. We've discussed the possibility of maybe just showing the results after the audit, but then our website is only relevant for the 2012 presidential election (the warden would have the ability to create new elections). In our work before GR4, we created a nearly complete computer prototype of the warden and then decided to focus on the auditor and throw that out. In re-doing this process, we should have identified the need to narrow our project down to just the auditor earlier and stuck to that.
...