Design
Our final design for our interface had several key changes (the first column are screenshots from GR4, the second column are screenshots from GR5):
|
|
|
|
|
|
|
|
On the welcome page, we removed the busy background and restyled the page to make the information more salient and readable for our elderly user population. We modified the login page similarly (not shown).
On the main audit page, we kept the right side of the interface since that seemed to work well and changed the left sidebar. We made only the previous 3 selections visible, the user can scroll up to see the other selections for this ballot. Having only three selections allowed us to increase the size of the text, which should be easier for our user population to view.
On the fix mistake menu, we threw out nearly all parts of our GR4 design. The right panel was being used inefficiently in a distracting help menu and the button selections were too small on the left sidebar. Instead, we moved the ballot selections to the right, making them tabs, so the entire ballot can be viewed at once. The left panel only has a small amount of help text and a few essential buttons.
On the results menu, we added the overall results for the election and restyled the page to make it more clear.
In addition, we added a tutorial for the user when they enter the website. There is also a help link on all the pages, so the user can go back and view the tutorial.
Implementation
The Truevote backend was implemented using Django. Because of the sensitive nature of vote count information, we chose to have all our model logic and persistence on the server. Whenever a user makes a selection, a request saves the selection to the PostgreSQL database.
Pages on the front end always display information loaded from the server, not stored in any local Javascript data structure. This way, we ensure that any user navigation (e.g. the browser’s back button, accidental page refreshing) does not compromise the precision of our vote count report. As the information is centralized on the server, users get to see aggregate totals and predicted winners on the results page.
The downside of server-side logic is that frequent server requests cause some delay. This was occasionally evident in our GR5 staging, as we deployed our project on a free -and inherently slow- Heroku account. However, because the reading and switching of paper ballots restricts the speed of the election audit task, a blazingly fast system is not immediately necessary.
We used bootstro to implement the interface tutorials, a Javascript library for easy interface tours.
Evaluation
For our user tests, we tested 3 employees of the Town of Concord. One was the town clerk, and the other two worked in the office. All of them have administered elections and counted ballots. We did not conduct a demo, because our interface has a tutorial built in and would not have a demo in normal circumstances.
Briefing
You will be entering ballots today as part of an election audit. You have been assigned 5 ballots and will complete the audit. This is the auditor side interface, so assume a town clerk or warden had already set up the election options and given you a username.
Tasks
- Enter the site and make your way to the main audit page. (We let them choose whether or not to take the tutorial in order to get a feel of whether people would take it).
- Enter 3 ballots with given values.
- Fix a mistake on the current ballot.
- Fix a mistake on the previous ballot.
- Complete 2 more ballots.
- Fix a mistake on ballot 1 (i.e. restart the election).
- Finish entering ballots and find the number of votes that Elizabeth Warren received.
Evaluation of the Issues discovered during the User Tests:
Issue |
Severity |
Possible Solutions |
Didn’t notice that the Fix Mistake Menu had a tutorial |
Minor |
Despite not using the tutorial, she was still able to navigate the menu and complete her tasks. This is just a matter of changing the wording at the end of our first tutorial to make things clearer. |
User wanted to try the tutorial but accidentally clicked “Skip” instead of “Continue” |
Cosmetic |
We think this is because the “Skip” button has a brighter color (red). This should be an issue of just changing the button colors to be more intuitive. |
Expected to see a ballot-like UI, which listed candidates in order. Commented that this felt slower. (Current approach is based on a pie-menu) |
Minor |
This is a new UI, but we put a lot of thought into how to make this most efficient. After a bit of practice, we think this would be faster. This implies our learnability could be higher but the tradeoff is for long-term efficiency. |
The order of candidates on the page seemed random |
Minor |
Again, we considered efficiency and placed the candidates based on party and frequency with which that party is selected. Once again this is probably a trade-off we consciously make between learnability and efficiency. |
No place to write in the name of a write-in candidate |
Minor |
We were told in our first interview that usually the write-in ballots are put to the side and not recorded, so we are handling the needs of many of our users. |
User didn’t see the “cancel” button to return from the fix mistake menu |
Minor |
She just went back to the previous element so it was not too inefficient. |
“This would be so much better than the hand counts” |
Good |
She thought this system as a whole was a success and an improvement on the existing methods. |
Restarting from a mistake somewhere on this ballot or the previous ballot rather than allowing a fix of just one mistake was a good decision for elderly users |
Good |
She thought it was very helpful that the elderly users won’t need to jump around pages but can just back track a little bit to be sure of accuracy. |
Wanted a “fix one mistake” option for ballots more than 2 ago. Currently we assume that means the audit is off and requires a reset. |
Minor |
Mistakes that long ago were not a major concern of hers, and she said this makes sense too. |
Reflection
We originally had considered two user classes: the auditor and the warden. In our subsequent GRs, we focused on the auditor's needs, but we haven't addressed the warden. The warden acts as an administrator and would have the ability to create/assign/view results. Without the warden account, our website just focuses on the input side of the election, not really the output, and isn't really meaningful in the context of an election. We've discussed the possibility of maybe just showing the results after the audit, but then our website is only relevant for the 2012 presidential election (the warden would have the ability to create new elections). In our work before GR4, we created a nearly complete computer prototype of the warden and then decided to focus on the auditor and throw that out. In re-doing this process, we should have identified the need to narrow our project down to just the auditor earlier and stuck to that.
In addition, in our first paper prototype, we focused a lot on the main audit page and added the fix mistake page as an afterthought. We should have spent more time refining that design before, so we could have arrived at our current GR5 implementation earlier.
In evaluating our results, it was difficult to anticipate the feedback that we got. Each auditor has their own personal process and it seemed like they were reluctant to switch from hand counting. This might be an aspect of our elderly user population, but if they were given time to learn the system, it seems much more efficient to use a website over tally sheets.