Question wording experiments on technology and administrative issues

To get the ball rolling, here are some very preliminary question wording experiments we wrote up at the AEI-Brookings Election Reform Project.

 ~Tim Ryan

Public opinion on election administrative and election technology issues.

 
On non-attitudes and the effects of framing: Converse (1964) finds that citizens are often inconsistent in their views on government policy.  Hochschild (1981), Lane (1962), and Feldman and Zaller (1992), similarly point to ambivalence and the effect that "framing" can have on public opinion. A number of studies (Krosnick and Berent, 1993; among others) use experiments in question wording to design better questions. Others (Kinder and Sanders, 1996) have used experiments in question wording to mimic elite frames political debates.

 
Hypothesis #1: More respondents will express an opinion about election reform issues when questions are framed than when they are not.

 
Hypothesis #2: Responses to policy questions about voting technology are prone to suggestion and will be easily swayed by the provision of a small amount of information to that might justify a given policy.

 

A. DRE vs. Optical Scan

 Newkirk (2008) finds that the public is generally confident in both optical scan and DRE technology. In his study (based on a poll), DREs have a trust score of 51 while optical scan has a score of 48. In contrast, AccuPoll (2004) finds much more trepidation about DREs. Forty-four percent do not have confidence in the reliability and integrity of computerized voting systems; 63% say computerized voting systems are not always reliable because of the risk of programming error or fraudulent hacking by insiders who designed the system. Yet another study[AEI1] by  the Information Technology Association of America (2004) reports that 88 percent of voters using electronic voting machines trust the accuracy of those machines and that seven in ten voters are unconcerned about the security of electronic machines.

 
Pace University (2004) finds the public evenly split. Twenty-six percent trust an electronic ballot more than a paper one; 25% trust a paper ballot more; 43% trust them both equally.

 
There are a number of problems with these polls. The wording of the AccuPoll question about computerized voting systems pushes respondents by pointing to the possibility of "programming error or fraudulent hacking" without noting concomitant advantages of DREs or drawbacks to paper. The ITAA and PaceUniversity polls use the vague terms "electronic ballot" and "computerized voting systems," respectively. Do this refer only to DREs? All voting systems other than lever machines are, in some regard, computerized. I doubt that respondents have a good notion of what systems are being asked about here.

 

 Four possible test conditions:

 Stripped (control) condition

 
As you may know, there is a lot of disagreement about the best way to record votes when voters go to the polls on Election Day.Some people think that voters should make their choices directly on a computer screen and have the computer count the results. Others think that voters should make their choices on paper ballots and then run them through a scanner.[AEI2]  How about you? Do you have an opinion on this issue?[AEI3] 

A)    Yes

B)    No

 

(If Yes) Which do you think should be used[AEI4] , an electronic ballot or a paper-based ballot?

 

A)    I strongly think that electronic ballots should be used instead of paper ballots.

B)    I somewhat think that electronic ballots should be used instead of paper ballots.

C)    I somewhat think that paper ballots should be used instead of electronic ballots.

D)    I strongly think that paper ballots should be used instead of electronic ballots.

 

Framed to support DRE

 

As you may know, there is a lot of disagreement about the best way to record votes when voters go to the polls on Election Day.Some people think that voters should make their choices directly on a computer screen and have the computer count the results because computers can be used by people with disabilities, like the blind. Others think that voters should make their choices on paper ballots and then run them through a scanner.[AEI5]  How about you? Do you have an opinion on this issue?

 

(If Yes) Which do you think should be used, an electronic ballot or a paper-based ballot?

 

 

Framed to support Op-scan

 

As you may know, there is a lot of disagreement about the best way to record votes when voters go to the polls on Election Day. Some people think that voters should make their choices directly on a computer screen and have the computer count the results. Others think that voters should make their choices on paper ballots and then run them through a scanner[AEI6]  because computer systems might be prone to fraud. How about you? Do you have an opinion on this issue?

 

(If Yes) Which do you think should be used, an electronic ballot or a paper-based ballot?

 

 

Doubly framed

 

As you may know, there is a lot of disagreement about the best way to record votes when voters go to the polls on Election Day. Some people think that voters should make their choices directly on a computer screen and have the computer count the results because computers can be used by people with disabilities, like the blind. Others think that voters should make their choices on paper ballots and then run them through a scanner because computer systems might be prone to fraud. How about you? Do you have an opinion on this issue?

 

(If Yes) Which do you think should be used, an electronic ballot or a paper-based ballot?

 

 B. Paper trails

 

UniversityofGeorgia (2006) finds that voters using a paper trail system express high confidence. AccuPoll (2004) finds that 74% prefer a paper trail.

 

The UniversityofGeorgia poll is an exit poll where the respondents just finished using paper-equipped DREs, so I doubt its external validity. The respondents aren't comparing a paper trail system to anything else.

 

Again the AccuPoll question wording is one-sided:

 

"Now tell me which of the following two statements most closely reflects your views. Some elected officials say that computerized voting systems should produce a paper audit record of the ballot that you can see to verify that the computer recorded your vote correctly. Other elected officials say that computerized voting systems do not need to include the extra cost of producing a paper record for you to verify. They say you can trust what you see on the computer screen to verify that your vote has been accurately recorded by the computer."

 

This wording offers the respondent a compelling justification for a paper trail: the ability to verify that the computer recorded the vote correctly. In contrast, the wording of the alternative option makes elected officials sound aloof and cheap if they do not offer paper verification.

 

 

Four possible test conditions:

 

Stripped (control) condition

 

There has been a lot of talk lately about electronic voting machines and whether or not they should produce a paper record that each voter can see. How about you? Do you agree that voting machines should produce a paper record of each vote?

 

 

A)    Strongly agree that voting machines should produce a paper record of each vote

B)    Somewhat agree that voting machines should produce a paper record of each vote

C)    Neither agree nor disagree

D)    Somewhat disagree that voting machines should produce a paper record of each vote.

E)    Strongly disagree that voting machines should produce a paper record of each vote

 

 [AEI7] 

 

 

Framed to justify paper trails

 

There has been a lot of talk lately about electronic voting machines and whether or not they should produce a paper record that each voter can see. Some feel that a paper record of each vote would prevent fraud and hacking, while others don't think it is necessary. How about you? Do you agree that voting machines should produce a paper record of each vote?

 

 

Framed to justify paperless voting

 

There has been a lot of talk lately about electronic voting machines and whether or not they should produce a paper record that each voter can see. Some fear that an election could be robbed if these records were stolen or copied, while others feel that they are necessary. How about you? Do you agree that voting machines should produce a paper record of each vote?

 

 

Doubly framed

 

There has been a lot of talk lately about electronic voting machines and whether or not they should produce a paper record that each voter can see. Some feel that a paper record of each vote would prevent fraud and hacking. Others fear that an election could be robbed if these records were stolen or copied. How about you? Do you agree that voting machines should produce a paper record of each vote?

 
C. Voter Identification

 

Background: Ansolabehere (2007)[AEI8]  finds a large majority (75%) supports an ID requirement. There are interesting demographic splits (southerners, Republicans support ID requirements; so do Blacks, although to a substantially smaller degree.)

 

 

Pew: On Election Day, should voters be required to show an official photo identification, such as a Drivers' License, or shouldn't they have to do this?

 

 

Four possible test conditions:

 

Stripped (control) condition

 

Some people think that voters should have to show an official photo identification, such as a drivers' license, when they go to vote on Election Day. Others don't think that this requirement is necessary. How about you? Do you agree that voters should have to show a photo identification in order to vote?

 

A) Strongly agree that voters should have to show a photo identification in order to vote.

B) Somewhat that voters should have to show a photo identification in order to vote.

C) Neither agree nor disagree

D) Somewhat disagree that voters should have to show a photo identification in order to vote.

E) Strong disagree that that voters should have to show a photo identification in order to vote.

 

 [AEI9] 

 

Framed to justify ID requirement

 

Some people think that voters should have to show an official photo identification, such as a drivers' license, when they go to vote on Election Day in order to prevent voter fraud. Others don't think that this requirement is necessary. How about you? Do you agree that voters should have to show a photo identification in order to vote?

 

 

Framed to oppose ID requirement

 

Some people think that voters should have to show an official photo identification, such as a drivers' license, when they go to vote on Election Day. Others think that a photo ID requirement would be unfair to a lot of people who don't happen to own an ID. How about you? Do you agree that voters should have to show a photo identification in order to vote?

 

 

Doubly framed

 

Some people think that voters should have to show an official photo identification, such as a drivers' license, when they go to vote on Election Day in order to prevent voter fraud. Others think that a photo ID requirement would be unfair to a lot of people who don't happen to own an ID. How about you? Do you agree that voters should have to show a photo identification in order to vote?

 
D. Absentee voting

 

Pew question: Some states are using an election system in which residents are allowed to cast their vote through mail in the weeks leading up to Election Day if they want to.  Do you strongly favor, favor, oppose, or strongly oppose allowing people to vote by mail before Election Day instead of in a voting booth on Election Day?

 

Stripped (control) condition:

 

Some states are using an election system in which residents are allowed to cast their vote through mail in the weeks leading up to Election Day if they want to.  How do you feel about allowing people to vote by mail before Election Day instead of in a voting booth on Election Day?

 

A)    I strongly favor allowing people to vote by mail before Election Day.

B)    I somewhat favor allowing people to vote by mail before Election Day.

C)    I don't have an opinion on this issue

D)    I somewhat oppose allowing people to vote by mail before Election Day.

E)    I strongly oppose allowing people to vote by mail before Election Day.

 

 

Framed to support VBM

 

Some states are using an election system in which residents are allowed to cast their vote through mail in the weeks leading up to Election Day because it is more convenient to voters.  How do you feel about allowing people to vote by mail before Election Day instead of in a voting booth on Election Day?[AEI10] 

 

 

Framed to oppose VBM

 

Some states are using an election system in which residents are allowed to cast their vote through mail in the weeks leading up to Election Day. But some people are concerned that this could lead to problems, because there is no way to tell who filled out the ballot. How do you feel about allowing people to vote by mail before Election Day instead of in a voting booth on Election Day?

 

 

Doubly framed

 

Some states are using an election system in which residents are allowed to cast their vote through mail in the weeks leading up to Election Day so that don't have to go through the hassle of driving to the polls. But some people are concerned that this could lead to problems, like a husband pressuring his wife to vote a certain way. How do you feel about allowing people to vote by mail before Election Day instead of in a voting booth on Election Day?
Works Cited[AEI11] 

 

AccuPoll. 2004. National Survey of 780 likely voters, conducted by Public Opinion Strategies, August 3-5.

 

Converse, Philip E. 1964. "The Nature of Belief Systems in Mass Publics." In Ideology and Discontent, ed. David E. Apter. New York: Free Press.

 

Feldman, Stanley, and John Zaller. 1992. "The Political Culture of Ambivalence: Ideological Responses to the Welfare State. American Journal of Political Science36: 268-307.

 

Hochschild, J.L. 1981. What's Fair? American Beliefs about Distributive Justice. Cambridge, MA: HarvardUniversity Press.

 

Information Technology Association of America. 2004.

 

Kinder, Donald R., and Lynn M. Sanders. 1996. Divided by Color: Racial Politics and Democratic Ideals. Chicago, IL: UniversityofChicago Press.

 

Krosnick, Jon A., and Matthew K. Berent. 1993. "Comparisons of Party Identification and Policy Preferences: The Impact of Survey Question Format." American Journal of Political Science 37: 941-964.

 

Lane, Robert E. 1962. Political Ideology. New York: Free Press.

 

Newkirk, M. Glenn. 2008. Trends in American Trust in Voting Technology. Retrieved May 30, 2008 from InfoSENTRY Services, Inc.: http://www.infosentry.com

 

Pace University. 2004. Survey of 662 first-time voters.

 

UniversityofGeorgia. 2006. November 2006 General Election Paper Voting Trail Exit Poll Study. Survey of 459 voters. From http://electionupdates.caltech.edu/UGA_Study_VVPAT_Nov_2006.pdf

---

 AEI1This info is only available from a press release.
 AEI2Some of this wording is borrowed from Glenn Newkirk's InfoSentry survey. Do we need to ask his permission?
 AEI3Asking "Do you have an opinion?" as its own question is a strong filter for non-opinions.
 AEI4Would there be any advantage to saying something more like "Which do you trust more?"
 AEI5In all these questions, perhaps the "some/other" statements should be randomly rotated. I don't think this is necessary, but it is a thought.
 AEI6Some of this wording is borrowed from Glenn Newkirk's InfoSentry survey. Do we need to ask his permission?
 AEI7We can also add an "don't know" option, but I'm not sure we should. I think it would basically mean the same thing as choice C.
 AEI8No info on this question wording yet.
 AEI9Perhaps a "don't know" option.
 AEI10Rather than "so they don't have to go through the hassle," we could say something like "in case they are sick or traveling." That would be a stronger rationale. But I don't think it's the right one because nobody really wants to end that type of absentee voting. This question is more about absentee voting just for the sake of convenience.
 AEI11This was done casually and should be revisited before any sort of publication.