The Susan and Paul M. Sniderman Best Experimental Paper in Europe Award

por Alejandra Suarez | Dic 14, 2017 | General, Research

The award is given annually in recognition of a published article in a peer-reviewed journal, which has made a significant contribution to knowledge or made important discoveries using experiments in the field of political science broadly conceived.
Eligible papers must have been published on any date before the nomination deadline. At least one of the main authors must have their main affiliation at an institution in Europe as noted on the publication.
A Scientific Committee appointed by the coordinators of the Barcelona-Gothenburg-Bergen Workshop in Experimental Political Science will select the best paper. The winner will be invited to present the paper and receive the award at the annual workshop meeting held in May.
The purpose of the award is to recognize and further the innovative use of experiments in European political science academia. Through a series of collaborations with scholars in Europe from the 1990s and onwards, Paul M. Sniderman, with the unstinting support of his wife Susan Sniderman, used the experimental method to make discoveries and advance knowledge in political science and, by his example, encouraged scholars across Europe to do the same.
Nomination Instructions:
Please send an electronic copy of the published article to polexp@ub.edu (by e-mail) indicating “Susan and Paul M. Sniderman Award Nomination” in the subject. All nominations should include a one-page letter motivating the nomination. Self-nominations are accepted. Nomination Deadline: January 30, 2018.
Scientific Committee:
Jordi Munoz, Barcelona, jordi.munoz@ub.edu
Nonna Mayer, Sciences Po, Paris, nonna.mayer@sciencespo.fr
Rune Slothuus, Aarhus, Slothuus@ps.au.dk
Diana C. Mutz, U Penn, mutz@upenn.edu

“The effect of electoral polls in a volatile party system”

por Roberto | May 22, 2017 | General, Research

Basilio Moreno Peralta

It is not unusual to find disagreement between what scientific studies claim and what much of the people think of a given topic. That is the case with electoral polls. When asked in a survey, a large majority of citizens deny any significant influence of electoral polls on their own behaviour on election day. However, political scientists have been discussing about this topic for decades (Gallup & Rae, 1940, Irwin & Van Holsteyn, 2000) and the debate is still alive. One of the reasons is that knowing about polls’ estimates may not affect all voters the same way. Because being more confident about the winning prospects a party has can be a strong argument to abandon another candidacy with farfetched options… If the policy positions they hold are close enough, on the voters’ judgement.

Aiming at understanding better the way in which voters make their mind up when they see changes in the preferences of the rest of society, we wanted to study the possibility of different bandwagon effects (i.e. parties performing well on polls are more likely to be supported by voters given their momentum) for traditional and new political parties in party systems under change. The idea is that, possible electoral gains set an incentive for voters to coordinate around political platforms they support (to some extent). But contexts of profound transformation can make that coordination tough. Under such circumstances, are the estimates of electoral polls more reliable than the previous experience for voters?

Searching for an answer, we set a survey experiment in Spain as a country in which the previous party system has experienced important changes in recent years. Taking advantage of the similar shares of vote that polls reported for PSOE and Unidos Podemos, the two main parties on the left, we wanted to see how people reacted to polls pointing to a clear gain in support of one of these two parties against the other. Considering the uncertainty attached to a reorganization of the party system, would voters rely more on past experiences or polls’ estimates?

What we did to find out was an online survey with the collaboration of NetQuest for a sample of 1,019 people living in Spain. The quota sample was set to meet some general features of the population of interest that could be unfulfilled otherwise—like age distribution. Among other questions, respondents were presented with a picture describing the evolution of the four main national parties in polls. For a third of them, chosen at random, this graph (the treatment) showed no change in the vote share of any party for the last months. Meanwhile, another third saw that in recent times the old-known party on the left, the PSOE, was going up while the newcomer Unidos Podemos was in a clear downwards trend. For the other third of the sample the situation was the opposite: Unidos Podemos was gaining support and PSOE was losing it.

After prompting respondents with that slide, we asked them how likely they were to vote for each of the main four parties on a 0 (not at all) to 10 (completely sure) scale. Additionally, we included a manipulation check asking survey-takers what happened on the treatment picture they saw. This is meant to measure how reliable the treatment effect can be pointed as the cause of observable variations in the experiment. If the participants didn’t pay attention and ignored the treatment vignette, it is unlikely this had any impact on them.

The analysis of the survey experiment takes into consideration how other participants’ features might affect their likelihood to vote for any of those parties. Those features considered are age, sex, educational attainment, ideological self-placement on a left-right scale and the fact that they felt somewhat identified with some party. To account for the fact that politically aware people could be harder to trick with the fake polls, news consumption is considered as a control variable too. The result of the manipulation check described was also computed in the statistical analysis.

What are the results? The graph below shows the probability of voting PP, PSOE, Unidos Podemos or Ciudadanos (on the vertical axis) for people of each of the three treatment groups. The first point on the left depicts the reported probability of voting for people in the treatment group with no change in the polls. The one in the centre is for the group presented with the polls favourable to PSOE and the point on the left is referred to the treatment showing Unidos Podemos on the rise.

The vertical colour lines represent the confidence intervals of the probability estimate. The range of those intervals across columns overlaps most of the times. So, we can find a single value of y that meets those three confidence intervals, which means that most of the results are not statistically significant at 5% or 10% levels. In other words, the differences in the variable probability to vote party n for each treatment group are not large enough (in a sample of this size) to be considered proven. In turn, this results provide some evidence that voters are not affected by polls estimates that much.

However, there is one case in which the predicted probability to vote for one party is indeed affected by the experiment in a scale large enough to be considered. It is not the probability to vote for UP the one that is altered, but the probability to vote for the (incumbent) People’s Party. Supporters of PSOE’s main political opponent in recent decades are more willing to say they will vote for the PP when presented with evidence that the socialist party is gaining ground. PP voters seem to react to tighter electoral competition with greater support for their preferred party, while voters from the other three main Spanish parties do not seem to do so.

Apart from the treatment analysis, the results show ideology self-placement as the strongest predictor for the probability to vote each of the four parties, and that result is statistically significant for PP, Unidos Podemos and Ciudadanos—but not so for PSOE.

Back to those arguments explained at the start, this evidence is not supporting the hypothesis of electoral coordination within ideological blocs; not for the party with more volatile support, due to its recent integration in the party system, nor the veteran with a record of electoral victories in the past. Actually, the entry of new parties has been identified as an obstacle for effective voter coordination in comparative studies. As mentioned earlier, the only group of voters that react to the change in polls’ estimates are PP voters, who increase their probability to vote the party as the opponent PSOE improves its electoral outlook. In conclusion, the only type of voter reaction registered in this experimental setting for the Spanish case is restricted to higher competitiveness between ideological blocs and not electoral coordination within an ideological bloc.

“Should you Sacrifice your Constituents? Moral Dilemmas and the Evaluation of Politicians”

por Roberto | May 2, 2017 | General, Research

Enrique Hernández[1]

Citizens expect politicians to be honest, moral, rational and capable of handling crises, and they are likely to evaluate politicians according to these expectations. However, contravening moral principles might sometimes be the most effective way for politicians to handle crises and maximize aggregate welfare. In some cases, politicians must decide between adopting a utilitarian decision (a decision that, even if it might contravene moral principles, leads to the maximization of aggregate welfare) or a deontological decision (a decision guided by the idea that there are moral standards that should never be violated, even if violating them leads to a maximization of aggregate welfare). How will citizens evaluate a politician that adopts a deontological or utilitarian decision when facing such a dilemma?

To analyze this question, I conducted a survey experiment. The survey experiment was based on a sacrificial moral dilemma that I designed following the classic trolley problem. Specifically, I examined respondents’ perceptions of a politician that in the context of a hypothetical terrorist threat makes either a deontological judgement and decision or a utilitarian judgement and decision.

Respondents read a vignette about an impending terrorist attack that would kill 50 innocent people that had gathered in a public square. Respondents were told that the mayor of the city could stop the attack, but doing so would require that 10 other innocent bystanders travelling in a tramcar near the square die.[2] Half of the respondents read about a mayor that refused to stop the attack (deontological treatment) arguing that “sacrificing 10 innocent people is just wrong, even if doing so saves 50 other innocent people”. The other half read about a mayor that decided to stop the attack (utilitarian treatment) arguing that “it is better to save 50 people, even if doing so requires sacrificing 10 other innocent people”.[3]

After reading about the mayor’s decision, respondents were asked how much they trusted the mayor and how they evaluated the decision he/she adopted. The results summarized in the table indicate that respondents tend to favor deontological decisions.[4] In contrast with mayors that adopt a utilitarian decision, mayors who act in a deontological way tend to be more trusted and their decision is better evaluated by respondents. However, it is important to note that, while the differences between the deontological and utilitarian treatment groups are statistically significant, they are also of reduced magnitude. In any case, the results about the mayor’s trustworthiness are in line with recent findings in social psychology which suggest that upholding a moral principle (for example, sacrificing innocent people is wrong) and acting in line with it—even if it will lead to a sub-optimal outcome—signals trustworthiness.[5]

Respondents were also asked whether they believed that the mayor described in the vignette possessed certain desirable traits such as being rational or moral/ethical.[6] The results summarized in the figure indicate that politicians who act in a utilitarian way are consistently considered more rational, while those who act in a deontological way are considered more moral/ethical. The differences between the deontological and utilitarian treatment conditions are substantially larger regarding the latter characteristic (whether the mayor is a moral/ethical person). This result is probably related to the fact that a substantial number of respondents believe that a mayor who acts in a deontological way can still be considered a rational person, but much fewer respondents believe that a mayor who adopts a utilitarian decision is a moral person.

Figure: Proportion of respondents that believe the mayor is rational and moral/ethical by treatment

Overall, the results of the experiment indicate that the way in which politicians react to moral dilemmas is consequential for how citizens evaluate them. Further research should analyze whether citizens’ evaluations are driven by their own preferences about how moral dilemmas should be resolved, and whether factors, such as individuals’ ideology, moderate the effects of utilitarian and deontological political decisions.

[1] Enrique Hernández is a postdoctoral fellow at the Department of Political Science at the Universitat Autònoma de Barcelona.

[2] See appendix for treatments wording (in Spanish)

[3] I also manipulated the gender of the mayor (male/female). The results suggest that the gender of the mayor does not interact with the deontological/utilitarian treatment for any of the outcomes. The results summarized here do not make any distinction depending on the gender of the mayor.

[4] All results are restricted to those respondents that passed a manipulation check about the deontological/utilitarian treatment (N = 709).

[5] See: Everett, J., Pizarro, D., Crockett, M., 2016. Inference of trustworthiness from intuitive moral judgments. Journal of Experimental Psychology 145, 772–787

[6] Respondents were asked whether the phrases “is rational” and “is ethical” described the mayor “quite well” or “not too well”.

“Public Opinion and Promoting Democracy in Autocracies: Does Oil Matter?”

por Roberto | Abr 6, 2017 | General, Research

Lala Muradova, Doctoral Researcher at KU Leuven, Belgium

Major international events of recent years have shocked many democracies into a discomforting recognition that the mere existence of autocracies abroad could have negative effects for their domestic policies. As of 2016, only 40 percent of the world’s total population lived in fully free countries, according to Freedom House, despite much money and efforts spent by world’s leading democracies and international organizations to promote democracy and the protection of human rights in their external policies. Under the rhetoric of democracy promotion, the United States (US), the European Union (EU) and other actors have increasingly used military interventions (e.g. Haiti, Iraq, Libya); imposed extensive economic or diplomatic sanctions (e.g. South Africa, South Korea, Iran); and spent billions in funding civil society initiatives in non-democracies (e.g. former Soviet Union countries).

However, the democracy promotion policy towards different regimes has rather been inconsistent and heterogeneous, with some regimes receiving important amount of international opprobrium (i.e. North Korea, Iran), while others receiving far less reproach or condemnation (i.e. Middle East, Central Asian countries).  Some attribute this disparity to the strategicness of some autocracies. For example, oil autocracies are a rather “distinctive, and often troublesome, force in the international system”[1] and they differ from other non-democracies chiefly due to their energy importance for western democracies. Some argue that energy security interests propel democracies into turning a blind eye to the violation of human rights in oil autocracies.

How reasonable are these claims? Given the importance of public opinion for foreign policy decision-making in democracies, I have tested this assumption with a research in public opinion.  What shapes public attitudes towards autocracies? Are citizens less hostile to one type of autocracies than to others? Does public support for punitive measures against autocracies vary along the strategicness of the adversary? With an aim of shedding light on these questions I designed, fielded and analyzed original survey experiments in a representative sample of 1019 Spanish citizens.

The experiment consisted of a short vignette describing a hypothetical situation. Individuals were asked to read a series of bullet points containing the detailed information about the situation.[2]  The first point explained that the Human Rights situation in the country X has deteriorated during recent years. It was then followed by a short sentence indicating the region of the country, Western Africa. The next bullet point was about the oil wealth. I randomly varied this information. The country was either an “oil and gas exporter” or “d[id] not export oil or gas”, which was the only manipulated factor between the first and second experimental groups.

Participants were randomly assigned to one of the 3 experimental groups. Those who were in the third group, prior to reading the vignette, were exposed to a short paragraph[3]: “The world remains energetically dependent on fossil fuels, such as oil and gas, raw materials that are in the hands of a few countries. The conflict with these countries may jeopardize the energy supply of a significant part of the population, or cause the increase of the overall price of energy”.  With this manipulation I tested to see if the information about potential costs of punitive measures against an autocracy would deter the citizens from punishing it.  Citizens were later asked to indicate their support for democracy promotion measures in the country.

Findings reveal that while most citizens are in general willing to support democracy promotion abroad (approx. 89 percent), they seem to disfavour a military solution for it (78 percent against).

Figure1. Main effect of the treatment on public support for democracy promotion measures

The fact that the country exports oil and gas has a significant and systematic effect on citizens’ decisions to strike or sanction a foreign autocracy (figure 1). Contrary to my expectations, however, citizens prefer attacking and sanctioning an oil autocracy more than a non-oil one, all else being equal.

Further, I expected that when citizens are primed on potential energy costs of entering into a conflict with an oil exporter, they would be discouraged from punishing the country in question. My empirical findings show, nonetheless, that it is not the case. Irrespective of energy costs, participants in the experiment are still willing to punish an oil exporter more than a non-oil one.

Some critics may contend that the oil is commonly associated with the Muslim Middle East, a sui generis region that holds almost half of the global proven crude reserves and it is difficult to disentangle these factors. However it is an unwarranted speculation in my case, since I tried to curtail this by explicitly indicating that the country X was hypothetical and located in Western Africa.

How can the variation in public opinion be explained? One conjecture is that the mass may consider it especially necessary to promote democracy in oil-rich autocracies, given the potential strategic benefits of democratisation in these countries for western citizens. Another speculative explanation is that the public may consider these measures more effective in destabilising an oil regime, which is heavily reliant on foreign money for its survival in power.

However as these are mere speculations, further research is needed to shed light on possible explanatory factors for the variance in public attitudes towards autocracies.

[1] Ross, M. and Voeten, E. (2015). Oil and international cooperation. International Studies Quarterly, p. 15

[2] The design of the vignette, was similar to the one used by Michael Tomz and Jessica Weeks (2013)

[3] The paragraph was adapted from the website of the Spanish Ministry of Foreign Affairs and Cooperation.

“What the Fact? A Survey Experiment on Post-Truth Politics”

por Roberto | Mar 24, 2017 | General, Research

Berta Barbet[1], Antoni-Italo de Moragas[2] and Guillem Vidal[3]

The victory of Brexit in the United Kingdom’s European Union membership referendum and the election of Donald Trump in the 2016 United States Presidential elections spread the use of the term “post-truth” politics. The expression is used to characterize the circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal beliefs. Beyond its recent popularity, the idea that technical arguments are less efficient at convincing voters than stories with emotional ties had already been proposed in studies on issues as diverse as the attitudes towards immigration (Katwala, Ballinger, & Rhodes, 2015) or the inheritance tax (Graetz & Shapiro, 2005).[4] However, the support sustaining this claim is based mostly on anecdotal evidence and no proper test identifying the extent to which voters are indeed more influenced by emotions and personal belief than facts has been developed yet.

We have conducted an online survey experiment in Spain to test if post-truth is indeed a phenomenon[5]. In the experiment, we asked all respondents to specify their level of agreement or disagreement on a scale from 0 to 10 for raising tuition fees in universities in order to increase the number of nursery places for kids between 0 and 3 years old, a policy they were told would be a more efficient way of spending the resources. The policy at stake was one that has not been politicised by parties or any other influential group in the Spanish political debate. Hence, respondents were expected to have few preconceptions, allowing for any new information to have strong effects.

While the question was the same for all participants, we randomly assigned participants to three different treatment groups. The “expert group” was told that prestigious economist James Heckman had shown that spending at early ages was more efficient at increasing student’s skills. The “normative group” was told that institutions had the responsibility to ensure kids developed their skills. The “emotional group” was shown a picture of a young girl and were told that she would benefit from the policy as currently she could not attend nursery[6]. Lastly, the “control group” was asked the policy question directly.

Comparing the levels of support for the policy of each of the treatments to the control group allows us to isolate the specific effects of providing respondents with different cues[7]. As results in Figure 1 illustrate, all else being equal, the support for the policy is significantly larger for those in the “expert group”. Likewise, those in the “emotional group” specified a greater support for the policy. This was not the case for the “normative group”, where no significant difference was found. Moreover, we find no significant difference in the effects of the emotional and the expert treatments.

Figure 1. Marginal Effects of Treatments.

While being presented with cues from an expert appeared to influence respondent’s level of support for the policy in average, it did not necessarily affect all subgroups of the sample population equally. Motivated by the fact that education has been often pointed out as strong predictor of vote for parties or political options that embrace post-truth politics, we also tested the effect of the treatments depending on the level of education. As Figure 2 shows, education does indeed play a role in determining the impact of the expert treatment. Expert cues increase support towards the policy for respondents with medium and high education. However, the treatment those in the low education group do not seem to be affected by the expert cue.

Figure 2. Marginal Effects of Treatments by Education Levels

To wrap up, we find that both experts’ cues and emotional cues increase the support of a given policy but there is no evidence of citizens responding more to emotions than to experts and evidence. It is true that individuals with lower education seem to respond less to the expert’s advice, a similar finding when looking at the groups overrepresented in the Brexit or Trump vote. However, they also seem to respond less to the emotional treatment too, albeit the difference is not significant in that case. Further research should try to check if these dynamics are sustained in other settings and with other policies. So far, the ironic implication of this finding is that the growing concern that citizens ignore facts presented by experts does not seem to be based on facts.

[1] Berta Barbet is a Postdoctoral fellow at the Department of Political Science at the Universitat Autònoma de Barcelona.

[2] Antoni-Italo de Moragas is a Ph.D. Researcher in the Department of Economics at the European University Institute.

[3] Guillem Vidal is a Ph.D. Researcher in the Department of Social and Political Sciences at the European University Institute.

[4] Graetz, M. J., & Shapiro, I. (2005). Death by a thousand cuts: The fight over taxing inherited wealth. Princeton: Princeton University Press.

Katwala, S., Ballinger, S., & Rhodes, Vm. (2015). How to talk about immigration. London.

[5] N=1000. Netquest’s online panel in Spain. See: http://www.ub.edu/polexp/resolucion-convocatoria-polexp-netquest/

[6] See appendix for concrete wording of the treatments.

[7] The results here presented are the marginal effects of a model that controls for age, gender, education, region, economic preferences, political information, level of trust in modern science, saliency of the issue for the respondent and whether or not respondents have sons age 7 to 25