Experiments reveal that it is difficult for people to separate the quality of an argument from their own political opinions
When considering political topics, we need to be able to fairly assess the trustworthiness of various claims and evidence. However, multiple experiments have shown that people have difficulty separating their pre-existing political opinions from the quality of the claims.
For everyday arguments prior beliefs play a larger role on perceived argument quality than argument quality itself - ScienceDirect
People struggle to separate argument quality from their own political opinions
https://www.psypost.org/people-struggle-to-separate-argument-quality-from-their-own-political-opinions/
Most people tend to use cognitive shortcuts, such as 'trustworthiness of the speaker,' 'consistency with their own opinions,' 'emotional language,' and 'social cues,' to determine which claims to trust. In contrast, people with high media literacy are thought to avoid automatic shortcuts and instead assess the legitimacy of claims by assessing the source of information, the context and logical consistency of the claim, and misleading images and headlines.
Calvin Deans-Brown , a doctoral student in the Department of Psychology and Linguistics at University College London, and his colleagues conducted an experiment to examine how people evaluate the quality of arguments for political causes such as the legalization of abortion.
'I began researching this topic during the height of the COVID-19 pandemic in 2020, a time when poor quality information was rampant, including, unfortunately, misinformation about how COVID-19 spreads and how to protect yourself from the virus,' Deans-Brown said.
The research team conducted three experiments to investigate the relationship between the quality of claims and people's pre-existing beliefs. In each experiment, the quality of claims was manipulated by controlling the degree to which the information presented in each claim was related to the core of the claim.
'Good arguments' included statistical evidence supporting claims such as, 'The gun murder rate in the United States is 25 times higher than the average for 22 similarly high-income countries,' as well as causal evidence such as, 'When we heat our homes, run our cars, and operate our factories, our emissions warm the planet.' On the other hand, 'bad arguments' contained significantly weaker evidence and contained a variety of flaws, such as
In the first experiment, 101 participants were presented with eight political topics and asked to report their beliefs on a scale ranging from 'very false' to 'very true.' They were then asked to read claims about each topic and rate the quality of the claims.
The results showed that participants did indeed distinguish between 'good' and 'bad' arguments, but the breakdown of their distinctions varied between topics they supported and those they disagreed with. Quantitatively, the participants' agreement with a topic also correlated with their evaluation of the quality of the argument.
In the graph below, the vertical axis represents the evaluation of the quality of the arguments, with the higher the axis, the higher the quality is rated. The horizontal axis represents the degree of support for the topic, with the left indicating more opposition and the right indicating more support. Of the two graphs, the top graph represents the evaluation of 'arguments in favor of the topic,' while the bottom graph represents the evaluation of 'arguments against the topic.' Blue represents 'good arguments' and yellow represents 'bad arguments.' We can see that, overall, subjects rated 'good arguments' higher, but when it came to topics they supported, they rated the 'bad arguments' of the supporting opinions higher and the 'good arguments' of the opposing opinions lower, with the opposite trend being seen for topics they did not support.
In the second experiment, the same topics and claims were used, but half of the 200 participants were assigned to a group where they first rated the quality of the claim and then reported their beliefs about the topic, and the other half were assigned to a group where they first reported their beliefs about the topic and then rated the quality of the claim.
The purpose of this study was to explore whether describing the topic before rating the claims might affect how people evaluated the claims. However, the results of the second experiment were similar to those of the first: the order in which participants reported their support for the topic and rated the quality of the claims did not affect the results.
In a third experiment, 101 participants were shown 'bad claims' that were systematically manipulated: half were based on contradictory evidence, and half were based on appeals to authority. In the contradictory claims, some evidence supported the claim, while the other evidence contradicted it.
The results of the experiment confirmed that participants tended to rate arguments based on 'contradictory evidence' as higher quality than arguments based on 'appeals to authority.' Left-leaning people rated 'contradictory arguments in favor of left-leaning topics' higher, while right-leaning people rated 'contradictory arguments in favor of right-leaning topics' higher. In other words, people's pre-existing beliefs influenced their evaluations of the quality of arguments.
'The contradictory claims we used in our study don't make much sense when read carefully, because claims cannot simultaneously support opposing viewpoints,' Deans-Brown said. 'We were therefore surprised that participants considered these contradictory claims superior to claims based on appeals to famous people, which are much easier to understand.'
The results of this study suggest that people's pre-existing political beliefs influence their evaluations of arguments on political topics, but that making a good argument remains important, as participants generally rated 'good arguments' based on good evidence and arguments more highly.
'I want to emphasize that while people tend to view claims that are consistent with their personal beliefs more positively, they also view claims that are supported by high-quality evidence more positively,' Deans-Brown said. 'While I point out the limitations of trying to persuade people with high-quality evidence alone, I don't want to give the impression that basing claims on high-quality evidence is a futile endeavor or that such claims should not be made.'
Related Posts:
in Science, Posted by log1h_ik