Detecting and responding to moralised misinformation

Outline

The concept of moral misinformation.
A tool for detecting moral messages.
An application of the tool.
Potential applications for the Russo-Ukraine War.

“Now, what if these [American] biolabs are handling very dangerous pathogens that may lead to the deaths in people in the areas? There are reports of possibly deadly pathogens escaping these biolabs in places like Ukraine, Georgia, Kazakhstan, that is responsible for killing people.” (Greene, 2022)

The consequences of moral convictions

License to be uncompromising
- Intolerance of disagreement (Garrett & Bankert, 2020; Skitka, Hanson, Morgan, & Wisneski, 2021).
- Harder to persuade (Kodapanakkal, Brandt, Kogler, & van Beest, 2022; Ryan, 2017).

License to act with with the like-minded
- Cooperation and selflessness (Curry, Mullins, & Whitehouse, 2019; Hoover, Johnson, Boghrati, Graham, & Dehghani, 2018; Spring, Cameron, & Cikara, 2018).
- Violence and hate (Kennedy et al., 2023; Solovev & Pröllochs, 2022).

License to share falsehoods
- Moral contagion (Brady, Gantman, & Van Bavel, 2020; Brady, Jackson, Lindström, & Crockett, 2023; Brady & Van Bavel, 2021).
- Illusory truth effects (Ecker et al., 2022).
- Pre-factual thinking and gist (Helgason & Effron, 2022).

Uncompromising
People who hold moralised attitudes, which portray an entity as morally righteous or evil, can be unwilling to compromise on their positions and resist counter-attitudinal messages, sometimes with anger
Act with the like-minded
(+) cooperation, selflessness; (-) violence; hate.
Collaborate more with an in-group and distance oneself from the out-group (differently-minded).
Compels us to put group’s goals before our own. Can be a source of cooperation and selflessness, but might compel one to permit or enact violence and hate.
Share falsehoods.
Generally, people are more likely to share negative, moralised messages on social media.
Repeated exposure to these messages can create feelings of familiarity, which conjure an illusion of truth.
On top of this, moral convictions might encourage us to share misinformation or excuse those who circulate falsehoods.
That is, disseminating misinformation can become a moral task in its own right.
In what has been called ‘pre-factual thinking’, one might share or permit the sharing of falsehood when the gist of that falsehood is congruent with a moral stance.
These three licenses give moral misinformation a broad and powerful appeal.

Moral Foundations Theory

Care/Harm

👼
Caring
Kindness

😈
Cruelty

Fairness/Cheating

👼
Fairness
Justice
Honesty
Trustworthiness

😈
Dishonesty

Loyalty/Betrayal

👼
Loyalty
Patriotism
Self-sacrifice

😈
Cowardice

Authority/Subversion

👼
Obedience
Deference

😈
Disobedience

Purity/Degradation

👼
Temperance
Chastity
Piety
Cleanliness

😈
Over-indulgence
Lust

An application: moralisation of the Great Barrier Reef by climate change contrarians

Broad scientific consensus that climate change is predominantly caused by human activity (Cook et al., 2013; Većkalov et al., 2023).

Contrarian arguments challenge scientific consensus and policy implications (Coan, Boussalis, Cook, & Nanko, 2021; Lamb et al., 2020).

Great Barrier Reef is a battleground for climate change debates (Konkes & Foxwell-Norton, 2021).

There is a broad scientific concensus that climate change is predominantly caused by human activity
However, contrarian arguments challenge scientific consensus or its policy implications, such as by claiming climate change is part of a natural fluctuation or that the consesquences are overstated by scientists.
Sometimes, this misleading contrarian messages might directly challenge scientific conclusions, but other times, they may take the form of diffuse messages that erode support for climate change mitigation policies.
The Great Barrier Reef is a historic battleground, where activists and contrarians have vied to persuade others of their cause.
Consider coral bleaching, where coral under stress loses their vibrant colours. A climate change activist might say …, whereas a contrarian might say …

An application: moralisation of the Great Barrier Reef by climate change contrarians

Research Question

Do climate change contrarians post more moralised messages about the Great Barrier Reef than other users who post about climate change?

Collected tweets from CSIRO’s Emergency Situation Awareness archive (CSIRO, 2023; Power, Robinson, & Cameron, 2023; Power, Robinson, Colton, & Cameron, 2014).
Retrieved English tweets posted by Australian users that mentioned the Great Barrier Reef.

An application: moralisation of the Great Barrier Reef by climate change contrarians

Contrarian

Used a contrarian phrase in a Great Barrier Reef tweet:
- “#ClimateCult”
- “climate hysteria”
- “alarmist”

564 users

7,797 tweets about the Great Barrier Reef

Baseline

Used a climate change phrase in a Great Barrier Reef tweet:
- “climate change”
- “#ClimateAction”
- “#ClimateEmergency”

16,862 users

187,081 tweets about the Great Barrier Reef

An application: moralisation of the Great Barrier Reef by climate change contrarians

Other applications

Identify moral characteristics of dis/misinformation
Detecting success of disinformation campaign
Identify users who may be predisposed to believe or share dis/misinformation

“Now, what if these [American] biolabs are handling very dangerous pathogens that may lead to the deaths in people in the areas? There are reports of possibly deadly pathogens escaping these biolabs in places like Ukraine, Georgia, Kazakhstan, that is responsible for killing people…

“What if that’s true? I think these are questions that we should ask because no american citizen wants to be held morally and ethically responsible and the U.S. government should not be funding something that’s killing people in a country that’s not even our own, let alone here at home.” (Greene, 2022)

Summary

Misinformation embedded in morals can be particularly potent

Moral convictions can motivate people to be uncompromising, be hostile to those with different views, and share misinformation.
We can detect moral messages with Natural Language Processing techniques.
Climate change contrarian tweets have a unique moral signature.

Misinformation may be more consequential and likely when messages are moral.
Moral messages can be detected using Natural Language Processing techniques.
By detecting morals, we can better understand users and messages.
Disinformation and misinformation are often embedded in broader moral convictions of what is right and wrong.
Moral convictions can motivate people to be uncompromising, be hostile to those with different views, and share misinformation.
Moralised messages are not necessarily false.
Detecting moralised messages can help identify issues where falsehoods are more likely to be shared, to be believed, and to be consequential.
Moralised messages can be detected using Natural Language Processing techniques.
An application demonstrated users who posted climate change contrarian content are more likely to post moralised messages about the Great Barrier Reef than other users.

Link to slides: matt-lab.github.io/workshop_moral-misinformation

Email: matthew.andreotta@csiro.au

References

Brady, W. J., Gantman, A. P., & Van Bavel, J. J. (2020). Attentional capture helps explain why moral and emotional content go viral. Journal of Experimental Psychology. General, 149(4), 746–756. https://doi.org/10.1037/xge0000673

Brady, W. J., Jackson, J. C., Lindström, B., & Crockett, M. (2023). Algorithm-Mediated Social Learning in Online Social Networks. OSF Preprints. https://doi.org/10.31219/osf.io/yw5ah

Brady, W. J., & Van Bavel, J. J. (2021). Estimating the effect size of moral contagion in online networks: A pre-registered replication and meta-analysis. OSF Preprints. https://doi.org/10.31219/osf.io/s4w2x

Coan, T. G., Boussalis, C., Cook, J., & Nanko, M. O. (2021). Computer-assisted classification of contrarian claims about climate change. Scientific Reports, 11(1), 22320. https://doi.org/10.1038/s41598-021-01714-4

Cook, J., Nuccitelli, D., Green, S. A., Richardson, M., Winkler, B., Painting, R., … Skuce, A. (2013). Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature. Environmental Research Letters, 8(2), 1–7. https://doi.org/10.1088/1748-9326/8/2/024024

CSIRO. (2023). Emergency Situation Awareness. https://esa.csiro.au/aus/about-public.html.

Curry, O. S., Mullins, D. A., & Whitehouse, H. (2019). Is It Good to Cooperate?: Testing the Theory of Morality-as-Cooperation in 60 Societies. Current Anthropology, 60(1), 47–69. https://doi.org/10.1086/701478

Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., … Amazeen, M. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), 13–29. https://doi.org/10.1038/s44159-021-00006-y

Garrett, K. N., & Bankert, A. (2020). The Moral Roots of Partisan Division: How Moral Conviction Heightens Affective Polarization. British Journal of Political Science, 50(2), 621–640. https://doi.org/10.1017/S000712341700059X

Greene, M. T. (2022). U.S. House of Representatives: House Session. C-SPAN.

Helgason, B. A., & Effron, D. A. (2022). It might become true: How prefactual thinking licenses dishonesty. Journal of Personality and Social Psychology, 123(5), 909–940. https://doi.org/10.1037/pspa0000308

Hoover, J., Johnson, K., Boghrati, R., Graham, J., & Dehghani, M. (2018). Moral Framing and Charitable Donation: Integrating Exploratory Social Media Analyses and Confirmatory Experimentation. Collabra: Psychology, 4(1), 9. https://doi.org/10.1525/collabra.129

Kennedy, B., Golazizian, P., Trager, J., Atari, M., Hoover, J., Mostafazadeh Davani, A., & Dehghani, M. (2023). The (moral) language of hate. PNAS Nexus, 2(7), pgad210. https://doi.org/10.1093/pnasnexus/pgad210

Kodapanakkal, R. I., Brandt, M. J., Kogler, C., & van Beest, I. (2022). Moral relevance varies due to inter-individual and intra-individual differences across big data technology domains. European Journal of Social Psychology, 52(1), 46–70. https://doi.org/10.1002/ejsp.2814

Konkes, C., & Foxwell-Norton, K. (2021). Science communication and mediatised environmental conflict: A cautionary tale. Public Understanding of Science, 30(4), 470–483. https://doi.org/10.1177/0963662520985134

Lamb, W. F., Mattioli, G., Levi, S., Roberts, J. T., Capstick, S., Creutzig, F., … Steinberger, J. K. (2020). Discourses of climate delay. Global Sustainability, 3, e17. https://doi.org/10.1017/sus.2020.13

Power, R., Robinson, B., & Cameron, M. (2023). Insights from a decade of Twitter monitoring for emergency management. Proceedings of the Information Systems for Crisis Response and Management Asia Pacific Conference 2022.

Power, R., Robinson, B., Colton, J., & Cameron, M. (2014). Emergency Situation Awareness: Twitter case studies. Information Systems for Crisis Response and Management in Mediterranean Countries, 218–231. Switzerland: Springer International Publishing. https://doi.org/10.1007/978-3-319-11818-5_19

Ryan, T. J. (2017). No Compromise: Political Consequences of Moralized Attitudes. American Journal of Political Science, 61(2), 409–423. https://doi.org/10.1111/ajps.12248

Skitka, L. J., Hanson, B. E., Morgan, G. S., & Wisneski, D. C. (2021). The Psychology of Moral Conviction. Annual Review of Psychology, 72(1), 347–366. https://doi.org/10.1146/annurev-psych-063020-030612

Solovev, K., & Pröllochs, N. (2022). Moralized language predicts hate speech on social media. PNAS Nexus, pgac281. https://doi.org/10.1093/pnasnexus/pgac281

Spring, V. L., Cameron, C. D., & Cikara, M. (2018). The Upside of Outrage. Trends in Cognitive Sciences, 22(12), 1067–1069. https://doi.org/10.1016/j.tics.2018.09.006

Većkalov, B., Geiger, S. J., White, M. P., Rutjens, B., Harreveld, F. van, Stablum, F., … Linden, D. S. van der. (2023). A 27-country test of communicating the scientific consensus on climate change. OSF Preprints. https://doi.org/10.31219/osf.io/bctm3