Large Language Models for the Automated Detection and Classification of Media Bias and Propaganda to foster Media Literacy among News Audiences

Abstract

Media bias is an enduring feature of news dissemination, reflecting the subjective perspectives of its creators across history. From archaic records like "The Victory Stele of Naram-Sin" to contemporary news channels, bias permeates media, influencing political, social, and public health narratives. This research aims to investigate the persistent phenomenon of media bias and the potential of large language models (LLMs)(Kojima et al., 2022) in its detection and classification, in order to deploy publicly available software tools aiming to enhance media literacy among news consumers.

Traditionally, media bias served the interests of ruling powers; even with the rise of modern journalism, objectivity is often compromised by commercial pressures and inherent human biases. (Rodrigo-Ginés et al., 2024). As media landscapes evolve, bias continues to shape public opinion, impacting democratic processes and public health perceptions—evident during the COVID-19 pandemic, where polarized media narratives swayed public health decisions and fueled misinformation. (Recio-Román et al., 2023)

Current research on the effects of labeling media bias or propaganda, whether automatically or with human involvement, highlights the complexity of the issue. Depending on different circumstances, labeling can lead to negative outcomes (such as reinforcing filter bubbles by providing means to avoid news with a different perspective), no change in news consumption behavior at all, or, in some cases, an actual improvement in media literacy as intended (Zavolokina et al., 2024).

This research aims to develop a technical solution for the automatic labeling of biased media content, emphasizing several proposals that we hope will lead to a positive effect on media literacy among those presented with the system’s assessments.

These proposals include using a fine-grained taxonomy of bias types rather than a simple binary left/right labeling, focusing on detailed explanations for each model decision in natural language, marking bias at the sentence level rather than at the article or publication level to provide more insights, fine-tuning autoregressive models like GPT-3.5 or Mistral with high-quality examples instead of using “simple” bidirectional models like BERT(Brown et al., 2020) or non-finetuned models, and focusing on the German language, which has not yet been properly explored for such systems.

Understanding readers' perceptions when exposed to bias-labeled content is another facet of this research. It will explore how bias labeling influences readers' views on credibility and neutrality and whether real-time bias indicators affect news consumption behaviors. As mentioned, practical applications serve as a cornerstone of this research. One aim is to implement bias detection systems in real-world settings, such as search engines and news aggregators, to promote balanced information consumption. The development of user tools, like browser extensions highlighting media bias, intends to address public need for transparent information evaluation.

In essence, this research contributes to media literacy enhancement by demystifying media bias through advanced computational methods. By refining detection mechanisms, classifying bias more effectively, and implementing practical tools, it aims to fortify democratic discourse and public understanding, thereby addressing the pervasive influence of media bias in today’s interconnected world.

References

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems (Vol. 33, pp. 1877–1901). Curran.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, 35, 22199–22213.

Recio-Román, A., Recio-Menéndez, M., & Román-González, M. V. (2023). Influence of Media Information Sources on Vaccine Uptake: The Full and Inconsistent Mediating Role of Vaccine Hesitancy. Computation (Basel). https://doi.org/10.3390/computation11100208

Rodrigo-Ginés, F.-J., Carrillo-de-Albornoz, J., & Plaza, L. (2024). A systematic review on media bias detection: What is media bias, how it is expressed, and how to detect it. Expert Systems with Applications, 237, 121641.

Keywords: Media Bias, Large Language Models, Bias Detection, Natural Language Processing, Journalism, Public Opinion, Taxonomy

Mehr zum Titel