Ethical Issues in AI-Generated Political Content: Deepfakes, Softfakes, and Responsibility in Political Communication

Abstract

Generative artificial intelligence has created new possibilities for political communication while raising ethical concerns about authenticity, disclosure, manipulation, public trust, and accountability. This article examines these issues through a qualitative comparative analysis of two 2024 cases: the AI-generated robocall imitating Joe Biden during the New Hampshire Democratic primary and the AI-assisted “Gemoy” persona used in Indonesia’s presidential campaign. Drawing on Hans Jonas’s responsibility ethics and rule-utilitarianism, the study distinguishes between deepfakes that fabricate political statements and softfakes that enhance political representation through AI-generated imagery. The analysis argues that synthetic political content is not inherently unethical; its ethical status depends on how it is produced, disclosed, distributed, and used. Deepfakes may undermine the evidentiary value of authentic political communication, while softfakes may influence political judgement through affective framing. Transparency and labelling are therefore necessary but insufficient. The article proposes a shared responsibility approach in which political actors, platforms, media institutions, developers, and audiences bear differentiated responsibilities according to their capacity to prevent or amplify harm.

Introduction

The text examines the ethical implications of generative AI in political communication, focusing on how AI-generated deepfakes and softfakes can affect authenticity, public trust, and political judgement.

It compares two cases from the 2024 electoral cycle:

• New Hampshire robocall: An AI-generated voice impersonated Joe Biden and discouraged voters from participating in the Democratic primary. Because the synthetic identity was concealed, the case involved direct deception and raised concerns about manipulation, political misinformation, and the reliability of audio evidence.

• “Gemoy” persona in Indonesia: AI-assisted images of Prabowo Subianto presented a younger, friendlier, and more approachable political image. Unlike the robocall, the main concern was not fabricated speech but affective framing—using synthetic imagery to influence how audiences perceive a candidate.

The study uses a qualitative comparative case-analysis method, relying on secondary sources such as regulatory records, academic research, institutional documents, and journalism. The cases are analysed according to four dimensions: authenticity, disclosure, audience capacity, and responsibility. The analysis applies Jonas’s theory of responsibility and a rule-utilitarian perspective derived from Mill.

The findings suggest that deepfakes and softfakes create different ethical problems. Deepfakes can deceive audiences about what a politician actually said or did, while softfakes can shape perceptions without necessarily making a false factual claim. Deepfakes may also contribute to the “liar’s dividend,” where widespread synthetic media makes people less certain that genuine political evidence is authentic.

The text argues that responsibility should not rest solely with audiences. Political campaigns, content producers, platforms, telecommunications providers, media organisations, and technology developers all have different capacities to prevent or reduce harm. However, responsibility should be distributed according to each actor’s ability to foresee, influence, and mitigate the consequences of synthetic political communication.

Conclusion

This article examined the ethical implications of AI-generated political communication through a comparison of the New Hampshire AI robocall and the Gemoy persona in Indonesia’s 2024 presidential campaign. The analysis shows that the ethical implications of synthetic political content depend not simply on the use of AI or the presence of factual falsehood, but on the relationship between technological power, communicative purpose, disclosure, audience capacity, and broader consequences. The two cases reveal different mechanisms of ethical risk. The New Hampshire robocall fabricated political speech and directly challenged the evidentiary value of recorded communication, while the Gemoy persona illustrates how AI-assisted representation may shape political perception through affective framing without necessarily fabricating a specific proposition. Responsibility should consequently be shared across the communication system but remain proportional to actors’ technological and communicative capabilities. Political actors bear primary responsibility for the purpose and use of synthetic content, while platforms, infrastructures, media organisations, developers, and audiences hold different supporting responsibilities. The analysis also shows that transparency is necessary but not sufficient. Disclosure can undermine concealed impersonation, as in the New Hampshire case, but may not resolve the persuasive or representational function of openly artificial content. The study therefore proposes functional disclosure, which considers not only whether AI was used but also what the synthetic content is intended to accomplish. The study has several limitations. Its qualitative comparative design limits generalisability, the Gemoy case relies largely on secondary evidence, and the study does not establish audience-level causal effects. Future research should therefore combine documentary analysis with audience experiments, platform data, and multilingual empirical research. Overall, responsible governance of political AI should address not only whether content is authentic, but also who creates, distributes, amplifies, and contextualises synthetic representations. Transparency should serve as a starting point for accountability, complemented by capability-proportional responsibility across the political communication system.

References

[1] S Battista, D. (2024). Political communication in the age of artificial intelligence: An overview of deepfakes and their implications. Society Register, 8(2), 7–24. [2] Chesney, R., & Citron, D. K. (2019). Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, 107(6), 1753–1820. [3] Chomanski, B., & Lauwaert, L. (2025). Automated propaganda: Labeling AI-generated political content should not be required by law. Journal of Applied Philosophy, 42(3), 994–1015. [4] Chowdhury, R. (2024). AI-fuelled election campaigns are here—where are the rules? Nature, 628(8007), 237. [5] Coeckelbergh, M. (2020). Artificial intelligence, responsibility attribution, and a relational justification of explainability. Science and Engineering Ethics, 26(4), 2051–2068. [6] Conrad, C., Nissen, A., Masoumi, K., Ramchandani, M., Fecury Braga, R., & Newman, A. J. (2025). Do truthfulness notifications influence perceptions of AI-generated political images? A cognitive investigation with EEG. Computers in Human Behavior: Artificial Humans, 5, Article 100185. [7] Diakopoulos, N., & Johnson, D. (2021). Anticipating and addressing the ethical implications of deepfakes in the context of elections. New Media & Society, 23(7), 2072–2098. [8] Dignum, V. (2019). Responsible artificial intelligence: How to develop and use AI in a responsible way. Springer. [9] Dobber, T., Kruikemeier, S., Helberger, N., & Goodman, E. (2024). Shielding citizens? Understanding the impact of political advertisement transparency information. New Media & Society, 26(11), 6715–6735. [10] Dobber, T., Metoui, N., Trilling, D., Helberger, N., & de Vreese, C. (2021). Do (microtargeted) deepfakes have real effects on political attitudes? The International Journal of Press/Politics, 26(1), 69–91. [11] Fallis, D. (2021). The epistemic threat of deepfakes. Philosophy & Technology, 34(4), 623–643. [12] Federal Communications Commission. (2024). In the matter of Steve Kramer: Notice of apparent liability for forfeiture (File No. EB-TCD-24-00036094, FCC 24-59). [13] Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines, 28(4), 689–707. [14] Groh, M., Sankaranarayanan, A., Singh, N., Kim, D. Y., Lippman, A., & Picard, R. (2024). Human detection of political speech deepfakes across transcripts, audio, and video. Nature Communications, 15, Article 7629. [15] Hameleers, M., van der Meer, T. G. L. A., & Dobber, T. (2024). Distorting the truth versus blatant lies: The effects of different degrees of deception in domestic and foreign political deepfakes. Computers in Human Behavior, 152, Article 108096. [16] Helberger, N., Pierson, J., & Poell, T. (2018). Governing online platforms: From contested to cooperative responsibility. The Information Society, 34(1), 1–14. [17] Jonas, H. (1984). The imperative of responsibility: In search of an ethics for the technological age. University of Chicago Press. [18] Kreiss, D., & McGregor, S. C. (2018). Technology firms shape political communication: The work of Microsoft, Facebook, Twitter, and Google with campaigns during the 2016 U.S. presidential cycle. Political Communication, 35(2), 155–177. [19] Kuipers, N., et al. (2024). Third time’s the charm: The youth vote and Prabowo’s victory in the 2024 Indonesian presidential election. Pacific Affairs. (Preprint/Online ahead of print). [20] Mill, J. S. (2001). Utilitarianism (G. Sher, Ed.). Hackett Publishing. (Original work published 1863) [21] Nightingale, S. J., & Farid, H. (2022). AI-synthesized faces are indistinguishable from real faces and more trustworthy. Proceedings of the National Academy of Sciences, 119(8), e2120481119. [22] Nissenbaum, H. (1996). Accountability in a computerized society. Science and Engineering Ethics, 2(1), 25–42. [23] Pawelec, M. (2022). Deepfakes and democracy (theory): How synthetic audio-visual media for disinformation and hate speech threaten core democratic functions. Digital Society, 1, Article 19. [24] Pawelec, M., & ?abuz, M. (2026). Are softfakes really “soft”? Semantic, ethical, and legal perspectives on deepfakes and AI-generated visualizations in political campaigning. International Journal of Politics, Culture, and Society. [25] Rini, R. (2020). Deepfakes and the epistemic backstop. Philosophers’ Imprint, 20(24), 1–16. [26] Sunstein, C. R. (2017). #Republic: Divided democracy in the age of social media. Princeton University Press. [27] Thompson, D. F. (1980). Moral responsibility of public officials: The problem of many hands. American Political Science Review, 74(4), 905–916. [28] Vaccari, C., & Chadwick, A. (2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society, 6(1).

Copyright

Copyright © 2026 Xinran Wang, Dr. Mohd Haizra Hashim. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.