Data Privacy in AI Chatbots: Understanding Risks and Ways to Protect User Information

Abstract

The point is as follows: we are all discussing conversational AI based on huge language models, in the areas where privacy is incredibly important, such as healthcare and finance. However, there is a massive struggle between the necessity of these models to process huge quantities of information and the inability to violate the law regardless of the cost in terms of privacy. These LLMs also search through huge heaps of sensitive data, which increases the chances of spills by a significant margin. Throughout this paper, I take a close look at the privacy traps that are presented at every step of the life of an LLM, beginning with the manner in which we collect vast amounts of data, all the way to the ways in which people might attempt to actively attempt to peep inside. I compare all three leading privacy- enhancing tools: Differential Privacy, Federated Learning, and Secure Multi-Party Computation by discussing their strengths and weaknesses as well as the additional effort they require and the potential impact of extra effort on the utility of the model.

Introduction

This paper examines the privacy and security challenges of Large Language Models (LLMs) and evaluates modern Privacy-Enhancing Technologies (PETs) and deployment architectures that can protect sensitive data while maintaining model performance.

Background and Motivation

LLMs have revolutionized natural language processing across industries, but their reliance on massive datasets creates significant privacy risks. Training data often contains sensitive information such as personally identifiable information (PII), health records, financial data, and biometric information. As model sizes and datasets grow, the likelihood of data leakage, misuse, and privacy violations increases, making robust technical and governance solutions essential.

Core Problem

The primary challenge is developing practical privacy-preserving methods that maintain model accuracy and usability. Existing privacy techniques often require a trade-off between privacy and performance, while conversational AI introduces new attack vectors that traditional cybersecurity measures cannot adequately address. The study aims to compare modern privacy mechanisms and identify the most effective approaches.

Privacy Threats in LLMs

The literature review categorizes privacy risks throughout the LLM data lifecycle:

• Data Collection Risks: Lack of transparency, unauthorized data usage, and governance failures can erode user trust.
• Passive Leakage Risks: Misconfigurations, system vulnerabilities, or model outputs may unintentionally expose sensitive information.
• Model Memorization: LLMs can memorize training data, enabling attacks that reveal whether specific records were included in training.
• Prompt Injection Attacks: Malicious prompts can manipulate models into revealing confidential information or bypassing safeguards.
• Membership Inference Attacks (MIAs): Attackers attempt to determine whether a particular individual's data was used in training.

Privacy-Enhancing Technologies (PETs)

Several advanced privacy-preserving approaches are discussed:

• Differential Privacy (DP): Adds controlled noise during training to prevent identification of individual records, though excessive noise can reduce model accuracy.
• Federated Learning (FL): Keeps data on local devices while sharing only model updates, reducing centralized data exposure.
• Secure Multi-Party Computation (SMPC): Enables collaborative computation without revealing participants' private data.
• Trusted Execution Environments (TEEs): Hardware-based secure enclaves that isolate sensitive computations from the rest of the system.

Research Methodology

The study evaluates privacy mechanisms across three deployment architectures:

1. Cloud-Based LLMs – High performance and scalability but greater privacy risks due to remote data processing.
2. On-Device LLMs – Strong privacy and low latency since data remains on user devices, though computational resources are limited.
3. Hybrid LLMs – Combine local processing for sensitive data with cloud resources for complex tasks, balancing privacy and performance.

Evaluation criteria include:

• Privacy guarantees
• Computational overhead
• Utility degradation
• Data classification and sanitization effectiveness
• Mean Time to Detect (MTTD) privacy incidents

Key Findings

Differential Privacy

• Provides strong privacy guarantees but often reduces model performance.
• Advanced techniques such as improved noise mechanisms can reduce the privacy-utility trade-off.

SMPC and TEEs

• Offer strong cryptographic protection for training and inference.
• Particularly valuable in sensitive domains like healthcare and finance.
• However, they introduce significant computational and communication overhead.

Deployment Architecture Trade-offs

• On-device models maximize privacy and reduce latency but have limited computational capacity.
• Cloud models offer scalability and easier updates but require transmitting user data to remote servers.
• Hybrid architectures provide the best balance between privacy, performance, and regulatory compliance.

Data Sanitization and Runtime Defenses

• Pre-training data cleaning, masking, and tokenization help prevent memorization of sensitive information.
• Runtime defenses such as prompt validation, reverse proxies, middleware filtering, and output monitoring help mitigate prompt injection attacks and accidental data disclosure.

Conclusion

The issue of privacy of data on AI chatbots essentially falls on the cross of technical design and operation, regulation and organizational culture. There are special difficulties in conversations since they are abundant and unstructured. Threats such as inference attacks, regulatory fines and shadow AI information leakage are not far-fetched as I Have read, however, a combination of privacy- enhancing technology, good governance, and user-focused controls can mitigate these risks significantly. In the future, studies ought to explore ways of improving the utility privacy trade-off of differentiating privacy of language models, enhancing secure and robust federated learning systems, and establishing more practical and effective unlearning systems to support deletion rights. To enable policymakers and practitioners to coordinate around more transparent, explainable, and accountable conversational AI, it will be necessary to ensure that chatbots develop in a way that supports our rights without compromising on the ability to provide genuine user and institutional benefits.

References

[1] [2406.07973] Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey-arXiv,accessedonNovember26,2025, https://arxiv.org/abs/2406.07973 [2] A Complete Survey on LLM-based AI Chatbots - arXiv, accessed on November 26, 2025,https://arxiv.org/pdf/2406.16937? [3] AI Privacy Risks & Mitigations–Large Language Models (LLMs) - European Data Protection Board, accessed on November 26, 2025,https://www.edpb.europa.eu/system/files/2025-04/ai-privacy-risks-and-mitigations-in-llms.pdf [4] The Right to Be Forgotten—But Can AI Forget?|CSA-Cloud Security Alliance, accessed on November 26, 2025, https://cloudsecurityalliance.org/blog/2025/04/11/the-right-to-be-forgotten-but-can-ai-forget [5] Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data-arXiv,accessedonNovember26,2025, https://arxiv.org/html/2503.01630v2 [6] Supervised Fine-Tuning(SFT) for PII Masking Using Axolotl - Medium,accessedonNovember26,2025, https://medium.com/@aakulkarni/supervised-fine-tuning-sft-for-pii-masking-using-axolotl-c306f3245bc6 [7] A Case Study on Samsung\'s ChatGPT Incident-Human Firewall, accessed on November 26, 2025,https://humanfirewall.io/case-study-on-samsungs-chatgpt-incident/ [8] Be Careful What You Tell Your AI Chatbot| Stanford HAI, accessed on November 26, 2025, https://hai.stanford.edu/news/be-careful-what-you-tell-your-ai-chatbot [9] Security Concerns for Large Language Models: A Survey-arXiv,accessedon November 26, 2025, https://arxiv.org/html/2505.18889v2 [10] Unveiling AI Agent Vulnerabilities Part III: Data Exfiltration | Trend Micro(US), accessed on November 26, 2025, https://www.trendmicro.com/vinfo/us/security/news/threat-landscape/unveiling-ai-agent-vulnerabilities-part-iii-data-exfiltration

Copyright

Copyright © 2026 Astha Rajput, Hema Sahu, Akhand Upadhyay, Yuvraj Singh, Ms. Heena Kausar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.