As more people use ChatGPT for legal, medical and mental health consultations or to handle company secrets, a warning has emerged that legal and security protections may be insufficient when entering sensitive information.

IT outlet Engadget said on Tuesday that over the past 18 months there have been cases in which ChatGPT conversations were sought for submission in lawsuits, exposed externally through sharing functions, and indexed in Google search results, urging caution.

The first area flagged was legal matters. Conversations with ChatGPT are not covered by legal privilege like consultations with a lawyer. OpenAI CEO Sam Altman also raised the issue in a 2025 interview, saying a court could demand that OpenAI submit users' chat records. In courts, more than 2,000 cases have been recorded in which fabricated precedents and fake citations generated by ChatGPT were found, including cases involving lawyers.

Mental health consultations were also cited as an area requiring caution. OpenAI estimated last year that about 0.15 percent of weekly ChatGPT users have conversations that suggest potential suicide plans. That amounts to more than 1 million people a week. While cases are increasing in which young users use ChatGPT like a counsellor or life coach, the conversations themselves do not receive legal confidentiality protections like talks with a therapist. Researchers also pointed to the possibility that if a chatbot shows so-called "sycophantic responses" that overly align with a user's existing beliefs, it could reinforce mistaken judgments or delusional thinking.

Entering company secrets is also risky. Last year, Madhu Gottumukkala, who had been acting director of the U.S. Cybersecurity and Infrastructure Security Agency (CISA), triggered a security alert and a Department of Homeland Security review after inputting a contract document marked "for official use only" into the public version of ChatGPT. Samsung Electronics also restricted the use of generative AI tools on company devices after it became known in 2023 that engineers had entered internal source code into ChatGPT. The company judged that information sent to external servers is hard to retrieve or delete and could be exposed to other users.

Medical information carries similar risks. Information provided to hospitals or medical staff may be covered by protections under relevant laws, but entering it into ChatGPT does not mean it receives the same protection. Engadget also introduced recent research suggesting that chatbot safeguards may be insufficient for certain conditions such as eating disorders, insomnia, substance abuse and bipolar disorder. In one case, a man in his 60s asked ChatGPT how to reduce chloride intake, then mistakenly understood sodium bromide as a salt substitute and ingested it.

Entering account information and personal data is an area to avoid in particular. Passwords and one-time authentication codes, passport numbers, resident registration-related information, card and bank account information, and photos of IDs could be misused for leaks or account takeovers if they remain in a conversation. Engadget also pointed out that ChatGPT has a function to remember information provided by users.

There have also been cases in which actual conversations were exposed externally. In July last year, it was confirmed that about 4,500 shared ChatGPT conversations appeared in Google search results. The cause was that users had set them to be searchable during the sharing process. After the reports, OpenAI discontinued the feature and removed the related conversations from search results.

It is also difficult to conclude that deleted chats immediately disappear in all situations. In 2025, there was also a case in which a court ordered OpenAI to preserve consumer chat records during a copyright lawsuit. The order later ended, but it showed that chat records could become subject to preservation depending on legal procedures.

In the end, unlike general uses such as research, drafting and explaining concepts, entering specific details of legal disputes, sensitive medical and mental health information, company secrets, and passwords and financial information into ChatGPT requires special caution. Altman has mentioned the need for a separate "AI privilege," but for now the key point is that ChatGPT conversations should not be assumed to receive the same confidentiality protections as conversations with lawyers, doctors and counsellors.