Gartner forecast that through 2029, most personal data breach incidents will stem not from direct leaks of personally identifiable information (PII) but from inferences AI generates about individuals.

Gartner said as companies reduce the amount of personal data they hold due to regulatory and cost burdens, threat actors have been able to carry out inference-based attacks using AI. Advances in generative AI and machine learning have made it possible to infer sensitive information such as an individual’s health status or behavioural patterns from anonymised and aggregated data.

Bart Willemsen (바트 빌렘센), a senior analyst at Gartner, said the pattern of privacy breaches is shifting from data leaks to insight leaks. He said companies have focused on protecting personal data, but AI can now reproduce deep personal insights without being constrained by existing data controls. Privacy risks are increasingly arising from what AI algorithms infer about individuals rather than from data exposure, he said.

He also said inference attacks are more dangerous because they often are not captured by existing detection systems. Even if actual records are not leaked, sensitive personal information can be revealed through conclusions derived by AI. This undermines data integrity and creates privacy breach risks that are difficult to detect, explain and mitigate, he added.

Gartner said companies are in a position where they must reconsider their privacy strategies, and security leaders must not only protect personal data but also manage how AI systems generate and use insights about individuals and how they act based on them.

It also forecast that as companies strengthen responses to risks such as AI profile errors, bias and unauthorised generation, spending to protect data integrity will reach the same level as investment to protect confidentiality by 2028.

Willemsen said companies that see privacy only as a data protection issue will become increasingly vulnerable to privacy breach incidents driven by AI-generated inferences. Future privacy risks will be more heavily influenced by how AI interprets data than by how companies store data, he said.