As the "policy as code" trend of writing corporate security and compliance rules in code spreads, cases of using AI to generate policy code are also increasing. Critics say AI-created policy code can look fine but still allow access that should not be permitted.
Apple senior security engineer and independent researcher Vatsal Gupta (바찰 굽타) said in an interview with SecurityWeek, "Policies generated by LLMs are often syntactically correct but semantically wrong." He added, "If a single condition is missing or an attribute is misinterpreted, who can access what changes completely."
Gupta identified five broad types of errors. The first is omission of contextual conditions. If a policy that should restrict access by region, department or ownership is missing such conditions, it may appear valid but apply across the full scope.
The second is omission of deny logic. Access-control policies are generally built on a default-deny principle with exceptions layered on top, but AI can capture only exceptions while leaving out the baseline restrictions.
The third is hallucination. AI can include attributes that do not exist in the actual system. The code compiles but behaves in unexpected ways at runtime.
The fourth is simplification of time and situational conditions. Access that requires a specific time window or an approval process is changed into always-on permission with no conditions. The fifth is misclassification of actions. A policy intended to restrict sensitive operations such as deletion may end up restricting a broader set of operations, or restricting an entirely different operation.
Gupta said such errors do not break builds or trigger alarms. "Quietly, gradually, they widen access," he said. "The bigger problem is that policies are not created once and then finished. As creation, modification and deployment repeat, small errors accumulate," he added. He warned, "If the generation process is not trustworthy, the risk spreads to the entire system. If small flaws spread across thousands of policies, each may look trivial but together they create an attack surface that is hard to grasp."
He suggested changing the trust model rather than abandoning AI itself. He said generated policies should not be treated as correct by default and that a validation step should sit between generation and application. "Automation itself must not become the goal. Accuracy, auditability and trust must be the goal," Gupta said. "In access permissions, 'almost right' is not enough."