Major AI companies such as Google, Anthropic and SpaceXAI are rolling out dedicated services aimed at lawyers and legal teams. But fabricated case law and incorrect legal reasoning produced by AI continue to be found in actual court documents, making reliability checks a key task.

On Sept. 29, IT outlet Engadget reported that 3 of the 4 major AI labs this year have launched legal-focused products or unveiled related services. Google started a preview of Gemini Enterprise for Legal in August with Cleary Gottlieb, Freshfields, Weil and Williams & Connolly. Anthropic operates Claude Legal Solutions, and SpaceXAI has also set up a Grok-based legal services page.

The problem is hallucinations. The Law Society of Ontario tribunal ordered lawyer Shahryar Mazaheri (샤흐리야르 마자헤리) to pay C$31,150 (29.74 million won) in legal costs in connection with nonexistent case law and incorrect legal reasoning found in material drafted by Grok in June. Using AI itself was not the reason for sanctions, but failing to verify the output was a major aggravating factor. In Canada, cases in which false citations were confirmed rose to 86 in 2025 from 7 in 2024, and 39 more were added in the first quarter of this year.

Big law firms were not exempt. Sullivan & Cromwell apologised to the U.S. Bankruptcy Court for the Southern District of New York in April, saying it found AI-generated false and inaccurate case citations and legal errors in an emergency filing. The law firm acknowledged that its internal AI use policy and citation verification procedures were not properly followed.

Google and Anthropic are adopting an approach that does not rely solely on a model's built-in knowledge. They are linking to legal data and document management systems such as NetDocuments, Everlaw and CourtListener to reduce such issues. SpaceXAI's legal page, by contrast, introduces legal research and citation features but did not disclose a detailed structure for tracing citations back to primary sources.

Even dedicated legal AI does not eliminate hallucination risks. Stanford University's RegLab found error rates of about 17 to 33 percent when it tested products based on Lexis+ AI and Westlaw AI in 2024. There is still no independent research that has verified Google's and Anthropic's latest legal products in the same way.

In the end, even if AI assists legal work, the principle remains unchanged that lawyers must directly verify citations and legal reasoning submitted to court.