As developers rely more on artificial intelligence coding tools, even productivity tests carried out without AI have become difficult, it has been found.

On May 29 local time, IT outlet TechCrunch reported that AI research group METR said some developers had become so dependent on AI that they refused to take part in an experiment.

At the core is a gap between perceived productivity and actual performance. In a 2025 study of open-source developers, METR compared the time it took when people worked directly with the time it took when they used AI. Participants felt AI raised productivity, but the measured results were the opposite. Code generation was faster, but overall work slowed because additional time was needed to find and fix errors and to wait for AI outputs.

METR planned to run the same experiment again in February 2026 to check improvements in AI performance and changes in developer proficiency, but it changed its plan. Developers were reluctant to participate because they did not want to work without AI. METR ultimately shifted in May to a survey of technology workers and confirmed that respondents perceived they had become twice as valuable to their organisations thanks to AI.

But cases that question such self-assessments have followed at companies. "Tokenmaxxing," which spread this year, refers to a trend of using AI usage, especially token consumption, as a proxy indicator of productivity. Amazon stopped operating an internal token tracking leaderboard called "Kirorank" after employees ran AI agents excessively and drove up costs.

The Information has also reported that Uber exhausted its 2026 AI budget in four months, and Andrew Macdonald (앤드루 맥도널드), Uber's chief operating officer, said those costs did not lead to project or productivity results.

Some have also pointed to downstream costs rather than speed. Programmer and writer James Shore said on Hacker News that if AI can write code twice as fast, it should be checked whether maintenance costs have also been cut in half. Otherwise, he said, it amounts to trading a temporary speed-up for permanent dependence. It is a warning that humans will ultimately have to keep carrying code produced quickly by AI.

Related indicators have also emerged. EntelligenceAI founder and CEO Aishwarya Sankar (아이슈와리야 산카르) claimed that 44 percent of tokens used by companies go to fixing bugs created by AI. Code review tool company CodeRabbit said its analysis of open-source pull requests found AI code created 1.7 times more issues than code written by humans. It noted, however, that the figures were presented by companies that sell related tools, a limitation.

Independent research pointed in a similar direction. Researchers at Singapore Management University warned in an April report that "AI-generated code can lead to long-term maintenance costs in future real projects." It means AI can raise development speed in the short term but does not guarantee what comes after.

As alternatives, using more AI and redefining human roles are being discussed together. Scott Wu (스콧 우), CEO of Cognition, which made the AI coding agent "Devin," said AI agents can work independently but assessed their current capability as between junior and mid-level developers depending on the task. That means it is not yet at a stage where work can be fully delegated.

By contrast, the Singapore Management University researchers presented more conservative operating principles. They said developers should understand what AI does well and poorly as precisely as the programming languages they use, and should have a strong quality assurance system tailored to AI. They also suggested AI outputs should be reviewed as carefully as a junior developer's code, and that the big picture, such as software architecture and security design, should still be handled by people.

AI coding tools have already become basic tools in development workplaces, but it is also becoming clear that it is hard to judge the effect of adoption solely by productivity gains. The point to watch ahead is not code generation speed, but how the code can be maintained and verified over the long term at lower cost.