Chip companies are releasing their own performance evaluation standards for the artificial intelligence (AI) agent era. They say theoretical compute speed or a single benchmark score makes it hard to gauge the amount of work an AI agent actually completes through multiple stages, making product performance difficult to assess properly. Companies are also disclosing measurement conditions and methods alongside results, leaving room for verification.

The backdrop to the new standards is the limits of existing performance indicators, industry officials say. TOPS (Tera of Operations Per Second), commonly used to describe AI chip performance, is a theoretical figure calculated at peak clock speed for the number of operations a neural processing unit (NPU) can handle per second. Chip companies believe that even if TOPS or theoretical throughput rises, it does not translate into real AI performance without sufficient memory bandwidth and software optimisation.

For example, AI benchmark results for the Snapdragon 8 Elite Extreme 6th generation, measured in a benchmarking session at Snapdragon Summit 2026, showed different gaps versus Qualcomm expectations depending on the measurement tool. A Qualcomm official said, "AI performance varies depending on end-to-end optimisation, software and the types of models being run, so benchmarking alone cannot fully verify platform improvements."

Qualcomm at the time introduced Evolve, a GPU and AI benchmark from Dutch firm Traverse Research, alongside existing benchmarks and carried out measurements. Evolve is designed to allow repeated measurements with customised task conditions, showing item-by-item performance differences rather than a single composite score.

Arm on Oct. 13 also disclosed a CPU performance evaluation methodology for agentic AI. Fujitsu, Meta and OpenAI supported the development of the methodology. Arm said AI agents behave like virtual users that call tools and services and evaluate results until a task is completed.

Arm's metric focuses on how much work can be completed while maintaining response speed. It says that if throughput drops as utilisation rises, the amount of work actually finished does not increase even if theoretical throughput grows. The methodology follows three principles: real agent use cases, repeatable measurement and verification of final task results.

Arm also released performance verification results based on the methodology. U.S. testing firm Signal65 compared an Arm AGI CPU with an AMD server chip. It said it ran the same benchmark script on both chips and calculated full-rack performance from measured values for a single server.

Because data centres operate within fixed power and space, the amount of work a single rack can handle is important, Arm said. Arm said the AGI CPU delivered an average 1.99 times the performance per rack versus the AMD EPYC 9755. The figure reflects calculations that more servers can be placed in a rack within the same power limit. The time an agent takes to open an isolated execution environment, or sandbox, was also up to 45 percent shorter than with the AMD EPYC 9965.

◆Disclosing conditions and allowing retesting to build trust... the question is real reproducibility

Agents are measured with CPUs because of how they work. A Signal65 report commissioned by Intel said AI agents execute instructions between moments when the model produces answers, edit files and compile code. The CPU also handles creating and managing sandboxes.

Under a standard method that runs tasks one by one, average server CPU utilisation was less than 10 percent. As the number of concurrent tasks increased, utilisation rose to about 54 percent. A score that measures a single task does not reveal the volume of agent work a server can handle.

Like Arm, chip companies are disclosing measurement conditions along with results to boost the credibility of performance figures. The Signal65 report included the fact that Arm and Intel each commissioned research, as well as the comparison conditions. Evolve, the Traverse Research GPU and AI benchmark used by Qualcomm, also provides separate scores by category so strengths in one area cannot mask weaknesses in another. Because the same program can be run on commercial devices, the figures companies present can be measured again. Traverse also disclosed a principle that no company is involved in ownership or operation and that performance cannot be bought with money. Traverse said Qualcomm and Samsung Electronics are currently running Evolve internally.

The new evaluation standards are still in the validation stage. Arm's rack-level figure is a modelling result calculated with an early system, and Signal65 said it plans to retest with a wider set of workloads and rival products using final mass-production hardware. With standards that aim to measure real workloads instead of theoretical figures and single scores, the key question is whether the same results can be reproduced in actual data centres and on commercial devices. The assessment is that only once reproducibility is confirmed can the standards released by chip companies be used as a common yardstick across the industry.