Google DeepMind has unveiled a new benchmark to evaluate AI agents' ability to carry out real professional work. It measures capabilities beyond answering simple questions, including analysing complex blueprints and medical images and completing deliverables such as reports and spreadsheets.
On Nov. 14, Taiwan media outlet IT Home reported that Google DeepMind jointly developed the Multimodal Knowledge Agent (MKA) benchmark with AI analytics firm Artificial Analysis.
MKA is designed to feed AI systems complex visual information such as architectural drawings, electronic circuit schematics and clinical images, and have them produce real work outputs based on those materials.
There are 200 evaluation tasks, covering 83 job roles across 10 fields including business, construction, engineering, finance, healthcare and science. Each task replicates work that skilled professionals carry out over 3 to 8 hours.
The evaluation does not stop at whether an answer is correct. Each task applies an average of about 70 detailed criteria, weighting 75 percent for factual accuracy and 25 percent for visual completeness.
To ensure a fair comparison, all models were given the same tools and working environment, with up to 250 work opportunities. AI evaluators open the final files and verify the content, formulas and visual composition. Agreement between expert and AI evaluations was 96 percent.
In the results, Anthropic's Claude Fable 5.2 ranked first with a 57 percent weighted attainment rate on high-difficulty items. Google's Gemini 4 Carbon placed second with 54 percent.
Costs differed widely. Claude Fable 5.2 averaged $31.97 per task, about 42,700 won, while Gemini 4 Carbon averaged $6.29, about 8,400 won, about one fifth as much.
Work efficiency also varied by model. The average number of work iterations needed to complete a task ranged widely from 30 to 130.
Google explained that in professional work, AI agents' competitiveness is expanding beyond simple accuracy rates to include output completeness and cost efficiency. It said the assessment was conducted in a controlled environment, so results may differ from workplace performance in real companies.