Industry

Nvidia urges benchmarking that weighs accuracy as AI performance metrics shift

Chipmakers including Nvidia are proposing new ways to measure AI performance as AI agents spread. They say theoretical compute speed or a single benchmark score does not capture how much work an agent completes through multiple steps or how accurate the outcome is. Nvidia’s Jeff Yen said AI must be evaluated for speed and accuracy, and that image, video and agent outputs often require human review. Other firms including Qualcomm and Arm have also introduced new methodologies.

AI & Enterprise

Meta unveils lightweight agent model Muse Ember at 10 cents per 1 million input tokens

Meta Superintelligence Lab has unveiled a lightweight AI model called Muse Ember, TechCrunch reported. The company describes Ember as the lightest and most cost-efficient among its models. It is designed for complex reasoning and repeated agent tasks such as tool calling, coding subtasks, routing, information extraction and long-document analysis. Meta priced it at 10 cents per 1 million input tokens and said it can share workloads with its larger Muse Wonder model.