An analysis of data from tens of thousands of Kubernetes clusters showed average utilisation of GPUs was 5 percent, CPUs 8 percent and memory 20 percent, TechZine reported on April 21, citing a Cast AI survey.

Kubernetes is an open-source platform that supports the automatic deployment and management of application containers spread across multiple servers. Critics say that as adoption of Kubernetes rises, the gap is widening between cloud costs paid by companies and actual usage.

TechZine said the results were paradoxical given Kubernetes is meant to help run large-scale infrastructure efficiently. It also said average GPU utilisation was limited to 5 percent even for AI and machine-learning workloads. Idle GPUs cost a few dollars per hour, while idle CPUs cost only a few cents, making the economic loss from GPU waste larger.

Cast AI said one-off resource resizing is a root problem. Even if settings are adjusted at the time of deployment, workloads keep changing and traffic patterns also shift. Settings that were appropriate 6 months ago may no longer fit now. The same applies to choosing spot instances, configuring autoscalers and managing node lifecycles.

Cast AI said autonomous and continuous optimisation is needed to respond to a situation in which infrastructure costs are moving in the wrong direction. Cast AI provides a platform that automatically optimises cloud costs in Kubernetes environments.