As the government steps up its 'AI for All' project to provide AI services to the entire population, concerns are emerging that standards for verifying performance are unclear. Critics say an objective verification mechanism is needed because the structure focuses on evaluation indicators and user numbers presented by the operators themselves.

The Ministry of Science and ICT and the National IT Industry Promotion Agency (NIPA) held a briefing on the project on Monday at Front1 in Seoul's Mapo district.

The core of AI for All is to provide the public with a general-purpose chatbot and public AI agents using domestically developed AI models. The government plans to reduce dependence on foreign AI services and narrow gaps in access to AI so that anyone can benefit from it.

The ministry will close the call for proposals by August at the latest and select a company or consortium to join the project by September at the latest. It will allocate 512 B200 graphics processing units (GPUs) that the government has secured this year to the selected company.

Companies joining the project must use domestically developed AI models. From the initial stage, the share of domestic models meeting the standard for an independent foundation model pretrained in-house must be at least 50 percent. At least 30 percent must come from domestic models developed by other companies. For advanced reasoning or multimodal functions, the use of foreign models is allowed on a limited basis.

The selected company must start a beta service by late September and launch an AI chatbot service for the entire population in December. The general-purpose AI chatbot is in principle to be provided for free without usage limits. From 2027, the plan is to upgrade the service into AI agents and provide 'one AI agent per person' nationwide.

But some say there is no clear standard for objectively judging service performance.

According to the project guide, companies participating in the call must propose annual goals and performance indicators in specific and quantitative terms. For adoption and diffusion performance indicators, the structure is that the operator presents metrics so they are 'verifiable on their own'. Companies also set targets and minimum levels for user numbers and token use, and submit the required GPU scale and the basis for their calculations.

The ministry will first conduct a written evaluation of AI model competitiveness, GPU utilisation plans and service development plans based on the proposals submitted by companies. It will then hold a presentation evaluation to determine final rankings.

In the presentation evaluation, service convenience and strategies to secure and retain users carry the biggest weight at 50 points. Service quality and safety are set at 30 points, while the 'service quality and performance' item that assesses response accuracy, usefulness, consistency, contextual understanding and performance verification plans accounts for only 10 points. The guide does not separately specify a common benchmark for comparing candidate services or a procedure for disclosing performance after selection.

At the briefing, the ministry did not lay out a specific plan on whether it would disclose benchmarks going forward. It explained that AI services are an area where users directly experience performance, and that competitiveness would be decided by user numbers.

The schedule is also tight. The ministry plans to start the beta service at the same time as signing the project agreement in September and launch the full service in December. The time from selection to full rollout is just over three months. Because companies can design the beta targets and operating method on their own, concerns are emerging about whether an effective beta will be possible.

A ministry official said, "We want to provide an AI for All service with performance comparable to free foreign versions." The official added, "If user numbers are markedly low, we naturally need to select a new company through annual evaluation. This is something we need to keep thinking about."