Motif Technologies has filed an objection to the results of the second-stage evaluation of the government’s Dokpamo project for an independent AI foundation model and requested disclosure of detailed scores and criteria by assessment item, as well as a review.

Motif Technologies issued a statement on Wednesday and said it formally submitted an objection to the Ministry of Science and ICT over the results of the second-stage selection under Dokpamo.

Motif Technologies said its Motif 3 model scored 47 on the Artificial Analysis Intelligence Index (AAII), a comprehensive performance indicator for global AI models, the highest among models participating in Dokpamo. It said Motif 3 was eliminated with the lowest score in the second evaluation announced on Aug. 18. The company said AAII scores were 37 for Upstage, 35 for SK Telecom and 31 for LG AI Research.

Motif Technologies said it "respected and accepted the evaluation results" but argued there was a need to disclose evaluation criteria and results in detail after the ministry explained that Motif 3 fell short in usability and applicability in expert and user evaluations.

It said the value of a foundation model lies in baseline performance that can be extended across multiple fields rather than chatbot usability. It also said putting weight on usability on the grounds that the performance of domestic models is now sufficient does not align with the purpose of the project, given that Dokpamo’s original goal was to secure performance of at least 95 percent compared with global frontier models.

Motif said it understood the evaluation consisted of 100 points in total, comprising 40 points for benchmarks, 35 for expert evaluation and 25 for user evaluation. It said the benchmark evaluation consisted of 25 points for AAII and 15 points for an assessment by the National Information Society Agency (NIA).

The company said performance gaps between models were excessively compressed when converting benchmark scores into the actual evaluation weighting. It said Motif 3’s 47 points versus LG AI Research’s 31 points is a 16-point gap in AAII, but the gap shrinks to 4 points when converted into a 25-point benchmark score.

Motif Technologies asked the ministry to disclose: detailed scores and evaluation criteria for each overall assessment item and for each company; the method and basis for calculating benchmark score weightings; detailed criteria and results for expert evaluation; and the methodology, detailed criteria and results for user evaluation.

On the user evaluation, it said it had requested a blind assessment because brand awareness differences between large companies and startups could affect the evaluation, but the evaluation method and conditions were not sufficiently disclosed. It said if performance differences shown in benchmarks were reversed through user evaluation, specific evidence was needed to verify the cause.

Motif Technologies said the objection was not intended to demand its re-selection. It said it would not participate in Dokpamo’s third-stage evaluation regardless of the outcome of its objection.

Motif Technologies said, "We hope this objection will become a process of building consensus on what Dokpamo is for and finding directions for improvement that all competitors can agree on." It added, "Going forward, we will prove that we are a team that can build Korea’s best frontier model through our own strength and prevail in global competition."