AI startup Motif Technologies said on Jan. 20 it has decided to take an active part in an additional call for proposals for the “independent AI foundation model” programme, known as Dokpamo, which the Ministry of Science and ICT plans to announce soon.
To that end, it is holding discussions with additional companies beyond a consortium formed in July last year. Motif Technologies said it will work to prove the independence of Korean technology based on its experience developing both high-performance large language models and large multimodal models as foundation models.
According to the company, its large language model “Motif 12.7B”, unveiled in November last year, is the result of carrying out the entire process directly, from model building to data training. It did not use the existing transformer structure as is. Instead, it developed and applied its own Grouped Differential Attention, or GDA, technology. The company stressed this went beyond simply training a model from scratch, and involved newly designing the attention function and the model architecture itself.
Chi-gyu Hwang
delight@d-today.co.kr