[Digital Today reporter Chi-gyu Hwang] "Companies introducing AI agents should set quantitative goals before automation."

Lee Hee-jin (이희진), a part leader at Samsung Electronics' MX Business Division, stressed the point while sharing a case in which Samsung applied AI agents to marketing for Galaxy Store, the app store it operates, at an "Industry Summit" recently held by Korea Microsoft at the COEX Grand Ballroom in Seoul.

Lee said the absence of quantitative goals is one factor that can prevent AI projects from delivering results.

"A goal like 'do as well as a person' is hard to use as a benchmark," he said. "People differ widely in ability, so it is hard to standardise." He said Samsung set a goal of increasing click-through rates by 15 percent over the previous level, but achieved a 105 percent rise, far exceeding the target.

Games are the core revenue base for Samsung's Galaxy Store. When users buy in-game items or paid products in Galaxy Store, Samsung receives a fee. That makes it essential to optimise each step from game exposure to clicks, installs and payments.

The problem was that the number of games needing promotion kept rising, while operating staff and data analysis capabilities had limits. Samsung therefore pushed to introduce 6 AI agents covering tasks from data analysis to content creation. It began preparations in December last year and entered a stage in April this year where it could confirm results. As it used AI, Samsung staff focused on showing games and driving clicks, and that led to results beyond expectations, Lee said.

AI-driven analysis of customer groups also had a major impact on the better-than-expected outcome. "We passed customer group information to a large language model (LLM), had it summarise characteristics by segment and create tailored messages, and it produced wording that users accepted more familiarly," he said.

Lee said that besides a lack of quantitative goals, factors behind companies failing to see effects from AX include LLM limitations, data and organisational culture.

Because LLMs are trained on general-purpose data, they fall short of outputs produced by specialists in each field. "In tasks like data analysis, AI is at about 85 percent of an expert," Lee said. "In coding or content creation as well, it can fall far short of expert level. The final 15 to 20 percent is a big wall on the path to commercialisation."

On data, he said it is important to refine it for AI. "Once some automation is achieved, you realise the data assets accumulated so far are insufficient," he said. "Samsung has built up data for more than 8 years operating Galaxy Store. Last year we moved to a new data platform and organised the data. We also applied a catalogue so agents can understand the meaning of the data. Different term definitions across departments were linked through an ontology," he said.

Organisational culture is the most important keyword in AX, Lee said. "It takes a long time to change decision-making processes that rely on intuition and experience," he said. "You need clear goals and objective results based on data to overcome the limits of intuition-based decision-making." He again stressed that for a company to be ready for AI, agents must be able to access all data and understand its meaning, and that key decisions must be made through machine learning and data analysis and verified through testing.

Samsung plans to actively use AI to increase revenue based on its results in lifting Galaxy Store click-through rates. Lee said it plans to additionally develop an install prediction model and a purchase conversion model.