Research backed by OpenAI is under way to apply generative AI to existing rule-based statistical forecasting. The core aim is to test whether complex consumer responses can be predicted by replacing behavior rules set manually by researchers with OpenAI’s GPT-based simulations.

Kwon Hyuk-gu (권혁구), chair professor of information systems at Nanyang Technological University (NTU) in Singapore, introduced policy simulation research based on large language models at a media session titled "AI use in policy research" held at OpenAI Korea in Gangnam, Seoul, on Nov. 11.

Kwon said the study predicts expected responses to policies by reflecting different living conditions and spending patterns across consumers, and then checks how closely those forecasts match actual behavior.

Rule-based agent-based modeling used in traditional statistical forecasting requires researchers to directly set behavior rules based on factors such as consumer income, age and living environment. But as the number of variables increases, the rules become more complex and the possibility of researchers’ subjectivity being involved also grows.

The researchers are running GPT-based simulations using financial transaction data from more than 1 million users of a South Korean household ledger app. They organise consumers’ income and spending patterns, enter them into GPT and predict changes in spending resulting from the 2025 distribution of livelihood recovery consumption coupons. They plan to verify accuracy by comparing forecasts with actual transaction records and the Bank of Korea’s analysis of policy effects.

Early research showed the forecasting error fell significantly. Kwon said the error was about 25 percent when the work began, raising doubts about feasibility. He said it has now fallen to about 10 percent and the team continues to refine the model by repeating various combinations.

He said those are provisional figures produced in the process of improving the GPT-based statistical forecasting model.

New challenges have also emerged from the use of AI. Kwon said the burden of setting rules has eased, but the source of bias has shifted from researchers to GPT. He said there is no model without bias.

That means that while the burden of directly setting behavior rules is reduced, it becomes important what data are provided to GPT, how they are processed, and how much the forecasts can be trusted, he said.

Such AI simulations are being used not only in policy research but also in areas such as election forecasting and market research. Kwon said attempts are also active in South Korea to replace existing survey panels with AI simulations.

Kwon said the role of AI simulation is to provide grounds for rational decision-making. He said that even if the same budget is forecast to have different consumption-boosting effects when paid to all citizens versus when concentrated on low-income groups, reflecting it in actual policy decisions requires separate value judgments.

Kwon said he does not think GPT or AI can make decisions based on those grounds. He said the aim is to convey expected policy effects at an accurate level and help constructive discussions proceed based on figures.

The research is part of OpenAI’s research support programme, "New Policy Ideas for the Intelligence Age". OpenAI supports 14 independent research projects worldwide with up to $2 million worth of research funding and API credits. The research teams are also given access to the latest frontier AI models.

OpenAI cited the company’s mission of ensuring that all humanity benefits from AI as the backdrop to the support. Eum Seong-won (음성원), head of communications at OpenAI Korea, said individuals can benefit by using AI models, but if AI is used in policy research so that policies work more effectively, everyone benefits.

He said selection of support recipients also considered factors such as social spillover effects and potential contributions to the public interest. In South Korea, a project titled "Evaluating democratic accountability and governance policy using AI" by a research team led by Hong Soon-man (홍순만), a professor in the Department of Public Administration at Yonsei University, was selected. The research analyses and evaluates the National Assembly’s checks and oversight of the executive branch using AI.

OpenAI drew a line on the possibility that the results of the policy research would be used for AI model development.

Separately, Kwon’s research team will verify the accuracy and limitations of the GPT-based statistical forecasting model by the end of this year. It plans follow-up research through March next year to compare effects by changing policy conditions such as eligibility, amount and usage period.