Banks are taking steps to cut sharply rising token costs as they expand the adoption of artificial intelligence (AI).

American Banker reported on June 29 that banks' cost burden is rising quickly as AI service billing shifts from subscriptions to token-based pricing.

AI has been presented as a tool to cut costs, but recently the cost of using AI itself has emerged as a new burden. Major providers such as Anthropic, OpenAI and Microsoft have shifted from models that allowed freer use for a set period to charging based on tokens, a unit of data processing. The more a company uses the services, the more rapidly its bills can grow.

In the field, more cases are emerging of companies publicly citing the cost burden. Royal Bank of Canada (RBC) CEO Dave McKay (데이브 맥케이) said in May the bank's token usage rose 500 percent from a year earlier. JPMorgan Chase chief data and analytics officer Zachary Anderson (재커리 앤더슨) has also said some employees spend more on tokens than their annual salary.

As PNC Financial Services Group CEO Bill Demchak (빌 뎀책) also voiced complaints about token costs, PNC sees the costs as potentially eroding productivity gains. Demchak said at a Morgan Stanley conference, "Even if AI increases bank productivity, the effect can disappear because of token costs." That is why banks have begun to reconsider which models to attach to which tasks, rather than simply expanding AI use.

Banks' first move has been to avoid using top-tier models for every task. As AI models have advanced to the stage of making their own judgments and performing work, token consumption has also increased, but many internal bank tasks do not require that level of sophistication. Neurometric CEO Rob May (롭 메이) said cutting-edge models can be excessive for simple tasks. Using a model that knows only as much as needed to handle a task is better in terms of cost, he said.

In line with the trend, some banks are using older models or small language models (SLM) that cost less even if performance is somewhat lower. Ned Carroll (네드 캐럴), head of data and automation at NC, said banks should use the right tool for the right problem, adding that a model capable of solving advanced math problems is not needed to understand policies or procedures related to returned checks.

Other ways to cut token use are also being reviewed. Using open-source models that can be used for free is a representative option. Saving and reusing existing AI answers when the same questions recur is also cited as a cost-cutting measure. May said if similar questions and answers are repeated, banks can save them and search a database first, calling the model only when needed to save token costs.

Expanding in-house infrastructure is also emerging as a key response. Banks are trying to reduce reliance on outside AI providers and secure their own computing resources. PNC said it plans to build its own graphics processing unit computing infrastructure to reduce dependence on external token use. Demchak said the bank will build internal capabilities to operate its own large language model.

Some tasks may be cheaper to handle with people than with AI, according to another view. In particular, tasks with high failure costs may be more rational to run with existing staff than through automation. As AI adoption does not necessarily lead to cost savings, banks are moving toward designing cost-management systems that include model selection, bringing infrastructure in-house and readjusting the scope of use by task.