[DigitalToday reporter Chi-gyu Hwang] Cisco shared the results of an experiment using AI to write reports for security incident response training. It concluded that time is saved, but risks remain.

According to a recent report by The Register, Nate Forth (네이트 포르스), a senior incident commander on the Cisco Talos incident response team, said in a blog post that large language models (LLMs) show "serious inaccuracies, odd conclusions and inconsistent writing style" when generating long technical documents.

LLMs were found to produce errors in four main ways.

First, they use different data each time even for the same question, making repeatable results difficult. Second, they draw different conclusions from the same data. In a data breach, for example, one time they recommend resetting passwords companywide, and at other times they recommend a limited reset. Third, the document structure and format change each time it is run. Fourth, they omit data, which can leave out key information.

The Talos team also developed ways to reduce these problems. Cisco said hallucinations and content errors fall sharply when the LLM is given a single task instruction that covers only a specific small part of a report. It is also effective to specify the sources to use and set rules for style and format.

Using the technique, Cisco cut by 50% the time needed to draft an incident report based on security training. In the quality review process, peer reviewers, professional editors and managers all gave positive assessments without knowing the report was written by AI. Some also said there were far fewer typos and grammar errors than in an average report.

But when multiple reports were edited in a single session, an error occurred in which content from an earlier report was mixed into the next report. The Talos team recommended starting a new session for each report and re-entering the prompt.

Cisco also developed prompts for spelling and grammar checks, but often created grammar errors that did not exist and failed to catch real errors. Cisco said the success rate was below 50%, calling it "unsuitable for practical use."

Forth said the approach can be applied to other cybersecurity reports. But he stressed that "the author must be responsible for every sentence in the final report." He said that without manual review, duplicated, irrelevant or impractical recommendations could end up in the final report.