A study found that students who studied with the help of artificial intelligence had lower accuracy and response rates in the actual test stage.

On April 8 local time, online media outlet GigaZine reported that researchers said in a paper titled "AI Assistance Reduces Persistence" that AI can improve learning performance but may weaken persistence in sticking with problems when the support disappears.

The researchers first ran an experiment on fraction calculation problems with 354 people. Participants were randomly split into an AI group and a non-AI group. Of 15 questions, the first 12 were practice problems that became progressively harder, and the final 3 were a test. Participants could skip questions at any time, and the AI group had AI access cut without warning when it entered the test stage.

The results were clear. The AI group performed better in practice, but when it moved to the test its accuracy fell sharply and its skipping rate rose. The non-AI group showed little difference in performance between the test and practice stages. The researchers summed this up by saying, "AI use improves performance during the learning stage but worsens test scores."

The researchers also conducted a second experiment to confirm the results. This time 667 people participated, and the first 3 questions of 17 total were placed as pre-questions to reduce the effect of differences in original ability. The non-AI group was also shown the same type of sidebar in the place where AI had been, eliminating differences in screen layout.

The same results were repeated in that experiment as well. Performance was good while using AI, but efficiency fell in the test.

How AI was used also made a difference. The researchers found that 61 percent of the AI group asked the AI directly for the correct answers. That subgroup recorded the biggest drop in test scores. By contrast, test scores did not worsen significantly when participants asked only for hints or explanations even if they used AI. That means results differed depending on whether AI was used as an answer-providing tool or as an auxiliary learning tool.

The researchers also checked whether the phenomenon was limited to mathematics. For this, they conducted a third experiment using SAT-style reading comprehension questions, with 201 people participating. The first question was a pre-question, questions 2 to 6 were practice, and questions 7 to 9 were test questions. Here too, the AI group’s scores plunged when it entered the test stage, and its skipping rate rose.

The researchers’ conclusion across the three experiments was consistent. The researchers concluded that "with AI support, people give up on problems or their performance worsens when the AI disappears," and ultimately summed it up by saying, "AI support lowers persistence."

The results have direct implications for how AI is introduced as a learning aid. Using it to get answers immediately can harm real-world performance, but the downsides may be relatively smaller when it is used mainly for hints and explanations. That suggests designs for AI learning tools may increasingly need to prioritise guidance toward solutions and explanation-based support over presenting correct answers.