AI advice made people 3x less accurate but 2x confident, researchers found
Source Entity
Hacker News
Research indicates that AI advice significantly reduces human accuracy while drastically increasing overconfidence. Users were far less likely to admit ignorance, trusting incorrect AI answers even in areas where models typically fail.
The Paradox of AI Assistance: Accuracy vs. Confidence
Recent research conducted by a collaborative team from three prestigious European institutions—the University of Milano-Bicocca, École Normale Supérieure, and Sapienza University of Rome—has revealed a disturbing trend in human-AI interaction. The study, authored by Valerio Capraro, Chiara Marcoccia, and Walter Quattrociocchi, suggests that while AI is often marketed as a tool for enhancement, it can actually degrade human judgment. The core of the finding lies in a psychological paradox: as people rely more on AI advice, their objective performance plummets, yet their subjective certainty skyrockets.
The Collapse of Intellectual Humility
One of the most striking metrics from the study is the total collapse of "judgment suspension." In a natural state, participants were willing to admit they did not know the answer to a question 44% of the time. However, once provided with AI-generated advice, this figure crashed to a mere 3%. This suggests that the presence of an AI suggestion effectively eliminates the human tendency to pause and recognize the limits of one's own knowledge. By filling the void of uncertainty with a definitive (though often incorrect) answer, AI suppresses the critical instinct to say "I don't know," leading users to commit to answers they would have otherwise questioned.
The Accuracy Gap and the Danger of Misinformation
The impact on actual performance was equally severe. The researchers found that accuracy dropped from 27% to 9%, meaning users became three times less accurate after consulting the AI. This indicates that AI advice did not simply fail to improve performance; it actively steered users toward incorrect conclusions. This phenomenon is particularly dangerous because it demonstrates a high level of trust in "hallucinations" or errors produced by the AI. When users outsource their cognitive processing to a model, they lose the internal verification mechanisms that typically protect them from obvious errors.
The Surge of Unwarranted Confidence
While accuracy was falling, confidence was surging. The study observed that confidence levels rose from 30% to 76%. This creates a precarious psychological state where individuals are not only wrong but are deeply convinced of their correctness. Associate professor Valerio Capraro noted that people became significantly worse in their output while becoming twice as confident. This divergence between performance and perception is a hallmark of the Dunning-Kruger effect, amplified here by the authoritative tone and perceived objectivity of artificial intelligence.
Targeted Failures: The Visual Detail Blind Spot
To ensure the robustness of their findings, the researchers deliberately utilized questions focusing on visual details—a known area where current AI models frequently struggle. The fact that accuracy dropped so sharply in this specific domain proves that humans are often unaware of the inherent weaknesses of the tools they use. Instead of treating AI as a fallible assistant that requires oversight, participants treated it as an oracle, trusting it even in specialized areas where the technology is fundamentally prone to error.
Broader Implications for the AI Era
These findings point toward a broader societal risk known as "automation bias," where humans over-rely on automated systems even when they contradict common sense or factual evidence. As AI becomes integrated into professional fields such as medicine, law, and engineering, the risk of this "confidence-accuracy gap" increases. If professionals stop suspending judgment and start trusting AI-generated visual or data analyses blindly, the potential for catastrophic systemic errors grows. The study highlights a desperate need for AI literacy that emphasizes skepticism and the importance of human-in-the-loop verification.
Conclusion
In summary, the research by Capraro and his colleagues serves as a critical warning. The transition from a 44% to a 3% rate of judgment suspension, coupled with a crash in accuracy and a spike in confidence, reveals that AI can act as a catalyst for overconfidence and error. To mitigate these risks, the future of AI integration must focus not just on improving the accuracy of the models, but on educating the users to maintain their intellectual humility and critical distance from the machine's output.