Scientists from HSE University in Moscow and the University of Lausanne published[1] in the Journal of Economic Behavior & Organization the results of a study analyzing how artificial intelligence models handle human behavior in the context of strategic decision-making. The study involved popular AI models, including ChatGPT-4 and Claude-Sonnet-4, which showed a tendency to overestimate the rationality of human thinking.
Research Methodology and AI Models Used
The research focused on the classic economic game “Guess the Number,” where participants select a number between 0 and 100, and the winner is the person whose chosen number is closest to half the average of all selected numbers. The study’s authors, Dmitry Dagaev, Sofia Paklina, Petr Parshakov from HSE University, and Iuliia Alekseenko from the University of Lausanne, tested five leading AI models[3] in 16 different scenarios against virtual opponents with varying knowledge levels, ranging from first-year economics students to attendees of game theory conferences.
Differences Between AI and Human Approaches to the Game
Results showed that AI models consistently chose lower numbers than humans[2], reflecting a more rational approach aligned with game theory equilibrium. While humans in classical experiments averaged numbers around 27, AI models opted for even lower values, assuming opponents would also think strategically. In matchups against game theory experts, AI chose numbers close to zero, whereas against students, it selected higher numbers, demonstrating the adaptation of strategy to the opponent’s level.
This systematic difference points to a fundamental limitation in how artificial intelligence interprets human behavior. Although AI models exhibited strategic thinking and adjusted their decisions according to opponent characteristics, they failed to recognize dominant strategies in simplified two-player versions of the game. Dmitry Dagaev emphasized that while AI responds to changes in game structure similarly to humans, its approach relies on step-by-step reasoning about possible actions of other players rather than an intuitive understanding of human behavior.
Implications of Findings for AI Applications in Business and Politics
The significance of these findings is particularly strong in the context of the growing role of AI systems in business operations and decision-making processes. Dmitry Dagaev noted that currently AI models are beginning to replace humans in many tasks, which increases economic efficiency. However, in tasks requiring decision-making, it is crucial for language models (LLMs) to behave in ways consistent with human thinking to avoid errors arising from excessive assumptions of rationality.
The studied game “Guess the Number” is an example of a so-called Keynesian beauty contest, a concept developed by British economist John Maynard Keynes in the 1930s. The game examines how participants predict the decisions of other players rather than merely making their own optimal choice. It has long been used to explain fluctuations in financial markets, where success depends on forecasting other investors’ behavior.
The study, which received support under the HSE University Basic Research Program, highlights that understanding areas where AI aligns with human behavior and where it diverges will be key for the future deployment of these systems in markets, politics, and daily life.