Main Catalog Informatics, Computer Engineering and Control System Analysis, Control, and Information Processing, Statistics

Analysis of the logical abilities of language models using the example of solving the Einstein problem

Authors: Tomskaya U.P., Zakharov М.А.
Published in issue: #4(99)/2025
DOI:
Category: Informatics, Computer Engineering and Control \| Chapter: System Analysis, Control, and Information Processing, Statistics
Keywords: language models, artificial intelligence, logical abilities, logical thinking, Einstein's problem, comparative analysis, efficiency
Published: 28.08.2025

The article examines various approaches to solving the Einstein problem used by language models and compares their effectiveness. The authors analyze the results of the models, identifying their strengths and weaknesses in the context of logical thinking. The article provides examples of solving the Einstein problem using language models from different families — claude, command-r, mistral, llama-2, llama-3, gpt2, gpt-3.5. For each model, an analysis of logical abilities is performed in solving this problem. In the end, the results of solving the problem by different models are summarized and conclusions and observations are made about how each of the models is able to reason.

References

[1] Losev A.F. Introduction to the general theory of language models. The history of linguophilosophical thought. Moscow, URSS Publ., 2004, 296 p. (In Russ.).

[2] Sitzhalilov A.L., Abduramanov Z.S. Language models for generative artificial intelligence applications. Information and Computer Technologies in Economics, Education and the social sphere, 2024, No. 1 (43), pp. 68-74. (In Russ.). EDN: IAHNLA.

[3] Lebedev A.S., Marishchuk B.V. Text generation technologies based on machine learning. Engineering Technologies, 2023, No. 2 (2), pp. 30-34. (In Russ.). EDN: SJAPYH.

[4] Lutsenko E.V. Knowledge engineering and intelligent systems. Krasnodar, Eidos Publ., 2020, 642 p. (In Russ.). https://doi.org/10.13140/RG .2.2.28085.91364

[5] Rogozin O.V. Solving the Einstein problem based on a frame model of knowledge representation. School Technologies, 2010, No. 3, pp. 151-156. (In Russ.).

[6] Nechaev N.P., Rogozin O.V. Implementation of a decision support mechanism using a frame model using the example of the Einstein problem. New Information Technologies in Automated Systems, 2007, No. 10, pp. 87-97. (In Russ.).

[7] Mammadova L.E., Ivanova L.N., Altaev E.S. The main aspects of artificial intelligence technology. Izvestiya vuzov EfiUP, 2023, No. 3 (57). (In Russ.). https://doi.org/10.6060/ivecofin.2023573.656

[8] Gorev S.V. Research of artificial intelligence methods and algorithms in determining the value of works of art. Izvestiya VUZov EfiUP, 2022, No. 4 (54). (In Russ.). https://doi.org/10.6060/ivecofin.2022544.622

[9] Byashimov P., Kakabayeva M., Dovletgeldyev Y., Oraev M. Construction and artificial intelligence: advantages and disadvantages. Symbol of Science, 2024, No. 4-1-2, pp. 32-34.

[10] Mamieva O.G., Baijanova G.N. Innovative approaches to ensuring information security in the era of digitalization. Symbol of Science, 2024, No. 4-1-2, pp. 55-57. (In Russ.).