Deep metamulti-agent machine learning method based on the maximum principle

Authors: Smirnova K.D.
Published in issue: #4(57)/2021
DOI: 10.18698/2541-8009-2021-4-694
Category: Informatics, Computer Engineering and Control | Chapter: Automation, Control of Technological Processes, and Industrial Control
Keywords: machine learning, reinforcement learning, meta learning, high-level system, Q-learning, multi-agent learning, discount factor, learning rate
Published: 29.04.2021

The concept of machine learning is considered and the target task is formulated as the search for the most universal teaching methods. The article gives a description of the discrete medium used in the experiment and the agents acting in it. The Q-learning method on which the developed algorithm is based is described. The results are presented of an experiment with a conceptual machine learning algorithm based on the maximum principle, where the algorithm used data from pretrained structures within the framework of metalearning. The operation of the algorithm with different sets of parameters is analyzed. The regularities of the influence of the key training parameters on the result are investigated and the prospects for their use are considered.

References

[1] Sjardin B., Massaron L., Boschetti A. Large scale machine learning with Python. Packt Publ., 2016. (Russ. ed.: Krupnomasshtabnoe mashinnoe obuchenie vmeste s Python. Moscow, DMK-Press, 2018.)

[2] Stadie B.C., Yang G., Houthooft R., et al. Some considerations on learning to explore via meta-reinforcement learning. URL: https://arxiv.org/abs/1803.01118 (accessed: 15.02.2021).

[3] Coelho L.P., Richert W. Building machine learning systems with Python. Packt Publ., 2015. (Russ. ed.: Postroenie sistem mashinnogo obucheniya na yazyke Python. Moscow, DMK Press Publ., 2016.)

[4] Nilsson N.J. Introduction to machine learning. URL: https://ai.stanford.edu/~nilsson/MLBOOK.pdf (accessed: 15.02.2021).

[5] A StarCraft II bot API client library for Python 3. github.com: website. URL: https://github.com/Dentosal/python-sc2 (accessed: 15.02.2021).

[6] Getting started with Gym. gym.openai.com: website. URL: https://gym.openai.com/docs (accessed: 15.02.2021).

[7] Flach P. Machine learning: the art and science of algorithms that make sense of data. Cambridge University Press, 2012. (Russ. ed.: Mashinnoe obuchenie. Nauka i iskusstvo postroeniya algoritmov, kotorye izvlekayut znaniya iz dannych. Moscow, DMK Press Publ., 2015.)

[8] Müller A.C., Guido S. Introduction to machine learning with Python: a guide for data scientists. O

[9] Sutton R.S. Reinforcement learning. MIT Press, 1998. (Russ. ed.: Obuchenie s pod-krepleniem. Moscow, Binom Publ., 2012.)

[10] Schweighofer N. Doya K. Meta-learning in reinforcement learning. Neural Netw., 2003, vol. 16, no. 1, pp. 5–9. DOI: https://doi.org/10.1016/S0893-6080(02)00228-9