MIPT Center for Cognitive Modeling

In April 2021, the program committee of the leading conference on neural network technologies in artificial intelligence NeurIPS 2021 approved the application of an international group of researchers, which includes employees of the Center for Cognitive Modeling at MIPT, Microsoft, Facebook, to hold an IGLU competition:Interactive Grounded Language Understanding. Participants will be asked to develop two types of agents: a builder and an architect. The builder must be able to follow the architect's language instructions for building various block configurations in the Minecraft environment. The architect, in turn, must learn how to issue instructions effectively. The first round will start in July 2021. As part of the Summer School of the Russian Association of Artificial Intelligence, which will be held in Sirius (Sochi), tutorials for participants of the competition and a separate track for Russian participants will be organized. Among the prizes will be access to computing clusters, commemorative prizes and the opportunity to present your results at a special workshop within the framework of NeurIPS 2021. The basic rules and dates are listed on the competition website - https://iglu-contest.net .

Currently, deep reinforcement learning (RL) shows impressive results in complex gaming and robotic environments. Often these results are achieved at the expense of huge computational costs and require an incredible number of episodes of interaction between the agent and the environment. There are two main approaches to improving the sample efficiency of reinforcement learning methods - using hierarchical methods and expert demonstrations. In this paper, we propose a combination of these approaches that allow the agent to use low-quality demonstrations in complex vision-based environments with multiple related goals. Our forgetful experience replay (ForgER) algorithm effectively handles errors in expert data and reduces quality losses when adapting the action space and states representation to the agent's capabilities. Our proposed goal-oriented structuring of replay buffer allows the agent to automatically highlight sub-goals for solving complex hierarchical tasks in demonstrations. Our method is universal and can be integrated into various off-policy methods. It surpasses all known existing state-of-the-art RL methods using expert demonstrations on various model environments. The solution based on our algorithm beats all the solutions for the famous MineRL competition and allows the agent to mine a diamond in the Minecraft environment.

New equipment from Canada has arrived at the Cognitive Modeling Center - a mobile robot with a Husky manipulator. This platform has been manufactured by Clearpath for several years and is one of the most reliable and popular experimental robots for conducting advanced research in the field of robotics and artificial intelligence. The laboratories of our Center plan to use Husky to test and demonstrate the work of our new algorithms in the field of mapping, navigation, segmentation and tracking of objects, trajectory planning, reinforcement learning. Our interns and employees have received a new modern experimental setup to implement their bold ideas. We thank our industrial partner, Dragon Tree Labs, thanks to which our Center will undoubtedly receive a new impetus in the development of our work!