Communications in Computer and Information Science, volume 1093, pages 30-43
Hierarchical Reinforcement Learning with Clustering Abstract Machines
Skrynnik Alexey
1
,
Panov Aleksandr I
1, 2
1
Publication type: Book Chapter
Publication date: 2019-10-13
Quartile SCImago
Q4
Quartile WOS
—
Impact factor: —
ISSN: 18650929
Abstract
Hierarchical reinforcement learning (HRL) is another step towards the convergence of learning and planning methods. The resulting reusable abstract plans facilitate both the applicability of transfer learning and increasing of resilience in difficult environments with delayed rewards. However, on the way of the practical application of HRL, especially in robotics, there are a number of difficulties, among which the key is a semi-manual task of the creation of the hierarchy of actions, which the agent uses as a pre-trained scheme. In this paper, we present a new approach for simultaneous constructing and applying the hierarchy of actions and sub-goals. In contrast to prior efforts in this direction, the method is based on a united loop of clustering of the environment’s states observed by the agent and allocation of sub-targets by the modified bottleneck method for constructing of abstract machines hierarchy. The general machine is built using the so-called programmable schemes, which are quite universal for the organization of transfer learning for a wide class of tasks. A particular abstract machine is assigned for each set of clustered states. The goal of each machine is to reach one of the found bottleneck states and then get into another cluster. We evaluate our approach using a standard suite of experiments on a challenging planning problem domain and show that our approach facilitates learning without prior knowledge.
Citations by journals
|
1
|
|
|
Lecture Notes in Computer Science
|
Lecture Notes in Computer Science
1 publication, 50%
|
|
1
|
Citations by publishers
|
1
|
|
|
Springer Nature
|
Springer Nature
1 publication, 50%
|
|
1
|
- We do not take into account publications that without a DOI.
- Statistics recalculated only for publications connected to researchers, organizations and labs registered on the platform.
- Statistics recalculated weekly.
{"yearsCitations":{"type":"bar","data":{"show":true,"labels":[2021],"ids":[0],"codes":[0],"imageUrls":[""],"datasets":[{"label":"Citations number","data":[2],"backgroundColor":["#3B82F6"],"percentage":["100"],"barThickness":null}]},"options":{"indexAxis":"x","maintainAspectRatio":true,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":1,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Citations per year","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"journals":{"type":"bar","data":{"show":true,"labels":["Lecture Notes in Computer Science"],"ids":[1022],"codes":[0],"imageUrls":["\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp"],"datasets":[{"label":"","data":[1],"backgroundColor":["#3B82F6"],"percentage":[50],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Journals","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"publishers":{"type":"bar","data":{"show":true,"labels":["Springer Nature"],"ids":[8],"codes":[0],"imageUrls":["\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp"],"datasets":[{"label":"","data":[1],"backgroundColor":["#3B82F6"],"percentage":[50],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Publishers","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}}}
Metrics
Cite this
GOST |
RIS |
BibTex
Cite this
GOST
Copy
Skrynnik A., Panov A. I. Hierarchical Reinforcement Learning with Clustering Abstract Machines // Communications in Computer and Information Science. 2019. Vol. 1093. pp. 30-43.
GOST all authors (up to 50)
Copy
Skrynnik A., Panov A. I. Hierarchical Reinforcement Learning with Clustering Abstract Machines // Communications in Computer and Information Science. 2019. Vol. 1093. pp. 30-43.
Cite this
RIS
Copy
TY - GENERIC
DO - 10.1007/978-3-030-30763-9_3
UR - https://doi.org/10.1007%2F978-3-030-30763-9_3
TI - Hierarchical Reinforcement Learning with Clustering Abstract Machines
T2 - Communications in Computer and Information Science
AU - Skrynnik, Alexey
AU - Panov, Aleksandr I
PY - 2019
DA - 2019/10/13 00:00:00
PB - Springer Nature
SP - 30-43
VL - 1093
SN - 1865-0929
ER -
Cite this
BibTex
Copy
@incollection{2019_Skrynnik,
author = {Alexey Skrynnik and Aleksandr I Panov},
title = {Hierarchical Reinforcement Learning with Clustering Abstract Machines},
publisher = {Springer Nature},
year = {2019},
volume = {1093},
pages = {30--43},
month = {oct}
}
Profiles