Open Access

Lecture Notes in Computer Science, volume 12886 LNAI, pages 243-255

Applying Vector Symbolic Architecture and Semiotic Approach to Visual Dialog

Kovalev Alexey K ^{1, 2}

Shaban Makhmud ²

Chuganskaya Anfisa A ¹

Panov Aleksandr I ^{1, 3}

Hide authors affiliations Show authors affiliations: 3 affiliations

Artificial Intelligence Research Institute FRC CSC RAS, Moscow, Russia

Artificial Intelligence Research Institute

Federal Research Center Computer Science and Control of the Russian Academy of Sciences

Hse University, Moscow, Russia |

Moscow Institute of Physics and Technology, Moscow, russia |

Publication type: Book Chapter

Publication date: 2021-09-14

Springer Nature

Journal: Lecture Notes in Computer Science

Quartile SCImago

Quartile WOS

—

Impact factor: —

ISSN: 03029743, 16113349, 18612075, 18612083

DOI: 10.1007/978-3-030-86271-8_21

Copy DOI

Abstract

The multi-modal tasks have started to play a significant role in the research on Artificial Intelligence. A particular example of that domain is visual-linguistic tasks, such as Visual Question Answering and its extension, Visual Dialog. In this paper, we concentrate on the Visual Dialog task and dataset. The task involves two agents. The first agent does not see an image and asks questions about the image content. The second agent sees this image and answers questions. The symbol grounding problem, or how symbols obtain their meanings, plays a crucial role in such tasks. We approach that problem from the semiotic point of view and propose the Vector Semiotic Architecture for Visual Dialog. The Vector Semiotic Architecture is a combination of the Sign-Based World Model and Vector Symbolic Architecture. The Sign-Based World Model represents agent knowledge on the high level of abstraction and allows uniform representation of different aspects of knowledge, forming a hierarchical representation of that knowledge in the form of a special kind of semantic network. The Vector Symbolic Architecture represents the computational level and allows to operate with symbols as with numerical vectors using simple element-wise operations. That combination enables grounding object representation from any level of abstraction to the sensory agent input.

By date By citations

Citations by journals

	1
ACM Computing Surveys	ACM Computing Surveys, 1, 50% ACM Computing Surveys 1 publication, 50%
Lecture Notes in Computer Science	Lecture Notes in Computer Science, 1, 50% Lecture Notes in Computer Science 1 publication, 50%
	1

Citations by publishers

	1
Association for Computing Machinery (ACM)	Association for Computing Machinery (ACM), 1, 50% Association for Computing Machinery (ACM) 1 publication, 50%
Springer Nature	Springer Nature, 1, 50% Springer Nature 1 publication, 50%
	1

We do not take into account publications that without a DOI.
Statistics recalculated only for publications connected to researchers, organizations and labs registered on the platform.
Statistics recalculated weekly.

{"yearsCitations":{"type":"bar","data":{"show":true,"labels":[2023],"ids":[0],"codes":[0],"imageUrls":[""],"datasets":[{"label":"Citations number","data":[2],"backgroundColor":["#3B82F6"],"percentage":["100"],"barThickness":null}]},"options":{"indexAxis":"x","maintainAspectRatio":true,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":1,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Citations per year","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"journals":{"type":"bar","data":{"show":true,"labels":["ACM Computing Surveys","Lecture Notes in Computer Science"],"ids":[2817,1022],"codes":[0,0],"imageUrls":["\/storage\/images\/resized\/XZDD1UbkaHV0BImS1Dm7kQfvovjiljgbqNi7vyqK_medium.webp","\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp"],"datasets":[{"label":"","data":[1,1],"backgroundColor":["#3B82F6","#3B82F6"],"percentage":[50,50],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Journals","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"publishers":{"type":"bar","data":{"show":true,"labels":["Association for Computing Machinery (ACM)","Springer Nature"],"ids":[1141,8],"codes":[0,0],"imageUrls":["\/storage\/images\/resized\/XZDD1UbkaHV0BImS1Dm7kQfvovjiljgbqNi7vyqK_medium.webp","\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp"],"datasets":[{"label":"","data":[1,1],"backgroundColor":["#3B82F6","#3B82F6"],"percentage":[50,50],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Publishers","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}}}

Metrics

Cite this

GOST |

Cite this

GOST Copy

Kovalev A. K. et al. Applying Vector Symbolic Architecture and Semiotic Approach to Visual Dialog // Lecture Notes in Computer Science. 2021. Vol. 12886 LNAI. pp. 243-255.

GOST all authors (up to 50) Copy

Kovalev A. K., Shaban M., Chuganskaya A. A., Panov A. I. Applying Vector Symbolic Architecture and Semiotic Approach to Visual Dialog // Lecture Notes in Computer Science. 2021. Vol. 12886 LNAI. pp. 243-255.

RIS |

Cite this

RIS Copy

TY - GENERIC

DO - 10.1007/978-3-030-86271-8_21

UR - https://doi.org/10.1007%2F978-3-030-86271-8_21

TI - Applying Vector Symbolic Architecture and Semiotic Approach to Visual Dialog

T2 - Lecture Notes in Computer Science

AU - Kovalev, Alexey K

AU - Shaban, Makhmud

AU - Chuganskaya, Anfisa A

AU - Panov, Aleksandr I

PY - 2021

DA - 2021/09/14 00:00:00

PB - Springer Nature

SP - 243-255

VL - 12886 LNAI

SN - 0302-9743

SN - 1611-3349

SN - 1861-2075

SN - 1861-2083

ER -

BibTex

Cite this

BibTex Copy

@incollection{2021_Kovalev,

author = {Alexey K Kovalev and Makhmud Shaban and Anfisa A Chuganskaya and Aleksandr I Panov},

title = {Applying Vector Symbolic Architecture and Semiotic Approach to Visual Dialog},

publisher = {Springer Nature},

year = {2021},

volume = {12886 LNAI},

pages = {243--255},

month = {sep}

}

Found error?

Publisher

Springer Nature

Journal

Lecture Notes in Computer Science

Quartile SCImago

Quartile WOS

—

Impact factor

—

ISSN

03029743 (Print)
16113349 (Electronic)
18612075 (Print)
18612083 (Electronic)

Labs

MIPT Center for Cognitive Modeling

Profiles

Panov, Aleksandr I