Cognitive Systems Research, volume 71, pages 52-63
Vector Semiotic Model for Visual Question Answering
Publication type: Journal Article
Publication date: 2022-01-01
Journal:
Cognitive Systems Research
Quartile SCImago
Q1
Quartile WOS
Q1
Impact factor: 3.9
ISSN: 13890417
Artificial Intelligence
Software
Experimental and Cognitive Psychology
Cognitive Neuroscience
Abstract
In this paper, we propose a Vector Semiotic Model as a possible solution to the symbol grounding problem in the context of Visual Question Answering. The Vector Semiotic Model combines the advantages of a Semiotic Approach implemented in the Sign-Based World Model and Vector Symbolic Architectures. The Sign-Based World Model represents information about a scene depicted on an input image in a structured way and grounds abstract objects in an agent’s sensory input. We use the Vector Symbolic Architecture to represent the elements of the Sign-Based World Model on a computational level. Properties of a high-dimensional space and operations defined for high-dimensional vectors allow encoding the whole scene into a high-dimensional vector with the preservation of the structure. That leads to the ability to apply explainable reasoning to answer an input question. We conducted experiments are on a CLEVR dataset and show results comparable to the state of the art. The proposed combination of approaches, first, leads to the possible solution of the symbol-grounding problem and, second, allows expanding current results to other intelligent tasks (collaborative robotics, embodied intellectual assistance, etc.).
Citations by journals
1
|
|
ACM Computing Surveys
|
ACM Computing Surveys
1 publication, 14.29%
|
Electronics (Switzerland)
|
Electronics (Switzerland)
1 publication, 14.29%
|
Frontiers in Artificial Intelligence
|
Frontiers in Artificial Intelligence
1 publication, 14.29%
|
Pattern Recognition and Image Analysis
|
Pattern Recognition and Image Analysis
1 publication, 14.29%
|
Lecture Notes in Computer Science
|
Lecture Notes in Computer Science
1 publication, 14.29%
|
Studies in Computational Intelligence
|
Studies in Computational Intelligence
1 publication, 14.29%
|
1
|
Citations by publishers
1
2
|
|
Springer Nature
|
Springer Nature
2 publications, 28.57%
|
Association for Computing Machinery (ACM)
|
Association for Computing Machinery (ACM)
1 publication, 14.29%
|
Multidisciplinary Digital Publishing Institute (MDPI)
|
Multidisciplinary Digital Publishing Institute (MDPI)
1 publication, 14.29%
|
Frontiers Media S.A.
|
Frontiers Media S.A.
1 publication, 14.29%
|
Pleiades Publishing
|
Pleiades Publishing
1 publication, 14.29%
|
1
2
|
- We do not take into account publications that without a DOI.
- Statistics recalculated only for publications connected to researchers, organizations and labs registered on the platform.
- Statistics recalculated weekly.
{"yearsCitations":{"type":"bar","data":{"show":true,"labels":[2022,2023],"ids":[0,0],"codes":[0,0],"imageUrls":["",""],"datasets":[{"label":"Citations number","data":[3,4],"backgroundColor":["#3B82F6","#3B82F6"],"percentage":["42.86","57.14"],"barThickness":null}]},"options":{"indexAxis":"x","maintainAspectRatio":true,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":1,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Citations per year","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"journals":{"type":"bar","data":{"show":true,"labels":["ACM Computing Surveys","Electronics (Switzerland)","Frontiers in Artificial Intelligence","Pattern Recognition and Image Analysis","Lecture Notes in Computer Science","Studies in Computational Intelligence"],"ids":[2817,5130,28928,9753,1022,2714],"codes":[0,0,0,0,0,0],"imageUrls":["\/storage\/images\/resized\/XZDD1UbkaHV0BImS1Dm7kQfvovjiljgbqNi7vyqK_medium.webp","\/storage\/images\/resized\/MjH1ITP7lMYGxeqUZfkt2BnVLgjkk413jwBV97XX_medium.webp","\/storage\/images\/resized\/4QWA67eqfcfyOiA8Wk7YnqroHFqQbTsmDJUYTCTg_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp","\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp","\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp"],"datasets":[{"label":"","data":[1,1,1,1,1,1],"backgroundColor":["#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6"],"percentage":[14.29,14.29,14.29,14.29,14.29,14.29],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Journals","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}},"publishers":{"type":"bar","data":{"show":true,"labels":["Springer Nature","Association for Computing Machinery (ACM)","Multidisciplinary Digital Publishing Institute (MDPI)","Frontiers Media S.A.","Pleiades Publishing"],"ids":[8,1141,202,208,101],"codes":[0,0,0,0,0],"imageUrls":["\/storage\/images\/resized\/voXLqlsvTwv5p3iMQ8Dhs95nqB4AXOG7Taj7G4ra_medium.webp","\/storage\/images\/resized\/XZDD1UbkaHV0BImS1Dm7kQfvovjiljgbqNi7vyqK_medium.webp","\/storage\/images\/resized\/MjH1ITP7lMYGxeqUZfkt2BnVLgjkk413jwBV97XX_medium.webp","\/storage\/images\/resized\/4QWA67eqfcfyOiA8Wk7YnqroHFqQbTsmDJUYTCTg_medium.webp","\/storage\/images\/resized\/oZgeErrVFhuDksyqFURLvYS1wtVSBWczh001igGo_medium.webp"],"datasets":[{"label":"","data":[2,1,1,1,1],"backgroundColor":["#3B82F6","#3B82F6","#3B82F6","#3B82F6","#3B82F6"],"percentage":[28.57,14.29,14.29,14.29,14.29],"barThickness":13}]},"options":{"indexAxis":"y","maintainAspectRatio":false,"scales":{"y":{"ticks":{"precision":0,"autoSkip":false,"font":{"family":"Montserrat"},"color":"#000000"}},"x":{"ticks":{"stepSize":null,"precision":0,"font":{"family":"Montserrat"},"color":"#000000"}}},"plugins":{"legend":{"position":"top","labels":{"font":{"family":"Montserrat"},"color":"#000000"}},"title":{"display":true,"text":"Publishers","font":{"size":24,"family":"Montserrat","weight":600},"color":"#000000"}}}}}
Metrics
Cite this
GOST |
RIS |
BibTex
Cite this
GOST
Copy
Kovalev A. K. et al. Vector Semiotic Model for Visual Question Answering // Cognitive Systems Research. 2022. Vol. 71. pp. 52-63.
GOST all authors (up to 50)
Copy
Kovalev A. K., Panov A., Shaban M., Osipov E. Vector Semiotic Model for Visual Question Answering // Cognitive Systems Research. 2022. Vol. 71. pp. 52-63.
Cite this
RIS
Copy
TY - JOUR
DO - 10.1016/j.cogsys.2021.09.001
UR - https://doi.org/10.1016%2Fj.cogsys.2021.09.001
TI - Vector Semiotic Model for Visual Question Answering
T2 - Cognitive Systems Research
AU - Kovalev, Alexey K
AU - Panov, Aleksandr
AU - Shaban, Makhmud
AU - Osipov, Evgeny
PY - 2022
DA - 2022/01/01 00:00:00
PB - Elsevier
SP - 52-63
VL - 71
SN - 1389-0417
ER -
Cite this
BibTex
Copy
@article{2022_Kovalev
author = {Alexey K Kovalev and Aleksandr Panov and Makhmud Shaban and Evgeny Osipov},
title = {Vector Semiotic Model for Visual Question Answering},
journal = {Cognitive Systems Research},
year = {2022},
volume = {71},
publisher = {Elsevier},
month = {jan},
url = {https://doi.org/10.1016%2Fj.cogsys.2021.09.001},
pages = {52--63},
doi = {10.1016/j.cogsys.2021.09.001}
}
Profiles