Cognitive Systems Research, volume 71, pages 52-63

Vector Semiotic Model for Visual Question Answering

Publication typeJournal Article
Publication date2022-01-01
Quartile SCImago
Q1
Quartile WOS
Q1
Impact factor3.9
ISSN13890417
Artificial Intelligence
Software
Experimental and Cognitive Psychology
Cognitive Neuroscience
Abstract
In this paper, we propose a Vector Semiotic Model as a possible solution to the symbol grounding problem in the context of Visual Question Answering. The Vector Semiotic Model combines the advantages of a Semiotic Approach implemented in the Sign-Based World Model and Vector Symbolic Architectures. The Sign-Based World Model represents information about a scene depicted on an input image in a structured way and grounds abstract objects in an agent’s sensory input. We use the Vector Symbolic Architecture to represent the elements of the Sign-Based World Model on a computational level. Properties of a high-dimensional space and operations defined for high-dimensional vectors allow encoding the whole scene into a high-dimensional vector with the preservation of the structure. That leads to the ability to apply explainable reasoning to answer an input question. We conducted experiments are on a CLEVR dataset and show results comparable to the state of the art. The proposed combination of approaches, first, leads to the possible solution of the symbol-grounding problem and, second, allows expanding current results to other intelligent tasks (collaborative robotics, embodied intellectual assistance, etc.).

Citations by journals

1
ACM Computing Surveys
ACM Computing Surveys, 1, 14.29%
ACM Computing Surveys
1 publication, 14.29%
Electronics (Switzerland)
Electronics (Switzerland), 1, 14.29%
Electronics (Switzerland)
1 publication, 14.29%
Frontiers in Artificial Intelligence
Frontiers in Artificial Intelligence, 1, 14.29%
Frontiers in Artificial Intelligence
1 publication, 14.29%
Pattern Recognition and Image Analysis
Pattern Recognition and Image Analysis, 1, 14.29%
Pattern Recognition and Image Analysis
1 publication, 14.29%
Lecture Notes in Computer Science
Lecture Notes in Computer Science, 1, 14.29%
Lecture Notes in Computer Science
1 publication, 14.29%
Studies in Computational Intelligence
Studies in Computational Intelligence, 1, 14.29%
Studies in Computational Intelligence
1 publication, 14.29%
1

Citations by publishers

1
2
Springer Nature
Springer Nature, 2, 28.57%
Springer Nature
2 publications, 28.57%
Association for Computing Machinery (ACM)
Association for Computing Machinery (ACM), 1, 14.29%
Association for Computing Machinery (ACM)
1 publication, 14.29%
Multidisciplinary Digital Publishing Institute (MDPI)
Multidisciplinary Digital Publishing Institute (MDPI), 1, 14.29%
Multidisciplinary Digital Publishing Institute (MDPI)
1 publication, 14.29%
Frontiers Media S.A.
Frontiers Media S.A., 1, 14.29%
Frontiers Media S.A.
1 publication, 14.29%
Pleiades Publishing
Pleiades Publishing, 1, 14.29%
Pleiades Publishing
1 publication, 14.29%
1
2
  • We do not take into account publications that without a DOI.
  • Statistics recalculated only for publications connected to researchers, organizations and labs registered on the platform.
  • Statistics recalculated weekly.
Metrics
Share
Cite this
GOST |
Cite this
GOST Copy
Kovalev A. K. et al. Vector Semiotic Model for Visual Question Answering // Cognitive Systems Research. 2022. Vol. 71. pp. 52-63.
GOST all authors (up to 50) Copy
Kovalev A. K., Panov A., Shaban M., Osipov E. Vector Semiotic Model for Visual Question Answering // Cognitive Systems Research. 2022. Vol. 71. pp. 52-63.
RIS |
Cite this
RIS Copy
TY - JOUR
DO - 10.1016/j.cogsys.2021.09.001
UR - https://doi.org/10.1016%2Fj.cogsys.2021.09.001
TI - Vector Semiotic Model for Visual Question Answering
T2 - Cognitive Systems Research
AU - Kovalev, Alexey K
AU - Panov, Aleksandr
AU - Shaban, Makhmud
AU - Osipov, Evgeny
PY - 2022
DA - 2022/01/01 00:00:00
PB - Elsevier
SP - 52-63
VL - 71
SN - 1389-0417
ER -
BibTex
Cite this
BibTex Copy
@article{2022_Kovalev
author = {Alexey K Kovalev and Aleksandr Panov and Makhmud Shaban and Evgeny Osipov},
title = {Vector Semiotic Model for Visual Question Answering},
journal = {Cognitive Systems Research},
year = {2022},
volume = {71},
publisher = {Elsevier},
month = {jan},
url = {https://doi.org/10.1016%2Fj.cogsys.2021.09.001},
pages = {52--63},
doi = {10.1016/j.cogsys.2021.09.001}
}
Found error?