From traditional algorithms to artificial intelligence: a review of the development of sensory substitution sonification methods

This review investigates trends in the development of image sonification methods applied in visual-to-auditory sensory substitution systems. The authors consider both traditional visual-to-auditory sensory substitution sonification methods, such as The vOICe and EyeMusic, and modern approaches with artificial intelligence (AI). The papers for the review were collected from Google Scholar and Lens search databases using thoroughly composed search queries based on selected keywords. As a result of the review, the authors proposed a classification of traditional sonification methods depending on the set of visual-to-auditory mappings they use: (1) column-by-column sonification, (2) sonification of the entire image into one complex auditory signal, and (3) 3D sonification—using depth as the main visual input. AI-based papers were classified into algorithms which use AI for input data preprocessing and algorithms which use AI for output data postprocessing in the visual-to-auditory sensory substitution systems. The authors describe the existing relationships between traditional visual-to-auditory sensory substitution sonification methods and visual-to-auditory sensory substitution sonification methods with AI. In conclusion, the advantages and disadvantages of existing solutions are formulated; in particular, it is shown how traditional and up-to-date solutions attempt to solve the problem of sensory–cognitive overload experienced by users of visual-to-auditory sensory substitution systems.