Subjective Distance Estimates and Sense of Agency in Robotic Wheelchair Control

Seghezzi S., Convertino L., Zapparoli L.

• Abnormal agency experiences characterize different movement disorders. • Agency disturbances are associated with SMA structural and functional abnormalities. • Different measures of the sense of agency often provide divergent results. Sense of agency refers to the experience that one’s self-generated action causes an event in the external environment. Here, we review the behavioural and brain evidence of aberrant experiences of agency in movement disorders, clinical conditions characterized by either a paucity or an excess of movements unrelated to the patient’s intention. We show that specific abnormal agency experiences characterize several movement disorders. Those manifestations are typically associated with structural and functional brain abnormalities. However, the evidence is sometimes conflicting, especially when considering results obtained through different agency measures. The present review aims to create order in the existing literature on sense of agency investigations in movement disorders and to provide a coherent overview framed within current neurocognitive models of motor awareness.

Wen W., Yun S., Yamashita A., Northcutt B.D., Asama H.

Driving assistance technology has gained traction in recent years and is becoming more widely used in vehicles. However, drivers usually experience a reduced sense of agency when driving assistance is active even though automated assistance improves driving performance by reducing human error and ensuring quick reactions. The present study examined whether driving assistance can maintain human sense of agency during early deceleration in the face of collision risk, compared with manual deceleration. In the experimental task, participants decelerate their vehicle in a driving simulator to avoid collision with a vehicle that suddenly cut in front of them and decelerated. In the assisted condition, the system performed deceleration 100 ms after the cut-in. Participants were instructed to decelerate their vehicle and follow the vehicle that cut-in. This design ensured that the deceleration assistance applied a similar control to the vehicle as the drivers intended to, only faster and smoother. Participants rated their sense of agency and their driving performance. The results showed that drivers maintained their sense of agency and improved driving performance under driving assistance. The findings provided insights into designing driving assistance that can maintain drivers’ sense of agency while improving future driving performance. It is important to establish a mode of joint-control in which the system shares the intention of human drivers and provides improved execution of control.

Sorokoumov P.S., Rovbo M.A., Moscowsky A.D., Malyshev A.A.

Problem statement: Wheelchairs have a widespread usage both as a rehabilitation tool and as an assistive device in the daily lives of people with disabilities. However, some disabilities make it difficult to use traditional wheelchairs that are operated manually or with a joystick. This paper describes an intelligent control system for wheelchair automatization, which allows the user to give commands by different means—voice, eye movements, muscle tension. The object recognition system and logical processing of commands supports a wide variety of interfaces and commands. To achieve these goals, a semiotic model of the world is used. Purpose of research: development of a control system for a robotic wheelchair that supports multimodal interfaces and has a high level of automation that enables efficient operation for users with various disabilities. Results: The paper describes the developed architecture of the control system based on the semiotic model of the world, modules for the speech interface, gaze control, and the interface based on myosensors. The navigation system and the processing module for the semiotic world model ensure the safe execution of user commands, including movement, object recognition and processing of commands that contain references to known objects. The system supports interfacing with a manipulator, which is controlled using a linguistic model: a language for describing actions representing the admissible movements of the manipulator in the form of a formal grammar. The study tested the proposed system on a developed model of the robot and a detailed model of the room in a Gazebo environment, as well as on the corresponding software and hardware implementation of the robotic wheelchair. Practical significance: Automation and the use of models and methods of artificial intelligence in the development of wheelchairs allows us to make them more versatile, which increases the number of users for whom a particular model is suitable, reduces the strain on the operator of the wheelchair and expands the capabilities of the wheelchair. The proposed usage of the sign models achieves these goals by combining logical processing of commands as a means of handling multimodal interfaces and executing complex commands.

Anjum A., Yazid M.D., Fauzi Daud M., Idris J., Ng A.M., Selvi Naicker A., Ismail O.H., Athi Kumar R.K., Lokanathan Y.

Spinal cord injury (SCI) is a destructive neurological and pathological state that causes major motor, sensory and autonomic dysfunctions. Its pathophysiology comprises acute and chronic phases and incorporates a cascade of destructive events such as ischemia, oxidative stress, inflammatory events, apoptotic pathways and locomotor dysfunctions. Many therapeutic strategies have been proposed to overcome neurodegenerative events and reduce secondary neuronal damage. Efforts have also been devoted in developing neuroprotective and neuro-regenerative therapies that promote neuronal recovery and outcome. Although varying degrees of success have been achieved, curative accomplishment is still elusive probably due to the complex healing and protective mechanisms involved. Thus, current understanding in this area must be assessed to formulate appropriate treatment modalities to improve SCI recovery. This review aims to promote the understanding of SCI pathophysiology, interrelated or interlinked multimolecular interactions and various methods of neuronal recovery i.e., neuroprotective, immunomodulatory and neuro-regenerative pathways and relevant approaches.

Petrini F.M., Valle G., Bumbasirevic M., Barberi F., Bortolotti D., Cvancara P., Hiairrassary A., Mijovic P., Sverrisson A.Ö., Pedrocchi A., Divoux J., Popovic I., Lechler K., Mijovic B., Guiraud D., et. al.

Recreated missing leg sensations, connected to the prosthesis sensors, restored leg functional abilities and promoted its cognitive integration. Good vibrations The lack of sensory feedback from the leg prosthesis in lower limb amputees is associated with risk of falls, low mobility, and perception of the prosthesis as external object. Here, Petrini et al. tested a leg neuroprosthesis, which provided real-time on-demand tactile sensory feedback through nerve stimulation in three transfemoral amputees. The stimulation improved mobility, decreased falling episodes, and increased the perception of the prosthesis as part of the body. Active complex tasks were accomplished with reduced effort when the nerve stimulation was turned on. The results suggest that real-time nerve stimulation could help restore natural sensation in lower leg amputees. Lower limb amputation (LLA) destroys the sensory communication between the brain and the external world during standing and walking. Current prostheses do not restore sensory feedback to amputees, who, relying on very limited haptic information from the stump-socket interaction, are forced to deal with serious issues: the risk of falls, decreased mobility, prosthesis being perceived as an external object (low embodiment), and increased cognitive burden. Poor mobility is one of the causes of eventual device abandonment. Restoring sensory feedback from the missing leg of above-knee (transfemoral) amputees and integrating the sensory feedback into the sensorimotor loop would markedly improve the life of patients. In this study, we developed a leg neuroprosthesis, which provided real-time tactile and emulated proprioceptive feedback to three transfemoral amputees through nerve stimulation. The feedback was exploited in active tasks, which proved that our approach promoted improved mobility, fall prevention, and agility. We also showed increased embodiment of the lower limb prosthesis (LLP), through phantom leg displacement perception and questionnaires, and ease of the cognitive effort during a dual-task paradigm, through electroencephalographic recordings. Our results demonstrate that induced sensory feedback can be integrated at supraspinal levels to restore functional abilities of the missing leg. This work paves the way for further investigations about how the brain interprets different artificial feedback strategies and for the development of fully implantable sensory-enhanced leg neuroprostheses, which could drastically ameliorate life quality in people with disability.

Karpov V.E., Malakhov D.G., Moscowsky A.D., Rovbo M.A., Sorokoumov P.S., Velichkovsky B.M., Ushakov V.L.

V.E. Karpov, PhD, Head of the Laboratory of Robotics1; Leading Researcher, Research Institute of Information Technologies2; Associate Professor, Institute of Nano-, Bio-, Information, Cognitive and Socio-Humanistic Sciences and Technologies2; D.G. Malakhov, Research-Engineer1; A.D. Moscowsky, Research-Engineer1; M.A. Rovbo, Research-Engineer1; P.S. Sorokoumov, Research-Engineer1; B.M. Velichkovsky, DSc, Professor, Corresponding Member of the Russian Academy of Sciences, Chief Researcher1, 3; Senior Professor4; V.L. Ushakov, PhD, Associate Professor, Leading Researcher1; Senior Researcher5

Huang Q., Chen Y., Zhang Z., He S., Zhang R., Liu J., Zhang Y., Shao M., Li Y.

Objective. In this study, we combine a wheelchair and an intelligent robotic arm based on an electrooculogram (EOG) signal to help patients with spinal cord injuries (SCIs) accomplish a self-drinking task. The main challenge is to accurately control the wheelchair to ensure that the randomly located object is within a limited reachable space of the robotic arm (length: 0.8 m; width: 0.4 m; height: 0.6 m), which requires decimeter-level precision, and is still undemonstrated for EOG-based systems as well as EEG-based systems. Approach. A novel high-precision EOG-based human machine interface (HMI) is proposed which can effectively translate two kinds of eye movements (i.e. blinking and eyebrow raising) into various commands. For the wheelchair, positional precision can reach decimeter-level and the minimal steering angle is . For the intelligent robotic arm, shared control is implemented based on an EOG-based HMI, two cameras and the arm’s own intelligence. Main results. After brief training, five healthy subjects and five paralyzed patients with severe SCIs successfully completed three experiments. For the healthy subjects/patients with SCIs, the system achieved an average accuracy of 99.3%/97.3%, an average response time of 1.91 s/2.02 s per command and an average stop-response time of 1.30 s/1.36 s with a 0 false operation rate. Significance. The EOG-based HMI can provide sufficient precision control to integrate a wheelchair and a robotic arm into a system which can help patients with SCIs to accomplish a self-drinking task. (ChiCTR1800019764)

Kirsch W., Kunde W., Herbort O.

The present study examined the role of voluntary motor commands in the subjective temporal attraction between an action and its sensory consequence termed as intentional binding. Participants either pressed a key voluntarily or involuntarily while seeing a rotating clock hand. The key press was followed by a short beep tone in some blocks of trials. Then, the position of the clock hand at action or tone occurrence was judged. Trials in which key presses and tones occurred separately provided baseline measures. A direct comparison of baseline uncorrected estimates between both action conditions indicated less binding for involuntary than for voluntary movements as reported by previous studies. However, this effect disappeared after a baseline correction and when we controlled for the temporal predictability of critical events. These results cast substantial doubts on a close link between action intention and intentional binding, but instead highlight the role of causal inference and multisensory integration processes.

Yun S., Wen W., An Q., Hamasaki S., Yamakawa H., Tamura Y., Yamashita A., Asama H.

It is important to evaluate and maintain driver’s sense of agency (SoA), because poor SoA of assisted driver may result in slow and inaccurate response in case decisions are required from the driver. This study investigated the relationship between SoA and alpha-band power of EEG in a simulated driving environment.

Haggard P.

The experience of controlling our own actions is an important feature of human mental life. The processes giving rise to this experience are thought to be disrupted in some psychiatric disorders. In this article, Haggard describes recent developments in our understanding of the cognitive processes and neural mechanisms underlying the sense of agency. In adult life, people normally know what they are doing. This experience of controlling one's own actions and, through them, the course of events in the outside world is called 'sense of agency'. It forms a central feature of human experience; however, the brain mechanisms that produce the sense of agency have only recently begun to be investigated systematically. This recent progress has been driven by the development of better measures of the experience of agency, improved design of cognitive and behavioural experiments, and a growing understanding of the brain circuits that generate this distinctive but elusive experience. The sense of agency is a mental and neural state of cardinal importance in human civilization, because it is frequently altered in psychopathology and because it underpins the concept of responsibility in human societies.

Moore J.W.

Sense of agency refers to the feeling of control over actions and their consequences. In this article I summarize what we currently know about sense of agency; looking at how it is measured and what theories there are to explain it. I then explore some of the potential applications of this research, something that the sense of agency research field has been slow to identify and implement. This is a pressing concern given the increasing importance of ‘research impact.’

Kirsch W., Pfister R., Kunde W.

The temporal interval between an action and its ensuing effect is perceptually compressed. Specifically, the perceived onset of actions is shifted towards their effects in time and, vice versa, the perceived onset of effects is shifted towards their causing actions. In four experiments, we report evidence showing that action-effect binding also occurs in the spatial domain. Participants controlled the location of a visual stimulus by performing stylus movements before they judged either the position of the stylus or the position of the visual stimulus. The results yielded spatial binding between the perceived stylus position and the perceived stimulus position when the stimulus was under full control of the hand movement compared to control conditions without direct control.

Braun N., Thorne J.D., Hildebrandt H., Debener S.

The sense of agency (SoA) refers to the phenomenal experience of initiating and controlling an action, whereas the sense of ownership (SoO) describes the feeling of myness an agent experiences towards his or her own body parts. SoA has been investigated with intentional binding paradigms, and the sense of ownership (SoO) with the rubber-hand illusion (RHI). We investigated the relationship between SoA and SoO by incorporating intentional binding into the RHI. Explicit and implicit measures of agency (SoA-questionnaire, intentional binding) and ownership (SoO-questionnaire, proprioceptive drift) were used. Artificial hand position (congruent/incongruent) and mode of agent (self-agent/other-agent) were systematically varied. Reported SoO varied mainly with position (higher in congruent conditions), but also with agent (higher in self-agent conditions). Reported SoA was modulated by agent (higher in self-agent conditions), and moderately by position (higher in congruent conditions). Implicit and explicit agency measures were not significantly correlated. Finally, intentional binding tended to be stronger in self-generated than observed voluntary actions. Results provide further evidence for a partial double dissociation between SoA and SoO, empirically distinct agency levels, and moderate intentional binding differences between self-generated and observed voluntary actions.

Dewey J.A., Knoblich G.

The sense of agency (SoA) refers to perceived causality of the self, i.e. the feeling of causing something to happen. The SoA has been probed using a variety of explicit and implicit measures. Explicit measures include rating scales and questionnaires. Implicit measures, which include sensory attenuation and temporal binding, use perceptual differences between self- and externally generated stimuli as measures of the SoA. In the present study, we investigated whether the different measures tap into the same self-attribution processes by determining whether individual differences on implicit and explicit measures of SoA are correlated. Participants performed tasks in which they triggered tones via key presses (operant condition) or passively listened to tones triggered by a computer (observational condition). We replicated previously reported effects of sensory attenuation and temporal binding. Surprisingly the two implicit measures of SoA were not significantly correlated with each other, nor did they correlate with the explicit measures of SoA. Our results suggest that some explicit and implicit measures of the SoA may tap into different processes.

Limerick H., Coyle D., Moore J.W.

The sense of agency is the experience of controlling both one’s body and the external environment. Although the sense of agency has been studied extensively, there is a paucity of studies in applied “real-life” situations. One applied domain that seems highly relevant is human-computer-interaction (HCI), as an increasing number of our everyday agentive interactions involve technology. Indeed, HCI has long recognized the feeling of control as a key factor in how people experience interactions with technology. The aim of this review is to summarize and examine the possible links between sense of agency and understanding control in HCI. We explore the overlap between HCI and sense of agency for computer input modalities and system feedback, computer assistance, and joint actions between humans and computers. An overarching consideration is how agency research can inform HCI and vice versa. Finally, we discuss the potential ethical implications of personal responsibility in an ever-increasing society of technology users and intelligent machine interfaces.

Yashin A.S., Lavrov D.S., Melnichuk E.V., Karpov V.V., Zhao D.G., Dubynin I.A.

AbstractMobile robots have many applications in the modern world. The autonomy of robots is increasing, but critical cases like search and rescue missions must involve the possibility of human intervention for ethical reasons and safety. To achieve effective human–robot interaction, the operator needs to have a sense of agency (SoA) over the activities of the robot. One possible way to increase one's SoA in remote control could be the use of VR technology. The remote control situation has some important features, so indicators of SoA need to be reproduced there independently. In our study, participants controlled a mobile robot using either a monitor or a VR-headset as an output device. In both cases, active control was contrasted with passive observation of the robot's movement. In each trial, participants estimated the distance traveled by the robot—a putative implicit indicator of SoA. A significant difference between subjective distance estimates was found in the active and passive conditions with the monitor, but not in the active and passive conditions with VR. The effect obtained in the monitor conditions suggests that distance estimates can be used as an implicit indicator of SoA in robot remote control. We believe that the lack of difference between the active and passive conditions in VR was caused by motion sickness due to a mismatch of visual and vestibular sensory cues, leading to a weakened SoA.