Real-Time Cognitive Monitoring of Surgeons During Intraoperative Performance with a Multimodal Brain-Computer Interface

Author : Prof. OlivierOullie

Abstract : Surgical performance depends on the cognitive, affective and sensorimotor states of clinicians. These fluctuate within seconds, shaping intraoperative decision-making and fine-motor precision. Currently, assessing a surgeon's cognitive readiness or physical fatigue relies on self-reports or static physiological measures that operate on timescales too slow for accurate understanding of the complex neurophysiological and sensorimotor coordination dynamics at play. This methodological paper introduces a protocol for continuous multimodal bio-signal acquisition, fusion, analysis and real-time feedback during live surgery. We detail our technical implementation, operating room (OR) deployment, limitations and clinical rationale, with the ultimate goals of improving clinician well-being, surgical performance while enhancing patient safety and outcomes. The surgeon wears non-restrictive devices: a wireless electroencephalography headset beneath a sterile cap, eye-tracking glasses, physiological and motion sensors. Ambient microphones record audio for language and speech prosody analysis, while cameras capture movements and object manipulation. These multimodal data streams are processed by an AI-powered passive brain computer interface that tracks real-time changes in stress, attention and cognitive load. A visual readout is displayed outside the surgeon's direct field of view to prevent visual distraction. The surgeon can voluntarily consult this screen before critical surgical phases. To prevent any additional distraction in the already busy environment of the OR, the neuroadaptive AI system does not produce any active auditory or visual alerts. Three practical constraints define the surgery envelope: (1) sensors must preserve sterility and unrestricted movement; (2) motion and electrocautery corrupt signals; and (3) neurophysiological baselines vary among surgeons and drift intra-individually across consecutive procedures within a single operative day. Finally, we also discuss how neuroadaptive artificial intelligence systems process multimodal data streams to provide clinically actionable real-time feedback, support the development of cognition aware surgical robotic systems and generate enhanced learning materials for surgery trainees.

Keywords : Neuroadaptive AI; intraoperative electroencephalography; multimodal biosignal fusion; surgical neuroergonomics; cognitive load monitoring.

Conference Name : International Conference on Neuroscience and Clinical Applications (ICNCA-26)

Conference Place : Amsterdam, Netherlands

Conference Date : 11th May 2026