Developing a Convolutional Neural Network Model for Automated Assessment of Communication and History-Taking Skills in OSCE Transcripts

Author : Mehtap Bekhan, Tuba Mutluer

Abstract :The Objective Structured Clinical Examination (OSCE) is a widely adopted tool in medical education for assessing clinical competencies, including communication and history-taking skills. However, the current manual evaluation processes are subjective, time-consuming, and susceptible to human error. This research presents an innovative approach using Convolutional Neural Networks (CNNs) to automate the evaluation of medical students' communication skills in OSCEs. Our deep learning model analyzes both verbal and nonverbal communication by processing video and audio data from OSCE recordings, providing an objective assessment of medical students' clinical performance. Leveraging a dataset of 4,000 OSCE videos, the model extracts nonverbal cues (e.g., facial expressions, gestures, posture) and verbal elements (e.g., gathering information, showing empathy, providing structure) to comprehensively evaluate students' interactions with standardized patients. Pre-trained CNN architectures, such as GoogleNet and ResNet, are employed for video feature extraction, while audio analysis is conducted using natural language processing (NLP) techniques and speech recognition models like Whisper. By incorporating these diverse data points, the system enhances the objectivity and reliability of communication assessments, offering real-time feedback to both students and educators. Our findings demonstrate that this AI-driven system improves the consistency of OSCE evaluations and provides an efficient means to offer immediate, actionable feedback. Furthermore, this method contributes to reducing evaluator bias, ensuring fair and comprehensive assessments of clinical communication skills at medical schools.

Keywords :Educational Technology, Medical education, Automated assessment, Artificial intelligence, OSCE, Communication skills, Convolutional Neural Networks.

Conference Name :International Conference on Education and Technology (IC-ET-24)

Conference Place Edinburgh , UK

Conference Date 8th Nov 2024