LinguaLive: An AI-Powered Multimodal Real-Time Cross-Language Translation System

Author : Dr. A. Syed Mustafa

Abstract : LinguaLive is a real-time multilingual translation platform built on the fine-tuning and adaptation of the SeamlessM4T model for Indian languages. Integrating Automatic Speech Recognition (ASR), Neural Machine Translation (NMT), and Text-to-Speech (TTS) within a unified deep learning framework, LinguaLive enhances the model’s performance for low-resource regional languages such as Tamil, Hindi, and Malayalam. The system fine-tunes SeamlessM4T using datasets like jhu-clsp/seamless-align and pszemraj/t2t\_ re\_pretrain-small, which together provide large-scale multilingual speech-text alignment and text-to-text pretraining data suitable for improving model generalization across languages. Developed with a Python-based backend and React interface, it provides an efficient and interactive real-time translation experience. Experimental evaluation demonstrates significant improvements—BLEU scores increased by up to 19.4%, WER reduced by 22.6%, and latency reduced by 10.3% compared to the baseline SeamlessM4T. These results validate the effectiveness of domain-specific fine-tuning in addressing linguistic diversity and speech variability across Indian languages. LinguaLive thus establishes a practical pathway for inclusive, context-aware, and high-fidelity multilingual communication in sectors such as education, healthcare, and cross-cultural collaboration.

Keywords : Artificial intelligence, multilingual translation, Indian languages, fine-tuning.

Conference Name : International Conference on Engineering & Technology (ICET - 25)

Conference Place : Bangalore, India

Conference Date : 20th Dec 2025