BiSign: Bidirectional Cross-Lingual Transfer Learning for Sign Language Recognition

Author : Ankit Utkarsh Hota, Yash Ranjan, Brindha R, Dr. Malarselvi G, Dr. Ramesh S

Abstract : Sign language recognition (SLR) systems suffer from severe data scarcity outside of American Sign Language (ASL). We present BiSign, a cross-lingual transfer learning framework that trains a single shared encoder on ASL and Indian Sign Language (ISL) jointly, using a language-ID conditioning token and NT-Xent contrastive loss to align embedding spaces across languages. Evaluated on the WLASL (1,575 classes) and INCLUDE (239 classes) benchmarks over three random seeds, our key results are: (1) a 5-shot ASL-pretrained model achieves 53.9 ± 0.8% ISL Top-1 accuracy versus 0.6 ± 0.4% for 5-shot scratch training (+52.9 pp); (2) a 10-shot pretrained model reaches 66.7 ± 0.2% versus 57.1% fulldata scratch (+9.6 pp with 14× less labelled data); (3) zero-shot transfer achieves 32.0%, which is 77× above random chance. Ablations confirm the language-ID token contributes +9.3 pp and the shared encoder contributes +16.2 pp over an ISL-only separate encoder.

Keywords : Sign language recognition, cross-lingual transfer learning, few-shot learning, transformer, MediaPipe, WLASL, INCLUDE.

Conference Name : International Conference on Disability and Community Health Programs (ICDCHP-26)

Conference Place : Bhubaneswar, India

Conference Date : 4th Apr 2026