Tri-Modal Deep Learning Forecasting Framework for Tropical Cyclone Prediction Under Data Scarcity.

Author : Aishwarya Y. Kadam, Dr. Shwetambari Chiwhane

Abstract : Even with these advances, the complex nature of climatic processes in the atmosphere, sparse observation coverage, diverse data sources, and the dynamic nature of tropical cyclones (TCs) remain a challenge to predicting TC tracks and intensity. The existing deep learning approaches primarily rely on unimodal or bimodal observations, and frequently assume full observation or fully synchronized input, which are not always applicable and robust to observation deficient and data-scarce settings. The present work suggests a novel approach of taking a multi-modal deep learning approach to jointly predict the track and intensity of TCs using satellite imagery, atmospheric meteorological variables and historical cyclone track sequences. The results are hypothesized to be learnt by using an Improved Vision Transformer for spatial and structural information and historical cyclone trajectories and reanalysis variables for temporal and atmospheric information. Different models will be discussed for reconstructing missing modalities and information when it is corrupted to enable forecasting when information is missing. The relationships between the three modalities will then be learned with a cross attention-based fusion mechanism and physics-informed constraints will be added to enhance the physical consistency of forecasts. The proposed framework will also at the same time estimate the future parameters of cyclones like central pressure and maximum sustained wind. Its usefulness will be evaluated against well-available tropical cyclone track and intensity data from various basins by the errors in track and intensity, and the ability to reconstruct the cyclones, as well as by comparative experiments with available deep learning and multimodal track and intensity forecasting models. The aim is to produce a reliable and generally applicable forecasting framework that would maintain good forecasting skills when meteorological data is limited or not available.

Keywords : Tropical cyclone forecasting, tri-modal learning, missing modality reconstruction, multimodal data fusion, vision transformer, cyclone track prediction, cyclone intensity prediction.

Conference Name : International Conference on AI-driven Data Science for Environmental Monitoring (ICIADSEM - 26)

Conference Place : Pune, India

Conference Date : 26th Sep 2026