Abstract:

The rapid proliferation of unmanned aerial vehicles (UAVs) has increasingly posed significant challenges for airspace security, particularly under long-range and visually degraded conditions. Effective UAV recognition is thus critical, yet current methodologies typically depend on single-sensor inputs, such as infrared (IR) imaging and radio frequency (RF) analysis, which suffer inherent limitations in complex environments. Although multimodal sensing has been explored in UAV detection, the joint exploitation of IR imagery and RF signals for UAV type recognition remains largely underexplored. The structural heterogeneity between IR and RF features presents challenges for joint representation and decision-making, which remains underexplored in previous work. To address this gap, this paper proposes RF-IRSynNet, a multimodal UAV classification framework that integrates IR imagery and in-flight RF emissions to enhance recognition performance. In RF-IRSynNet, IR images are processed using YOLOv11 to detect UAV candidates and extract structured semantic features. Meanwhile, RF signals are modeled using reservoir computing, which efficiently encodes temporal and spectral dynamics via feature sequences. These modalities are fused through an adaptive confidence-weighted soft-voting strategy, dynamically balancing their contributions based on specific tasks. Experimental results demonstrate that RF-IRSynNet outperforms both unimodal baselines and existing multimodal approaches, achieving robust classification at long ranges. The framework maintains high accuracy even with reduced training data, indicating high efficiency for real-world UAV monitoring.

Key words: unmanned aerial vehicle (UAV) detection, multimodal, reservoir computing, YOLO, kernel canonical correlation analysis