Topological feature guided knowledge distillation for improved wearable sensor data analysis

Even though there have been significant advances in wearable sensor data analysis with deep learning, various challenges remain due to sensor perturbations, personal variability, and unknown real-world noise. While topological data analysis (TDA) has been integrated into model design to alleviate the problems, it inevitably requires substantial and additional computational costs and poses limitations in implementation on small devices, which is critical for wearable applications. Knowledge distillation (KD) has been widely adopted as an effective strategy for developing compact models that leverage knowledge from large teacher models. Recently, a variety of KD methods have been explored to further enhance the effectiveness of knowledge transfer. To use the robustness of topological characteristics within KD, prior studies employ a teacher model trained with persistence images (PI) as input, which also utilizes multiple teachers to provide additional knowledge. However, this approach increases the complexity and imposes challenges in aligning different modalities of representations and network architectures. To address these issues, we propose Topological Feature Guided Knowledge Distillation (TFKD), which transfers both local and global topological knowledge—the comprehensive set of structural and feature-based information captured by persistent homology (pH) from teacher to student by leveraging pH computed via the Sig2PI module in learning embedding spaces across multiple network stages. Our approach efficiently approximates PIs from both the teacher and the student, enabling multi-stage distillation with comprehensive topological supervision. As a result, a superior student model is distilled, which requires no additional computational power or resources and uses only time-series data during inference. For example, on GENEActiv dataset with 14-class, TFKD attains 71.30% of WRN16-1 student from WRN16-3 teacher in accuracy by the proposed framework, where the student model trained from scratch achieves 67.66%. We demonstrate the effectiveness of our framework on wearable sensor data with diverse perspectives, achieving superior generalization and robustness compared to prior methods.