HSKD: Hidden State Knowledge Distillation을 이용한 배터리 SoC 예측 모델 경량화

다국어 초록 (Multilingual Abstract)

Knowledge Distillation (KD) is one of the representative methods for AI model compression, where a student model learns by imitating the output of a teacher model. The student model has a smaller network than the teacher model, which can reduce inference time and save memory. This method should be applied for efficient AI model inference in limited computing environments, such as the vehicle controller. In this paper, we applied the Hidden State Knowledge Distillation (HSKD) method to a Bi-LSTM (Bidirectional Long Short Term Memory) model for predicting the State of Charge (SoC) of an electric vehicle battery. This model predicts the SoC 5 minutes ahead using the SoC of the past 5 minutes. In the experiment, we selected a teacher model with a hidden size of 1,024, which showed the highest accuracy, and compared the performance of hidden state knowledge distillation and general knowledge distillation models for models with a hidden size smaller than 1,024. And, we measured the inference time of the compressed models on controllers equipped with ARM Cortex-A53. As a result, the model with a hidden size of 32 had a loss of 0.008 in terms of R2 score compared to the teacher model, but the inference time was reduced by approximately 20.1x and the file size was compressed by 750.6x from 33,028 [KB] to 44 [KB].

목차 (Table of Contents)

• Abstract
• 1. 서론
• 2. 본론
• 3. 실험결과
• 4. 결론
• References