Online Adaptation of Compressed Models by Pre-Training and Task-Relevant Pruning

Thomas Avé, Matthias Hutsebaut-Buysse, Wei Wei, Kevin Mets

The 32th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium, 9-11 October 2024 

Deep learning models are increasingly deployed on edge devices, where they must adapt to new data in dynamic environments. These edge devices are inherently resource-constrained, making model compression techniques such as pruning essential. This involves removing redundant neurons from a model, making it more efficient at the potential cost of accuracy, creating a conflict between efficiency and adaptability. We propose a novel method for training and compressing models that maintains and extends their ability to generalize to new data, enabling better online adaptation without sacrificing compression rates. By pre-training the model on additional knowledge and identifying the parts of the deep neural network that actually encode task-relevant knowledge, we can effectively prune the model by 80% and achieve accuracies that are 16% higher when adapting to a new domain.