Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method
Shicheng Tan, W. Tam, Yuanchun Wang, Wenwen Gong, Shuo Zhao, P. Zhang, Jie Tang
Annual Meeting of the Association for Computational Linguistics