Distilhubert: Speech Representation Learning by Layer-Wise Distillation of Hidden-Unit Bert
Heng-Jui Chang, Shu-Wen Yang, Hung-yi Lee
IEEE International Conference on Acoustics, Speech, and Signal Processing