Knowledge Distillation
The process of transferring the knowledge of a large and complex model to a smaller model.
The small 'student' model in information distillation works to achieve similar performance with much less calculation source by mimicing the output potential distributions of the large 'student' model.
Member comments