# What is distillation in Neural Network?

Distillation seems to be a general technique to reduce the size of NLP/NN models. Can anyone help me to understand intuition and how does it work?

Some additional links with additional info on knowledge distillation are this documentation page from the Distiller package and this practical example from Weights & Biases.