Compression Method

Category Compression Method Term Knowledge Distillation Definition of Knowledge Distillation Training A Smaller Student Model To Mimic A Larger Teacher Model’s Outputs Notes About Knowledge Distillation Balances Performance And Efficiency

Category Compression Method Term Quantization Definition of Quantization Reducing Numerical Precision Of Model Weights And Activations To Lower Memory And Accelerate Inference Notes About Quantization Careful Calibration Preserves Accuracy