Three events that look alike need separate names.
Overview This post is divided into three parts; they are: • Pruning and Sparsity • Structured Pruning • Knowledge Distillation Pruning and Sparsity Pruning sets selected model parameters to zero or removes larger structures.
In this article, you will learn how information entropy and cross-entropy loss connect classical information theory to the way large language models are trained to...
Prerequisites You'll need Python 3.
Overview This post is divided into three parts; they are: • Why Activations Are Hard to Quantize • W8A8 Inference and SmoothQuant • Floating-Point Quantization Why Activations Are Hard to Quantize
In this article, you will learn how gradient boosting, random forest, neural networks, and tabular foundation models compare on accuracy, speed, and data requirements in...
In this article, you will learn how to serve a Scikit-LLM zero-shot text classification pipeline, powered by a local Ollama model, as a REST API...
Overview This post is divided into five parts; they are: • Asymmetric Affine Quantization • Post-Training Quantization with PyTorch • GPTQ and AWQ • Quantizing a Model to GGUF • Evaluating Weight-Only Quantization Note: The code examples in this post use TorchAO and require a CUDA-enabled PyTorch installation.
In this article, you will learn how to evaluate LLM applications systematically, from choosing scorers and designing reliable LLM judges to evaluating RAG systems and...
Overview This post is divided into four parts; they are: • Operator Fusion • FlashAttention-Style Kernels • Paged Attention Kernels • Fusion and CUDA Graphs with torch.