LLM math demystified for practitioners
LLMs reduce to repeated multiply-add at scale, not magic math Devoxx UKTL;DW
- AI at its core uses only multiplication, addition, and activation functions—simple math repeated at massive scale across billions of parameters.
- Vectors are numbers arranged vertically; matrices are arrays of arrays. Neural networks transform input vectors through hidden layers via matrix multiplication and addition.
- Floating-point numbers store sign, exponent, and fraction bits (32-bit format) to represent decimals in binary—e.g., pi requires 50 terabytes to store 100 trillion known digits.
- LLMs output raw logits from hidden layers, converted to probability distributions via softmax function to predict next token with each input.
- Gradient descent trains models by computing mean squared error (loss), measuring how weight changes affect loss, then iteratively adjusting weights to minimize error.
- GPT-3 has 175 billion parameters requiring ~700GB memory; quantization compresses 32-bit floats to 8-bit integers (Q8) for 75% size reduction with minimal accuracy loss.
- GPUs parallelize vector operations efficiently because graphics processing optimizes X, Y, Z axis computation; TPUs handle tensor (multi-dimensional array) operations via matrix multiplication.
- FLOPs (floating-point operations) ~2× parameter count; GPT-3 with 50-token input/100-token output requires ~52 trillion operations per request—explains GPU necessity.
- Carbon footprint research accounting for output quality (not just energy) shows selective model choice and human-AI comparison matters; most published studies ignore quality metrics.
- Binary underpins all computing; AI mythology (memory, personality, consciousness) obscures reality that LLMs compute probabilities through repetition of elementary arithmetic operations.
TL;DW
- AI at its core uses only multiplication, addition, and activation functions—simple math repeated at massive scale across billions of parameters.
- Vectors are numbers arranged vertically; matrices are arrays of arrays. Neural networks transform input vectors through hidden layers via matrix multiplication and addition.
- Floating-point numbers store sign, exponent, and fraction bits (32-bit format) to represent decimals in binary—e.g., pi requires 50 terabytes to store 100 trillion known digits.
- LLMs output raw logits from hidden layers, converted to probability distributions via softmax function to predict next token with each input.
- Gradient descent trains models by computing mean squared error (loss), measuring how weight changes affect loss, then iteratively adjusting weights to minimize error.
- GPT-3 has 175 billion parameters requiring ~700GB memory; quantization compresses 32-bit floats to 8-bit integers (Q8) for 75% size reduction with minimal accuracy loss.
- GPUs parallelize vector operations efficiently because graphics processing optimizes X, Y, Z axis computation; TPUs handle tensor (multi-dimensional array) operations via matrix multiplication.
- FLOPs (floating-point operations) ~2× parameter count; GPT-3 with 50-token input/100-token output requires ~52 trillion operations per request—explains GPU necessity.
- Carbon footprint research accounting for output quality (not just energy) shows selective model choice and human-AI comparison matters; most published studies ignore quality metrics.
- Binary underpins all computing; AI mythology (memory, personality, consciousness) obscures reality that LLMs compute probabilities through repetition of elementary arithmetic operations.
Walks through how transformers work from binary storage through weighted sums, ReLU activations, and softmax probability outputs. Covers gradient descent, MSE loss, and why GPT-3's 175B parameters demand 700GB RAM—then explains how int8 quantization and GPU parallelism make deployment practical.