Bloom filters, and when a probabilistic set is the right call
Trade a little certainty for a lot of memory — the math is friendlier than it looks.
Trade a little certainty for a lot of memory — the math is friendlier than it looks.
The cache that makes autoregressive decoding fast also makes it the thing that runs out of memory first.
Why my OOM kills moved around after switching to the unified hierarchy, and the three knobs that matter.