1 post
A practical accounting of where VRAM actually goes when serving a 70B model, and how quantization level and concurrency constrain each other.