Crucible

Run large models on one GPU

I use Crucible to explore where a single RTX 3090 Ti holds up with larger models and longer contexts. I compare weight quantisation, GPU offload and KV cache placement to see where a workload fits in VRAM and what happens when it needs system memory.

Measure the whole request

I track model load time, time to first token, prompt processing and decode separately. I also compare cold requests with requests that reuse a saved prefix, since cache reuse can change which configuration works best for an interactive session.

See published model builds and a decode benchmark on Localweights, or read the Qwen GGUF benchmarking note.