Production engines like vLLM drive the forward pass themselves, shard module outputs across GPUs, and run decode through CUDA graphs that Python hooks cannot reach. This post describes how NNsight’s vLLM integration gets intervention code into the engine, makes distributed tensors look local, and keeps NNsight semantics under continuous batching.

This post is published on the NNsight blog: read it on nnsight.net.