Teaching a docs agent to read the repo

Notes from turning kubeflow/docs-agent into an agentic RAG system — why documentation-only retrieval fails for infrastructure questions, and what it takes to index issues, code and manifests alongside prose.

SGLang, or a runtime that remembers

An architecture breakdown of the other serving engine — a radix tree over the KV cache, a scheduler that runs a step ahead of the GPU, and a grammar engine that skips tokens it can already predict.

Context engineering, after the window stopped being the problem

Million-token windows did not end context management, they changed what it is. Attention is the scarce resource now, not space — and that changes what compaction, retrieval and sub-agents are for.

vLLM from the inside

An architecture breakdown of the default open-source serving engine — the scheduler that stopped distinguishing prefill from decode, the block pool underneath PagedAttention, and where the CPU overhead went.

The harness is the product

Everyone ships the same models. What separates an agent that works from one that demos well is the loop around the model — context assembly, tool surfaces, error handling and the budget that stops it.

The two clocks

Almost everything hard about serving a large language model comes from one fact — generation runs on two different clocks, and they are not the same speed.

Kubeflow docs-agent

Agentic RAG over the whole Kubeflow surface — docs, GitHub issues, manifests and source — served as an MCP toolset behind the Kubeflow website.