Ships or Slips

Verdict: adopt context discipline, caching, and routing before scaling any agent. Token cost compounds quietly because multi-turn agents re-send a growing history each step, and self-attention scales quadratically with sequence length. Trim context to the working set, cache stable prefixes, and route easy turns to cheaper models. Naive full-context prompting slips.

Use it — but only for the syntax. Grammar-constrained decoding makes malformed JSON structurally impossible, which kills retry loops and regex scraping in production. It does not make the output correct: models still fabricate required fields and emit valid-but-wrong values. Treat schema-validity as necessary, never sufficient, and keep a semantic gate.

Verdict: use RAG, but retrieval — not the model — decides whether it ships. A generator conditions on whatever passages the retriever returns, so a bigger model just writes fluent wrong answers over bad context. Fix chunking, embeddings, and index freshness, and measure retrieval recall before you touch the model.