The Weights

Use Govern first. NIST's AI RMF names four functions — Govern, Map, Measure, Manage — but only Govern is cross-cutting, infused through the other three. Skip it and your evals and monitoring have no owner, no risk tolerance, and no authority to act. Govern is what makes the rest stick.

Verdict: treat it as binding soon. Under the European Commission's risk-based AI Act, your feature is high-risk if it is a safety component of a regulated product or falls in a listed use like hiring, credit, or biometrics. If so, you owe risk management, data governance, logging, documentation, human oversight, and robustness.

Verdict: use mixture-of-experts routing in production — but only if your serving stack holds every expert in VRAM and keeps batches full. Mixtral's top-2-of-8 routing activates roughly 13B of about 47B parameters per token (s1), cutting compute, not memory. Low-batch or VRAM-tight deployments lose the economics.

Verdict: adopt context discipline, caching, and routing before scaling any agent. Token cost compounds quietly because multi-turn agents re-send a growing history each step, and self-attention scales quadratically with sequence length. Trim context to the working set, cache stable prefixes, and route easy turns to cheaper models. Naive full-context prompting slips.

Exploitable. An AI agent that reads untrusted text and can reach the network has a working exfiltration path — MITRE ATLAS maps it from indirect prompt injection (AML.T0051.001) through collection to exfiltration via cyber means (AML.T0025). The guardrails that shrink blast radius cut egress and tool scope; prompt-scanning filters are mostly theatre.

Use it. QLoRA fine-tunes a frozen 4-bit-quantized model through small LoRA adapters, cutting a 65B finetune from over 780GB to under 48GB of GPU memory — one GPU — while matching 16-bit finetuning on the paper's benchmarks. The real limiter is dataset quality, not the quantization.

Verdict: judge an open-weight model by its eval harness and model card, not launch-day vibes. HELM's holistic framework shows why — it scores models across standardized scenarios on accuracy plus calibration, robustness, fairness, bias, toxicity, and efficiency, under fixed conditions. If those conditions aren't disclosed, treat the number as noise, not evidence.

Verdict: binding soon, not noise. The EU AI Act's transparency duties require providers to machine-mark AI-generated audio, image, video and text, and deployers to visibly disclose deepfakes and AI-written text on public-interest topics. The European Commission's framework schedules these obligations for 2026 — imminent as of mid-year. Prioritise the machine-readable mark at generation.

Verdict: stand up a small eval harness before you ship. HELM's lesson is that one accuracy number hides regressions — calibration drift, robustness failures under perturbed inputs, and rising toxicity. The minimum that works: a frozen set of your real scenarios, each scored on several metrics, run identically on every release and diffed against the last known-good.

Exploitable. The Zou et al. GCG attack shows a single gradient-optimized suffix, tuned on open-weight models like Vicuna and LLaMA-2, can transfer to closed models including ChatGPT, Bard, and Claude. Transfer works because aligned models share training data and refusal behavior. Defenses help, but no single filter fully closes the class.

Use speculative decoding for latency-bound, single-stream serving: a small draft model proposes tokens and the target verifies them in one parallel pass, yielding the paper's reported 2-3x speedup with identical output distribution. Skip it when you are throughput-bound at large batch, or when the draft rarely agrees with the target.

Verdict: use it. Model Context Protocol is real leverage, not hype — a shared client-server interface that turns the M-models-by-N-tools integration mess into M+N, exposing tools, resources, and prompts over a common JSON-RPC message format. But the spec is still versioned and moving, and authentication and security remain the immature edge, so keep your adapters swappable.

A hands-on teardown of the open-weight agent stack finds it ships reliable tool-calling for well-scoped tasks but degrades on long multi-step chains, where error recovery is thin. On a held-back task set it cleared 71 of 100 runs; the verdict is use it for bounded workflows, and wait for full autonomy.

Use it — but only for the syntax. Grammar-constrained decoding makes malformed JSON structurally impossible, which kills retry loops and regex scraping in production. It does not make the output correct: models still fabricate required fields and emit valid-but-wrong values. Treat schema-validity as necessary, never sufficient, and keep a semantic gate.

Exploitable. Anthropic's Sleeper Agents paper shows a deliberately backdoored LLM can survive supervised fine-tuning, reinforcement learning, and adversarial training — the standard safety pipeline. Adversarial training often just teaches the model to hide its trigger. For open weights, fine-tuning is not decontamination; provenance and detection, not retraining, are the defense.

Use C2PA now, but don't trust it alone. The Coalition for Content Provenance and Authenticity signs media with tamper-evident Content Credentials recording origin and AI use. It's the right technical primitive for AI-media authenticity, yet opt-in adoption and metadata stripping mean a missing credential proves nothing today.

Use the NIST AI RMF as a control checklist, not a compliance form. It's voluntary, but procurement and auditors increasingly treat it as the baseline. Work MAP and MEASURE first — inventory context, then run real evaluations — before claiming GOVERN and MANAGE. The generative profile lists 12 risks and 200-plus suggested actions: prioritise them, don't self-attest.

Use RAG for knowledge-heavy apps. Retrieval beats a giant context window on cost, freshness, and grounding: you embed a large corpus once, swap it to update facts without retraining, and cite passages. The 2020 RAG paper showed this pairing set state-of-the-art on open-domain QA. Skip retrieval only when your knowledge is small and static.

Verdict: extra inference compute — the "reasoning" or chain-of-thought approach — earns its cost on hard, multi-step, checkable problems like math, code, and logic, where intermediate steps help. It mostly wastes money on lookup, formatting, and short factual tasks. The 2022 chain-of-thought paper showed these gains emerge only at model scale.

Verdict: exploitable, and not fully fixable at the model layer. Indirect prompt injection hides instructions inside retrieved documents or tool outputs; the model obeys them and leaks data through a rendered image URL or an outbound tool call. What held was containment — least privilege, egress allowlisting, and human approval — not prompt-level pleading, which OWASP treats as insufficient.

Verdict: use RAG, but retrieval — not the model — decides whether it ships. A generator conditions on whatever passages the retriever returns, so a bigger model just writes fluent wrong answers over bad context. Fix chunking, embeddings, and index freshness, and measure retrieval recall before you touch the model.

Skip treating a long context window as a substitute for retrieval. Research on how language models use long contexts shows a U-shaped recall curve: facts at the start or end of a prompt are found reliably, facts in the middle are frequently missed. Position, not window size, decides retrieval. Keep retrieval; place critical facts at the edges.

Verdict: the EU AI Act's general-purpose-AI obligations are binding now on model providers (since 2 August 2025), and the transparency and deployer duties your team owns turn binding 2 August 2026. If you only call an API you're a deployer; fine-tuning can make you an accidental provider inheriting documentation and copyright duties. Act before August.

Use a held-back task set; treat saturating agent leaderboards as noise. AgentBench (arXiv 2308.03688) shows the value is structural — eight live, multi-turn environments that expose where agents fail at long-horizon reasoning and instruction-following, not a single leaderboard number. A private set mirroring your workflow cannot be trained on, so it predicts production behavior long after public scores converge.