Topic · 5 stories
Designing LLM features that hold up in production: RAG pipelines, agent orchestration, structured outputs, function calling, evals and reliability.
OpenAI’s Decisions API returns probabilities instead of a generated JSON label, and bills input tokens only. When that number changes how you route, flag and abstain, and when Structured Outputs or function calling is still the right call.
Sandboxing each agent doesn’t isolate them from each other. Injection spreads through the memory, documents and messages they share. Where it travels, why per-agent safety doesn’t compose, and the design that contains it.
A system prompt lowers how often an agent misbehaves; it can’t stop a prompt injection. Where controls actually hold: authorization in every tool, limits once untrusted content is in context, and checks before consequential actions.
A model upgrade changes behaviour your prompts depend on, and the failures return HTTP 200: a missing field, a tool that isn’t called, a refusal, a bill that grows. How to build a small regression suite from your own traffic and decide from the diff.
A single passing run shows an LLM agent can do a task, not that it will every time. How to run repeated trials, estimate pass@k and pass^k per task, separate grader and infrastructure noise, and pick the number that gates a release.
