Evaluation articles

Offline vs. online evaluation for generative AI

Use offline evaluation for reproducible release decisions and online evaluation for real production behavior, then connect both in one feedback loop.

LLM-as-a-judge: Design and validate model-based evaluators

Use LLMs as scalable evaluators without treating them as ground truth: design clear rubrics, calibrate against humans, and monitor bias and drift.

Turn production traces into regression tests

Convert representative production failures into sanitized, reproducible evaluation cases that protect future LLM and agent releases.

How to evaluate RAG systems

Evaluate retrieval and generation separately and end to end with representative datasets, groundedness checks, retrieval metrics, and production feedback.

How to evaluate AI agents

A practical framework for evaluating AI agent outcomes, trajectories, tool use, safety, latency, and cost before and after release.