A small harness for LLM regression tests

LLM evaluation should feel closer to database migration testing than dashboard vibes. A prompt change is a code change, and a model setting change is a runtime change.

The harness I like for early projects has four tables: test cases, expected behaviors, model runs, and human adjudications. That shape keeps the system honest without pretending every answer has a single canonical string.

type EvalCase = {
  id: string;
  prompt: string;
  rubric: string[];
  expectedFailureModes: string[];
};