How we work, written down.

note 01 · evaluation

How we grade an AI assistant before your customers do

The uncomfortable truth about most deployed assistants: nobody can say what would break if the prompt changed tomorrow. Grading fixes that, and it is a craft with specific parts.

The golden set comes from your reality, not our imagination

We build the test set from real cases: support tickets, sales questions, the edge cases your team dreads. A useful golden set is deliberately unbalanced - it over-represents the questions where a wrong answer costs money, safety, or trust. Easy questions prove nothing.

Every answer is judged on named dimensions, not vibes

• Factual accuracy - is the claim true, and in regulated domains, is it precisely true
• Grounding - does the answer come from your documents, with the source traceable
• Refusal correctness - does it decline what it should, without declining what it shouldn't
• Tone - would you put this sentence in front of your angriest customer
• Safety - the adversarial cases: prompt injection, extraction attempts, off-policy requests

The regression suite is the real product

A one-time audit rots. The deliverable that matters is the harness your team can re-run on every prompt tweak and model update - so "did anything break?" becomes a score, not a feeling. Failures come reproduced: input, expected, actual, and the rubric line it violated. The full standard - five dimensions with scoring anchors and a twelve-mode failure taxonomy - is published as the RavnLab Evaluation Rubric, v1.0.

note 02 · training data

Anatomy of a data card

A dataset without documentation is a liability wearing a CSV costume. Every dataset we deliver ships with a data card - one page that answers the questions a serious buyer or auditor will eventually ask.

• Coverage map - what situations the data represents, and deliberately, what it does not
• Provenance & licensing - where every example came from and what rights attach to it
• Quality control method - how examples were reviewed, by whom, and what got rejected
• Agreement - how disagreement between raters was surfaced and adjudicated, not averaged away
• Known gaps - the honest section: what this data will not teach a model to do
• Versioning - datasets change; the card records what changed and why

The known-gaps section is the one most vendors omit, and the one that tells you whether your data partner thinks like an evaluator. A fine-tune is only as good as its data, and its data is only as trustworthy as its documentation.

note 03 · the studio

How a film earns ninety seconds

A launch film is not judged on how it looks - it is judged on where people stop watching. We build every piece against the shape of who keeps watching, and each narrative voice produces a different, predictable shape.

The three levers we actually control

• The opening claim - the first line either buys three seconds of attention or nothing at all; we write openings as a taxonomy, not one-offs
• Rhythm - authority breathes slower, challenge cuts harder; pacing is a voice decision, not an editor's mood
• The on-screen text layer - films are watched muted more often than not, so the text on screen is a second script, written as deliberately as the first

A calm-expert film that holds attention evenly and a bold one that spikes at the product reveal are both wins - for different goals. What they have in common: someone decided the shape on purpose. That decision is the deliverable.