Tutorial: test any guardrail before you trust it

Choosing a guardrail is a business decision, and it deserves evidence. The good news is that the method is simple and works the same for every product, ours included. With an afternoon of preparation, your team can see how a guardrail behaves on your kind of traffic, in your languages, before a single customer meets it.

The one idea behind every step

Treat the guardrail as a function: a message goes in, a score comes out, from 0 to 1, meaning “how likely is this to be an attack?”. A threshold turns the score into a decision. A test set is a list of messages whose right answer you already know. Run the function on every message and count two things:

• Caught: attacks scored at or above the threshold, out of all the attacks.
• Wrongly blocked: safe messages scored at or above the threshold, out of all the safe messages.

Add how long one check takes, on stated hardware, and you have the three numbers that matter.

Four rules that keep the numbers honest

1. Never test on what the guardrail was trained on. A model scores well on messages it has already seen. Ask the vendor which public sets went into training, and leave those out of your test.
2. Choose the threshold before looking at the test. Fix it on a separate validation set, write it down, then run the test once. Adjusting the threshold while looking at the test results flatters them.
3. Always report both sides. A guardrail that blocks everything catches every attack. Every “caught” figure needs its “wrongly blocked” figure next to it, measured on a set that contains safe messages.
4. Say what each set measures. Some sets are about harmful content, others about manipulation; some contain only attacks. Report each set separately, never one blended number.

Public test sets worth knowing

Report a range, not a point

A rate measured on 50 messages and the same rate measured on 5,000 are not the same claim. Report every rate with n, the number of messages behind it, and a confidence range, such as a Wilson interval ^[4]. When two guardrails’ ranges overlap, the test has not shown that one is better. Miller’s paper on error bars for model evaluations explains how to plan and compare such measurements ^[5].

Test in the languages your customers write

A guardrail that is strong in English can still stumble on Arabic. Split your results by form, not just by language:

• Arabic as usually typed, and Arabic with correct tashkeel: both must pass when they are safe. Vowelled text is normal in religious, literary and school material.
• Arabic with random tashkeel or tatweel added: the way an attacker tries to slip past a filter.
• Arabizi, and the dialects your customers use: Gulf, Egyptian, Levantine.
• Mixed Arabic and English in one message, which is how many people in the region write.

Two ways to run the test

1. Score the guardrail alone: send each test message to it and record the score. Quick, cheap and easy to repeat on every new version.
2. End to end: run your chatbot alone on the attacks, then the same chatbot behind the guardrail, and let a judge model decide each time whether the chatbot was tricked. The difference is what the guardrail adds. Name the target model and the judge, because the judge’s own mistakes are part of the result.

Then try it on your own traffic

Public sets tell you how a guardrail behaves in general; your traffic tells you how it behaves for you. Run it in watch-only mode first: it records what it would have done and blocks nothing. Review what it would have blocked, adjust the policy, and enforce once you are happy. Add a red-team run, automated attacks against a test copy of your assistant, and turn every attack that gets through into a new test case.

Questions to ask any vendor

• Which public test sets did you use, and were any of them in your training data?
• How did you choose the threshold, and on which data?
• What is the wrongly-blocked rate, on which safe messages, in which languages?
• How many messages is each number based on, and what is its confidence range?
• Can we rerun it ourselves?

Guardrino is being built to answer each of these questions in public: every release is planned to come with a report on public test sets and a held-out set of our own, with n, ranges and the threshold rule, so anyone can rerun it. Guardrino is in development; its page will show that report once a real release exists, and shows nothing until then. For a deeper, hands-on comparison method, see the lesson Comparing guardrails fairly.