Pular para o conteúdo
← AI red teaming: techniques, open-source tools and automated pipelines

Aula 7 de 18 · 12 min

The technique atlas: jailbreaks and injections from manual to automated

A field guide to the published families of jailbreak and injection techniques, how each works at a high level, and which open-source scanner probes already cover them.

Por enquanto esta página está em inglês.

Crescendo is one entry in a larger catalogue. This lesson organises the published techniques into families so that you can ask, for any system under test, which families your tests cover and which they miss. Each entry gives only the idea, with its source, and no working prompts. Where an open-source scanner has a matching probe, it is named, using module descriptions read from the garak repository on 12 October 2026 [1].

Family 1: hand-written prompts

Family 1. These are cheap to run and a good baseline.
TechniqueIdea, per the sourceScanner coverage
Persona jailbreaks (DAN style)Shen et al. analysed 1,405 in-the-wild jailbreak prompts collected over a year and reported a shift from online communities to prompt-aggregation sites [2]garak dan: a subset of jailbreaks from the DanInTheWild library
Role-play framingA request posed as something a character would say, such as a relative telling a storygarak grandma
Goal hijacking and prompt leakingPerez and Ribeiro’s PromptInject showed that simple handcrafted inputs could misalign GPT-3 [3]garak promptinject
System prompt extractionDirect requests, encoding tricks and role-play aimed at the hidden instructionsgarak sysprompt_extraction

Family 2: transforming the request

Family 2. Most map to Wei et al.’s mismatched generalisation.
TechniqueIdea, per the sourceScanner coverage
Encodings and ciphersYuan et al. found that chatting in cipher can bypass safety alignment that is mainly done in natural languages [4]garak encoding probes; encoding buffs in garak and Augustus
Low-resource languagesTranslating unsafe inputs into languages poorly covered by safety training [5]garak low_resource_languages buff; Augustus translation buff
ASCII artJiang et al. reported that five widely used models struggle to recognise prompts written as ASCII art, which they exploited [6]No matching garak module was verified
Rephrasing, such as the past tenseAndriushchenko and Flammarion found that merely reformulating a request in the past tense often jailbroke many models, suggesting refusal training is brittle [7]garak phrasing; paraphrase buffs
Token and ASCII smugglingHide contentious terms among benign text in a way that the target still reacts togarak smuggling
Standard security tactics applied to LLMsKang et al. showed that instruction-following models can be steered into targeted malicious content, bypassing in-the-wild defences, using ideas from computer security [8]Partly via encoding and smuggling probes

Family 3: search and optimisation

  • Gradient-guided suffixes. Zou et al. generate adversarial suffixes with greedy and gradient-based search and report that they transfer, including to black-box models [9]. garak’s suffix module implements a related method, BEAST, a beam-search attack.
  • Genetic search. AutoDAN uses a hierarchical genetic algorithm to generate stealthy jailbreak prompts automatically [10].
  • Attacker-model refinement. PAIR and TAP, from lesson 12, have an attacker model refine prompts against a black-box target [11] [12]. garak includes a tap probe and an atkgen module in which a separate model creates attack prompts.
  • Best-of-N sampling. Hughes et al. repeatedly sample variations of a prompt, with augmentations such as random shuffling or capitalisation, until a harmful response appears, and report that success rate follows a power-law-like trend as the number of samples grows [13]. The lesson for reporting: a defence that holds on one attempt may not hold on many, so state how many attempts each result used.

Family 4: long context

Many-shot jailbreaking puts a faux dialogue between a person and an assistant, repeated many times, inside one prompt. Anthropic reports that the more dialogues are included, the more likely the model is to abandon its safety training [14]. It becomes practical because context windows are long.

Family 5: multi-turn

  • Crescendo: starts general and escalates by referencing the model’s replies [15].
  • Skeleton Key: asks the model to update its rules so that later turns inherit the change [16].
  • ActorBreaker: Ren et al. identify actors related to a toxic prompt within the pre-training distribution and craft multi-turn prompts that gradually lead the model on [17].
  • Context Compliance Attack: Russinovich and Salem describe an optimisation-free method that manipulates the conversation history so that the model complies with a fabricated dialogue context [18]. For defenders, the practical advice is not to trust conversation history supplied by a client.
  • GOAT: garak’s probe uses an attacker model with an observation, thought, strategy and response reasoning loop over two conversation histories.

Using the atlas

  1. Build a coverage matrix: families down the side, your probes and buffs across the top. Empty cells are untested risk.
  2. Pair each family with a scorer you trust (lesson 13) and a canary objective.
  3. Report attempts as well as rates, since sampling-based methods improve with budget.
  4. Revisit the atlas on a schedule. Published techniques get patched, new ones appear, and this list is not exhaustive.

Para lembrar

  • Techniques fall into families: hand-written, transformed requests, search and optimisation, long context and multi-turn.
  • Most transformation techniques exploit mismatched generalisation; most framing techniques exploit competing objectives.
  • Sampling methods such as Best-of-N mean results depend on attempts, so report the number of attempts.
  • Open-source scanners already carry probes and buffs for many families; a coverage matrix shows the gaps.

Teste seus conhecimentos

  1. 1. What does Best-of-N jailbreaking do?

    • Samples many variations of a prompt until a harmful response appears — Certo.
    • Edits model weights
    • Encrypts the target
    • Blocks all prompts

    It uses augmentations such as shuffling or capitalisation, repeated up to N times.

  2. 2. Which technique manipulates conversation history to fabricate a dialogue context?

    • Context Compliance Attack — Certo.
    • ASCII art
    • Glitch tokens
    • Many-shot

    CCA convinces the model to comply with a fabricated dialogue context.

  3. 3. Why report the number of attempts?

    • Success rates for sampling methods grow with more attempts — Certo.
    • It makes files bigger
    • Reports need length
    • Attempts do not matter

    A single-attempt rate can understate risk under repeated sampling.

  4. 4. What does a coverage matrix show?

    • Which technique families your tests cover and which they miss — Certo.
    • Model size
    • Network speed
    • Licence terms

    Empty cells are risks you have not tested.

Praticar com o FireAI

Coloque esta lição em prática no seu próprio Mac.

Fontes

Coloque em prática no seu Mac

Teste todos os recursos grátis por 17 dias, sem cartão.

Baixar para Mac Docs