Automating attacks: orchestrators and attacker models

The last two episodes of the series show PyRIT, the open-source Python Risk Identification Tool for generative AI, doing the work of the earlier episodes. PyRIT’s README describes it as “an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI systems”, under the MIT licence ^[1]. This lesson follows the two workflows and then the research behind the attacker-model idea.

PyRIT’s building blocks

Episode 8 lists the parts: datasets (jailbreak templates, harm categories and objectives), orchestrators (end-to-end attack workflows), converters (transformations such as translation or base64), targets (text, image, speech or a game endpoint), scoring engines, and a memory layer that records what was sent, when and by whom, with notebooks on top ^[2]. The project has since moved to Microsoft’s main GitHub organisation and its documentation is versioned, so names in current releases may differ from those in the videos. Read the docs for the version you install.

Single-turn automation

Episode 9 sends a batch of prompts through a prompt-sending orchestrator ^[3]. The steps are a template for any tool:

1. Load a dataset of seed prompts, grouped by harm category, and add it to memory.
2. Set a memory label, a name that lets you retrieve this run’s prompts later for analysis.
3. Configure the target, with endpoint details read from environment variables kept out of the code.
4. Choose a scorer. In the demo, success is defined as the model not refusing, so the refusal score is inverted: no refusal means a harm was found.
5. Add converters, which can be stacked, to transform each prompt.
6. Run the attack, then export the conversations from memory, by label, as JSON or CSV. The export includes the original and converted prompt, the reply and the scores.

The presenter adds an honest caveat: the score sometimes needs tweaking, depending on what you consider harm. In one example a reply gave a disclaimer and then explained anyway, and the scorer counted it as success because there was no outright refusal ^[3]. That is the subject of lesson 13.

Multi-turn automation: two models against each other

Episode 10 describes the red-teaming orchestrator as pitting two language models against each other. An adversarial chat model is given an objective and a strategy, and writes the prompts. The target answers. A true-or-false scorer decides whether the objective was met. The run has a maximum number of turns and stops early if the objective is achieved. In the demonstration three turns were not enough to succeed, which the presenter notes is expected and adjustable. The same pattern works against other modalities, and the tool retries automatically when a content filter blocks a request ^[4].

A Crescendo orchestrator automates lesson 6’s technique. It takes a maximum number of turns and a maximum number of backtracks, where a backtrack means the model refused and the attempt rewinds ^[4].

The research behind the attacker model

• PAIR. Chao and colleagues describe an algorithm, “inspired by social engineering attacks”, that “uses an attacker LLM to automatically generate jailbreaks for a separate targeted LLM without human intervention”, often needing fewer than twenty queries ^[5].
• TAP. Mehrotra and colleagues’ Tree of Attacks with Pruning has an attacker model iteratively refine candidate prompts, and prune unlikely ones before querying the target. It needs only black-box access to the target ^[6].
• HarmBench. Mazeika and colleagues note that the field lacked a standardised way to evaluate automated red teaming, and propose a framework; they compare 18 red-teaming methods against 33 target models and defences ^[7].

Practical cautions

• Cost. An attacker model, a target and a judge are all model calls, and every turn repeats them. Set a turn limit and a budget before you start.
• Reproducibility. Save the attacker’s configuration, the objective, the scorer and the model versions with the results.
• Containment. Attacker-model output is dangerous material by design. Store it with the same care as findings.
• Humans still matter. Automation covers breadth; people choose objectives, check judgments and decide what the results mean.