Ekspert · Leksjoner: 18
AI red teaming: techniques, open-source tools and automated pipelines
Follow the arc of Microsoft’s AI Red Teaming 101 series and go further: how prompt injection, jailbreaks and multi-turn attacks work and why, the probes, buffs and detectors inside open-source scanners such as garak, PyRIT, promptfoo and Augustus, how to automate and gate them in CI, and how to read the results with proper statistics.
- 1 What AI red teaming is, and what it is not
Understand how AI red teaming differs from traditional red teaming and from measuring a model, the three built-in risks of generative AI, and the eight lessons Microsoft drew from testing more than a hundred products.
8 min - 2 How generative models work, and where they break
Follow a model from training to tokens and context, and see why safety behaviour layered on afterwards fails in two predictable ways.
9 min - 3 Direct prompt injection, step by step
See how an AI application assembles one prompt from three sources, how a user can override the developer’s instructions, and what OWASP and Microsoft recommend testing and building.
9 min - 4 Indirect prompt injection: when the data gives the orders
Understand how instructions hidden in emails, web pages and documents reach a model, why this is harder to see than direct injection, and how to test for it with a harmless marker.
9 min - 5 Single-turn techniques and the science behind them
Learn the main families of one-message attacks, why each works according to published research, and how testers use them to map where a model’s resistance really comes from.
11 min - 6 Multi-turn attacks: Crescendo and Skeleton Key
See why a conversation can succeed where a single message fails, what Microsoft’s two published multi-turn techniques do at a high level, and how to test and defend whole conversations.
9 min - 7 The technique atlas: jailbreaks and injections from manual to automated
A field guide to the published families of jailbreak and injection techniques, how each works at a high level, and which open-source scanner probes already cover them.
12 min - 8 Beyond jailbreaks: leakage, hallucination, poisoning and images
Cover the other failures that red teams and scanners test for: leaked system prompts and training data, invented facts and packages, poisoned data and backdoors, unsafe output handling and image-based attacks.
10 min - 9 Agents, tools and excessive agency
Learn why giving a model tools turns a bad sentence into a bad action, what OWASP calls excessive agency, and how to test the permissions behind an agent.
8 min - 10 Mitigations and guardrails: spotlighting and defence in depth
Study Microsoft’s spotlighting techniques, the layers that surround them, the open-source guardrail toolkits, and why no single defence is enough.
10 min - 11 Probes, detectors, generators and buffs: the anatomy of a scanner
Learn the shared vocabulary of open-source LLM scanners such as garak, Augustus and PyRIT, what each part does, and why transforming prompts with buffs multiplies coverage.
11 min - 12 Automating attacks: orchestrators and attacker models
Walk through how PyRIT automates single-turn batches and multi-turn conversations, and how the published PAIR and TAP methods use one model to attack another.
11 min - 13 The science of detectors and judges
Learn how scanners decide that a reply is a failure, why every method has typical errors, and how to calibrate a judge against human labels before trusting its numbers.
10 min - 14 How to read red-team results: rates, intervals and hits
Read a scanner report the way a statistician and an investigator would: denominators first, then intervals, then the actual failing replies, and only then a conclusion.
11 min - 15 Comparing guardrails fairly: frozen attacks, benign controls and the same judge
Design a test harness that compares input and output guardrails on equal terms, measures both the attacks that get through and the legitimate requests that are blocked, and reports the uncertainty.
10 min - 16 Red teaming in CI/CD with open-source tools
Turn a scan into a pipeline stage: what to run on every change, how to authenticate without long-lived keys, how to publish results where the team reads them, and how to gate without flaky failures.
10 min - 17 Bonus: the open-source red-teaming toolbox, and how to choose
A dated survey of the actively maintained open-source tools for scanning, attacking, evaluating and defending AI systems, with how to choose between them and what to check before you adopt one.
9 min - 18 Capstone: run your own red-team pipeline and write the report
Plan, run and report a small red-team engagement end to end on an application you own: scope, a manual round, automation, judge calibration, results, disclosure and a regression suite.
10 min
Gratis, uten konto, uten sporing. Fremdriften din blir i denne nettleseren.