Lesson 5 of 9 · 9 min
Is there shared responsibility for LLMs?
See how vendor shared-responsibility models and the EU AI Act divide duties between model providers and the organisations that deploy them, and what that means for what you must test yourself.
Before you plan a red-team engagement you need to know whose system you are allowed and expected to test. Cloud security solved a similar problem years ago with the shared responsibility model: the provider secures the platform, the customer secures what they build on it. For LLMs the idea exists in two forms. One is vendor guidance describing who configures and monitors which controls. The other is law, mainly the EU AI Act, which splits obligations between providers of models and the parties that use them. Neither replaces your own testing. Both tell you where your side of the line is.
The vendor model: platform, application, usage
Microsoft’s AI shared responsibility model extends its cloud model to generative AI and stresses that the split varies by whether the AI is consumed as software as a service (SaaS), platform as a service (PaaS) or infrastructure as a service (IaaS). It describes three layers.
- AI platform: the infrastructure that runs the model, the training data and the configuration that changes model behaviour. A safety system at this layer filters harmful inputs and outputs in categories such as hate and jailbreaks.
- AI application: the interface users consume, from a simple front end to systems that ground prompts in extra context through a persistence layer, semantic index or plugins. Its safety system inspects the content sent to the model and the interactions with plugins and data connectors.
- AI usage: how the capability is used. Microsoft says protecting it is similar to protecting any computer system (identity and access, devices, data governance) and adds that acceptable use policies and user education need AI-specific updates.
The Cloud Security Alliance made a similar proposal in July 2023. Its post extends the cloud model to generative AI, with AI service providers on one side and AI service users on the other, and again splits it across IaaS, PaaS and SaaS tiers. That the idea appears in independent proposals is a sign of its usefulness, not of a legal standard. Microsoft states plainly that its article is illustrative governance guidance, “not intended to convey legal conclusions or to modify or contradict the terms of any agreement.”
Agents shift more to you
Microsoft’s separate model for AI agents adds three layers: orchestration, tools and actions, and memory and state. For a SaaS agent Microsoft operates the orchestrator, the model, safety systems and most tool connectors, and you own configuration, data access scoping, identity and usage. For a PaaS agent, where the platform provides runtime, model hosting and safety controls, you own the agent’s instructions, tool selection and permissions, orchestration logic, memory design, identity and authorisation. For an IaaS agent you own nearly everything. Its rule of thumb: “The more autonomy and the broader the tool and permission set that you grant an agent, the more of the responsibility matrix shifts to you.”
The same page lists responsibilities you always retain: your data, the agent’s identity and least privilege, authorisation of actions, human oversight, and acceptable use and governance.
What a PaaS split looks like in practice
The table below applies the sources’ logic to a common case, an organisation using a model through a provider’s API. It is this course’s illustration, not a contract.
| Area | Model provider or platform | Your organisation |
|---|---|---|
| Model and hosting | Hosts the model; provides platform safety controls; documents capabilities and limits | Reads the documentation and treats it as evidence about the provider’s side |
| Instructions and guardrails | Offers baseline input and output filtering | Writes the system prompt and configures guardrails; treats them as one layer, not a control boundary |
| Data, retrieval and tools | Nothing beyond the platform | Chooses what data the model can retrieve, which tools it can call and with which permissions |
| Outputs | Returns text | Validates and handles outputs before they reach other systems |
| Testing and monitoring | Evaluates the model (in the EU, for systemic-risk models this includes documented adversarial testing) | Tests the application, logs actions, responds to incidents |
| Usage | May expose some controls as configuration options | Sets acceptable use and trains users |
Some areas are genuinely shared. Prompt injection is one: a provider can work on the model’s resistance, but only you can limit what an injected instruction could do through tool permissions and data access. Data privacy is another: the provider’s terms decide how it handles data, while you decide what data goes in. Microsoft’s matrix lists customer data as the customer’s responsibility in every deployment model.
The legal form: the EU AI Act
The European Commission’s guidance says obligations for providers of general-purpose AI models began to apply on 2 August 2025. Providers must maintain technical documentation, give information and documentation to downstream providers so they understand the model’s capabilities and limitations, and put in place a copyright policy and a training-content summary. Providers of models with systemic risk must also evaluate the model, including conducting and documenting adversarial testing, report serious incidents to the AI Office, and keep the model and its infrastructure adequately secure. From 2 August 2026 the Commission will enforce the provider obligations, including through fines.
Deployers have their own duties. The Commission’s AI Act page says deployers must ensure human oversight and monitoring, and that providers and deployers report serious incidents and malfunctioning. The high-risk rules were delayed by the AI Omnibus, which entered into force on 27 July 2026: they now apply from 2 December 2027 for systems in listed high-risk areas and from 2 August 2028 for systems built into products.
What to do with this
- Test your side of the line: instructions, retrieval data, tools, permissions, output handling, logging.
- Treat the provider’s documentation as evidence about theirs, not as a guarantee about your application.
- Before adopting a model, read its documentation, its data-handling terms and how it accepts vulnerability reports.
Key takeaways
- Shared responsibility for AI exists as vendor guidance (platform, application, usage layers) and as law (EU AI Act duties for providers and deployers).
- The more autonomy and permissions you give an agent, the more responsibility shifts to you, whichever deployment model you use.
- You always retain responsibility for your data, identity and least privilege, authorisation of actions, human oversight and acceptable use.
- Provider documentation is evidence about the provider’s side only; the application around the model must be tested by its owner.
Check yourself
1. In Microsoft’s AI shared responsibility model, which layer covers plugins, data connectors and grounding of prompts?
- AI platform
- AI application — Right.
- AI usage
- Physical infrastructure
The application layer covers the interface users consume, including grounding, plugins and connectors, and its safety system inspects those interactions.
2. You build an agent on a managed platform (PaaS). Which of these stays clearly with you?
- Physical datacentre security
- Tool selection and per-tool permissions — Right.
- Base model hosting
- Nothing, the platform covers it
For PaaS agents the customer owns instructions, tool and plugin selection, tool permissions, orchestration logic, memory design and identity.
3. What do EU rules require of general-purpose AI providers toward downstream providers?
- Nothing
- Information and documentation that helps them understand the model’s capabilities and limitations — Right.
- Free licences
- Access to training data
The Commission’s guidance lists providing information and documentation to downstream AI system providers among the provider obligations applying from 2 August 2025.
4. Why is prompt injection described as a shared area?
- Providers cannot influence it at all
- A provider can work on model resistance, but only the deployer can limit what an injected instruction may do through permissions and data access — Right.
- Only users are responsible
- It is solved by the EU AI Act
Model-side resistance and application-side limits are complementary, which is why deployers cannot rely on provider filters alone.
Do it with FireAI
Put this lesson into practice on your own Mac.
- Rules: app, website, domain, IP or a range, forever or until you restart — Write a rule as precise as one address or as broad as an entire domain.
- Investigate a connection — Decide with the facts in front of you, not a vague warning.
- The World map — See where your data actually goes, not just a hostname you’d have to look up yourself.
- Requests by country and upload spikes — See at a glance where your Mac talks to, and notice at once when it suddenly sends a lot of data somewhere.
- Threat lists (opt-in) — Check your traffic against public threat data without sending it anywhere.
- Security modes: Home, Coffee shop, Paranoid, Under attack — Match FireAI’s strictness to where your Mac actually is, in one tap.
Sources
- Microsoft Learn: Artificial intelligence shared responsibility model
- Microsoft Learn: AI agent shared responsibility model
- Cloud Security Alliance: Generative AI proposed shared responsibility model (2023)
- European Commission: Guidelines on obligations for general-purpose AI providers (FAQ)
- European Commission: AI Act (regulatory framework and timeline)
Put it into practice on your Mac
Try every feature free for 17 days, no card needed.