Organisations that build on a hosted language model receive a safety-trained model and a platform from the vendor, and remain responsible for the application around it. Microsoft, the Cloud Security Alliance and the EU AI Act each divide the work between the model provider and the party that deploys it, using different vocabulary and different legal weight. This note compares the three and sets out a practical division for the common case of a model consumed through an API. It is a companion to How Anthropic red-teams Claude, and what it leaves to you, which examines the provider’s side of the line in one published case.
Background: the cloud model, extended to AI
The cloud shared-responsibility model assigns each control to the provider or the customer depending on the service type: software as a service (SaaS), platform as a service (PaaS) or infrastructure as a service (IaaS). Both Microsoft and the Cloud Security Alliance (CSA) extend that idea to generative AI. The extension changes the vocabulary more than the logic: the further down the stack a customer builds, the more controls the customer owns.
Vendor models
Microsoft: platform, application and usage
Microsoft’s AI shared responsibility model describes an AI-enabled application as three layers: the AI platform, the AI application and AI usage. The platform layer provides the model through APIs and includes a safety system that filters harmful inputs and outputs. The application layer is the interface the user consumes, with grounding, plugins and data connectors, and it needs its own application safety system. The usage layer covers how people consume the capability, and Microsoft points to identity and access controls, acceptable use policies and user education. [1]
How much of each layer the customer owns depends on the deployment type. The page states that responsibilities vary between SaaS, PaaS and IaaS, and it recommends starting with SaaS offerings such as Copilot, moving to PaaS services such as Azure OpenAI Service only when off-the-shelf capabilities do not fit, and reserving custom model building for organisations with deep expertise. The page adds that the guidance is illustrative, is used in a governance sense and does not modify any agreement with Microsoft. [1]
Microsoft: the agent extension
A second Microsoft page covers agents, which it distinguishes from a plain model because an agent acts without a human approving each step, plans and loops, holds memory, has its own identity and can compose with other agents. It adds three layers: agent orchestration, tools and actions, and memory and state. In its responsibility matrix, the customer keeps the agent’s instructions and scope, per-tool permissions and human approval for high-impact actions in the PaaS case, while runtime and orchestrator platform belong to Microsoft. [2]
The page lists responsibilities a customer always retains: data, identity and least privilege, authorization of actions, human oversight, and acceptable use and governance. It also states that autonomy never reduces accountability. [2]
Cloud Security Alliance: a 2023 proposal
In a blog post dated 28 July 2023, a CSA fellow proposed a model with three parties: the AI service provider, the AI service user that builds an application, and the enterprise or end user of that application. Under the proposal, an IaaS provider supplies infrastructure and foundation models while the user handles training, data validation, prompt filtering and application security; in the PaaS case the user supplies context and application security; in the SaaS case the user manages grounding, prompt filtering and IP protection. The post presents this as a proposal that separates duties, not as a standard. [3]
Law: the EU AI Act splits obligations by role
The EU AI Act assigns duties by role. According to the European Commission’s guidelines FAQ, a provider of a general-purpose AI model is the entity that develops the model, or has it developed, and places it on the market under its own name. Provider obligations include technical documentation, documentation for downstream developers about capabilities and limitations, a copyright compliance policy and a public summary of training content. These obligations applied from 2 August 2025. Models presumed to carry systemic risk, at or above 10^25 floating point operations of training compute, face further duties including adversarial testing and cybersecurity safeguards. [4]
The same FAQ gives 2 August 2026 as the date from which enforcement, including fines, applies to these providers, and 2 August 2027 as the deadline for models placed on the market before 2 August 2025. It also states that a downstream developer who modifies a model becomes a provider only in exceptional circumstances, such as using more than one-third of the original training compute, so that most fine-tuning does not shift the provider role. [4]
Deployers, meaning organisations that use AI systems, carry a separate set. A law-firm analysis dated 24 July 2026 lists disclosure of deepfakes and notice to individuals about emotion recognition and biometric categorisation as deployer duties from 2 August 2026. For high-risk systems it lists competent human oversight, notifying individuals that AI is used and consulting workers. [6] The Commission’s timeline page adds that deployers must ensure human oversight and monitoring and report serious incidents once systems are on the market. [5]
The Digital Omnibus postponement
The high-risk deadlines have moved. The Commission’s timeline page lists 2 December 2027 for high-risk systems in sensitive areas such as biometrics, employment and law enforcement, and 2 August 2028 for high-risk systems embedded in regulated products, and it credits the Digital Omnibus on AI, which it describes as entering into force in July 2026. [5] The Norton Rose Fulbright analysis reports the same two dates and states that the Omnibus has been published in the EU statute book. [6] Two independent sources therefore agree that the postponement was adopted and not merely proposed. Provider duties for general-purpose models and the transparency duties above are not moved by these two dates.
Dividing the work when a model is used through an API
For the PaaS case, in which an application calls a hosted model, the sources support the following division. The Microsoft rows follow the platform and application layers described above; rows on frontier-risk testing, system documentation and disclosure channels reflect the provider duties in the EU guidelines and practices that model vendors publish. The table is a synthesis by FireAI, not a text from any of the sources.
| Area | Provider side | Your side |
|---|---|---|
| Model behaviour | Safety training and alignment of the model | System prompt and application guardrails |
| Risk testing | Frontier-risk testing of the model itself | Testing your application, including its prompts, data and tools |
| Platform | Platform security and baseline input and output filters | Application safety checks on content, plugins and connectors |
| Documentation | System card and documentation for downstream developers | Reading it, and recording which model and version you deployed |
| Data and tools | None beyond the API contract | RAG data, tools, agents and their permissions |
| Outputs | Returns generated text | Handling output downstream: validation, escaping, human approval |
| Operations | Vulnerability disclosure channel for the model | Logging, monitoring and incident response for the application |
| People | Usage policy terms | Your own usage policy and user training |
Two areas are shared in the strict sense. The first is prompt injection: the provider trains the model to resist injected instructions, and the deployer limits what an injected instruction can accomplish through tool permissions and data access. Microsoft’s agent page describes the same split, telling customers to treat retrieved content, tool outputs and other agents’ messages as untrusted and to gate high-impact actions. [2] The second is data privacy: the provider sets retention and training terms, and the deployer decides what data enters a prompt or a retrieval index in the first place.
Recommendations
- Identify the deployment type first (SaaS, PaaS or self-hosted) and write down which rows of the division above the organisation owns.
- Test the deployer’s side directly: the system prompt, retrieval data, tool permissions and output handling. Vendor testing of the model does not cover them.
- Before adopting a model, read the provider’s system card, its data retention and training terms, and locate its vulnerability disclosure channel.
- Limit what an injected instruction can do: least-privilege tools, scoped data access and human approval for irreversible actions.
- Decide what data may enter prompts, and keep logs sufficient to reconstruct an incident.
- For learning material on frameworks and red teaming, see the FireAI University course on AI security frameworks and red teaming.
Relevance to FireAI
A local app or agent that calls a model API is an app on the Mac, and its destinations are visible to FireAI. Activity lists which apps went online, and Rules let a user allow or block an app or a specific destination. That is one control on the deployer’s side, on a single Mac. FireAI does not inspect prompts, judge model output, detect prompt injection or assess whether a provider meets any legal obligation.
Limitations
- The Microsoft pages are vendor guidance for Azure products, describe themselves as illustrative and do not change contract terms. The CSA text is a 2023 proposal.
- This note is not legal advice. Whether an organisation is a provider, a deployer or a high-risk operator under the EU AI Act depends on facts that this note does not assess, and national authorities may interpret the Act differently.
- The Digital Omnibus dates come from the Commission’s timeline page and one law-firm analysis. The text of the regulation itself was not reviewed for this note.
- The API table is a synthesis, and individual providers place some rows differently in their terms.
How FireAI and HisnLabs fit in
Your side of the line includes what leaves the Mac. FireAI shows it and lets you decide.
FireAI is HisnLabs’ own product: an on-device AI firewall for Mac. It shows every connection your apps make, in plain language, and lets you decide what leaves your Mac — its AI runs locally, so your traffic is never sent to us or anyone else. HisnLabs’ security research team is the group that keeps that decision-making accurate: cataloguing which domains are ordinary telemetry versus a real product, tracking the country and network behind a connection, and training the on-device model (its FireAI Pilot feature) on real traffic patterns, all without any of it leaving your Mac.
You can read the technical decisions behind it, or try FireAI for 17 days, at FireAI, by HisnLabs.