Integration · Available

Guardrino with LiteLLM

Cover every model behind LiteLLM with one set of checks, either inside LiteLLM or in front of it.

Status: Available. This integration is built and tested. Guardrino is delivered to pilot companies by HisnLabs, and some settings may still change before general release.

Developer reference · Install guide · AI platforms · Open model card · OpenAI-compatible proxy · Streaming · MCP security proxy · Tutorial: OpenAI-compatible app · Tutorial: LiteLLM · Tutorial: MCP tools

Try it today Guardrino-t7 Open, an earlier version of our classifier, is public on Hugging Face under Apache-2.0. The demo runs in your browser: what you type stays on your device. Live demo · Model card · Same demo on Hugging Face

Two ways

As a LiteLLM custom guardrail. The Guardrino plugin runs inside LiteLLM, before the call (pre_call) and after it (post_call).
In front of LiteLLM. Apps call Guardrino’s OpenAI-compatible proxy, and LiteLLM is its upstream.

As a custom guardrail

The plugin is one Python file, guardrino_guardrail.py, from the Guardrino package. Put it next to LiteLLM’s config.yaml. Each setting (api_base, tenant, app, fail_open, timeout) can also come from the environment: GUARDRINO_URL, GUARDRINO_TENANT, GUARDRINO_APP, GUARDRINO_FAIL_OPEN, GUARDRINO_TIMEOUT.

Use one entry for pre_call and one for post_call. A blocked message returns HTTP 400. If Guardrino cannot be reached, calls are allowed by default (fail_open: true) or refused with HTTP 503. post_call does not cover streamed replies: for those, use the proxy (below).

pip install "litellm[proxy]" httpx
# put guardrino_guardrail.py (from the Guardrino package) next to config.yaml, then:
litellm --config config.yaml

Then declare it under guardrails: in LiteLLM’s config.yaml.

guardrails:
  - guardrail_name: guardrino-input
    litellm_params:
      guardrail: guardrino_guardrail.GuardrinoGuardrail
      mode: pre_call
      default_on: true
      api_base: os.environ/GUARDRINO_URL
      tenant: acme
      app: support-bot
      fail_open: true
  - guardrail_name: guardrino-output
    litellm_params:
      guardrail: guardrino_guardrail.GuardrinoGuardrail
      mode: post_call
      default_on: true
      api_base: os.environ/GUARDRINO_URL
      tenant: acme
      app: support-bot
      fail_open: true

Every model LiteLLM routes to goes through the same checks, because the guardrail sits on the shared path.

With LiteLLM as the upstream

Here Guardrino checks and forwards, and LiteLLM routes. Apps change only their base URL, to Guardrino.

[proxy]
upstream = "http://litellm.internal:4000/v1"   # LiteLLM is Guardrino's upstream
api_key = "LITELLM_KEY"                        # optional

The OpenAI-compatible proxy

Request early access