Tutorial · Available

Add Guardrino to LiteLLM

One plugin, one config entry, and every model behind LiteLLM is checked before and after the call.

Status: Available. This integration is built and tested. Pilot companies receive the Guardrino package from HisnLabs.

Developer reference · Install guide · AI platforms · Open model card · OpenAI-compatible proxy · Streaming · LiteLLM · MCP security proxy · Tutorial: OpenAI-compatible app · Tutorial: MCP tools

Try it today Guardrino-t7 Open, an earlier version of our classifier, is public on Hugging Face under Apache-2.0. The demo runs in your browser: what you type stays on your device. Live demo · Model card · Same demo on Hugging Face

Step 1 · Install the plugin

The plugin is one Python file, guardrino_guardrail.py, from the Guardrino package. Install LiteLLM’s proxy extras, then put the file next to your config.yaml and start LiteLLM.

pip install "litellm[proxy]" httpx
# put guardrino_guardrail.py (from the Guardrino package) next to config.yaml, then:
litellm --config config.yaml

Step 2 · Declare the guardrail

Add an entry under guardrails: in your LiteLLM config.yaml. It runs before the call and after it.

guardrails:
  - guardrail_name: guardrino-input
    litellm_params:
      guardrail: guardrino_guardrail.GuardrinoGuardrail
      mode: pre_call
      default_on: true
      api_base: os.environ/GUARDRINO_URL
      tenant: acme
      app: support-bot
      fail_open: true
  - guardrail_name: guardrino-output
    litellm_params:
      guardrail: guardrino_guardrail.GuardrinoGuardrail
      mode: post_call
      default_on: true
      api_base: os.environ/GUARDRINO_URL
      tenant: acme
      app: support-bot
      fail_open: true

Use one entry for pre_call (the request) and one for post_call (the reply). A blocked message returns HTTP 400. post_call does not cover streamed replies: for those, use the proxy instead.

Step 3 · Restart and test

Restart LiteLLM the way you normally run it, then send one normal request and one your policy refuses, through any model you have configured.

Prefer not to run code inside LiteLLM? Put Guardrino’s proxy in front instead, with LiteLLM as its upstream.

[proxy]
upstream = "http://litellm.internal:4000/v1"   # LiteLLM is Guardrino's upstream
api_key = "LITELLM_KEY"                        # optional

Request early access