Guardrails

Checks that run before and after each request to a model or tool, and block, mask, or log what they find.

A guardrail checks a request or a response, and blocks it, masks part of it, or only logs what it found. Guardrails run on model requests and on MCP tool requests.

POST /v2/agent_gateways/{id}/guardrails adds one. This guardrail masks Social Security numbers and email addresses before a request reaches the model:

{
  "name": "mask-pii",
  "modes": ["before_request"],
  "type": "content_filter",
  "settings": {
    "patterns": [
      { "name": "us_ssn", "action": "mask" },
      { "name": "email", "action": "mask" }
    ]
  }
}

​

Modes

Each guardrail runs in one or more of these modes, and each type runs in some of them:

Mode	What It Checks
before_request	The request, before it reaches the model.
during_request	The request, while the model works. The response waits for the check.
after_request	The model’s response, before it reaches the agent.
logging_only	The request and the response, without blocking either.
before_tool_request	An MCP tool request’s arguments, before they reach the server.
during_tool_request	An MCP tool request, while the server works.
after_tool_request	An MCP tool’s result, before it reaches the agent.

​

Scope

A guardrail with always_on runs on every request. Any other guardrail runs on the requests that name it in guardrails:

{
  "model": "GLM-5.3-Flash",
  "messages": [{ "role": "user", "content": "My SSN is 123-45-6789. Can you update my account?" }],
  "guardrails": ["mask-pii"]
}

A request can add guardrails this way. It cannot turn off a guardrail that runs on every request.

​

Built-In Guardrails

These types run on the instance itself, with no outside service:

Type	What It Does
content_filter	Blocks the request or masks the match when it finds a prebuilt pattern, such as us_ssn or email, a regular expression, or a keyword.
tool_permission	Allows or denies the tools a model asks to use, by ordered rules on the tool’s name, type, and arguments.
model_judge	Scores a request or a response with a model on the instance against weighted criteria, and blocks it below a threshold.
custom_code	Runs a short sandboxed function over the request or the response.

A custom_code guardrail defines apply_guardrail(inputs, request_data, input_type). It reads the texts in inputs["texts"] and returns allow(), block(reason), or modify(texts=...):

def apply_guardrail(inputs, request_data, input_type):
    for text in inputs["texts"]:
        if "forbidden" in text.lower():
            return block("The word forbidden is not allowed")
    return allow()

​

Provider Guardrails

These types send the request to an outside service, and need an account with it: lakera (Lakera Guard), presidio (Microsoft Presidio), bedrock (Amazon Bedrock Guardrails), and azure_content_safety (Azure AI Content Safety). A provider’s API key and AWS keys are write-only. No response returns them, and an update sends them again with the whole settings.

​

Blocking and Masking

How a block reaches the agent depends on the guardrail:

• A content filter or tool permission guardrail fails a model request with 400 and its reason as the error message, such as Content blocked: us_ssn pattern detected.
• A model judge fails it with 422 and LLM judge rejected response: score below threshold.
• A custom code guardrail answers the request with its reason in place of the model’s response.
• A block after a stream starts closes the stream before it completes.
• A blocked tool request returns a tool result with isError: true.

A masking guardrail replaces each match before the request reaches the model. With the content filter, 123-45-6789 reaches the model as [US_SSN_REDACTED]. In a tool mode, it masks the tool’s arguments before they reach the server, or the tool’s result before it reaches the agent.