# Structured output: JSON mode and JSON Schema

> Get machine-readable JSON from a model through Tokens: JSON mode, JSON Schema with response_format, forced tool calls, and Anthropic's output_config, with examples, validation advice and what the gateway drops.

Structured output means asking a model for JSON your code can parse, instead of prose. There are four ways to do it through Tokens, and which one works depends on the model and on how the gateway reaches it:

| Method                                          | Endpoint                   | How strong the guarantee is                                         |
| ----------------------------------------------- | -------------------------- | ------------------------------------------------------------------- |
| JSON mode: `response_format: {"type": "json_object"}` | Chat Completions     | Valid JSON, but no promise about the keys                            |
| JSON Schema: `response_format: {"type": "json_schema", ...}` | Chat Completions | Output follows your schema (with `strict: true`), if the model supports it |
| A forced tool call whose arguments are your schema | Chat Completions, Messages | Arguments follow the schema as well as the model manages; works on more models |
| `output_config.format` with a JSON Schema        | Messages                   | Output follows your schema, on models that support it               |

None of this is enforced by Tokens. The gateway does not read, validate or repair the JSON. It forwards the request fields and returns the provider's answer, so the guarantees above are the provider's, and a model that does not support a feature either ignores the field or rejects the request. Always parse and validate the result in your own code.

## What the gateway does and drops

- On **Chat Completions** to a provider that speaks the OpenAI format, `response_format` is forwarded unchanged.
- On **Messages** to a provider that speaks the Anthropic format, `output_config`, `tools` and `tool_choice` are forwarded unchanged.
- When a model is only available from a provider that speaks the **other** format, the gateway translates the request, and the translation carries only a fixed list of fields. A Chat Completions request to an Anthropic-format provider does not carry `response_format` (nor `n`, `seed`, `logprobs` or the penalty fields). A Messages request to an OpenAI-format provider does not carry `output_config`. In both cases the field has no effect and nothing tells you so; the model just answers in free text.
- Tools survive translation in both directions, which is why a forced tool call is the most portable method. See [Tool calling](/docs/tool-calling) and [Messages](/docs/messages).

If a structured request comes back as prose, the likely causes are a model without support, or a translated path that dropped the field. Try the tool-call method on the same model.

## How to find out whether a model supports it

The model catalog at [/models](/models) records context window, prices and a description, but no field for JSON mode or JSON Schema support. To check:

1. Read the maker's documentation for the model (for example, whether it lists "JSON output" or "structured outputs").
2. Send a small request with the field and look at the result: a 400 means the model or provider rejects it, free-form prose means the field was ignored, valid JSON means it works for that request.
3. Run the test several times. Unconstrained JSON mode can pass once and fail on the next prompt.

[Choosing a model](/docs/choosing-a-model) lists the models Tokens compares most often.

## JSON mode

JSON mode makes the model return a valid JSON object, with no guarantee about which keys it contains. Tell the model in a message to produce JSON and describe the shape you want. OpenAI's documentation says you must instruct the model to produce JSON in some message in the conversation; without that the model can fill the response with whitespace until it hits the token limit, and the API can reject the request when the word "JSON" is missing from the context.

:::code-tabs

```bash title="cURL"
curl https://tokens.bd/v1/chat/completions \
  -H "Authorization: Bearer $TOKENS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "max_tokens": 400,
    "response_format": {"type": "json_object"},
    "messages": [
      {"role": "system", "content": "Reply with a JSON object with the keys \"language\" and \"summary\"."},
      {"role": "user", "content": "def add(a, b): return a + b"}
    ]
  }'
```

```python title="Python"
import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://tokens.bd/v1", api_key=os.environ["TOKENS_API_KEY"])

resp = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=400,
    response_format={"type": "json_object"},
    messages=[
        {"role": "system", "content": 'Reply with a JSON object with the keys "language" and "summary".'},
        {"role": "user", "content": "def add(a, b): return a + b"},
    ],
)

choice = resp.choices[0]
if choice.finish_reason == "length":
    raise RuntimeError("Cut off by max_tokens: the JSON is incomplete")
data = json.loads(choice.message.content)
print(data)
```

```typescript title="Node.js"
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://tokens.bd/v1", apiKey: process.env.TOKENS_API_KEY });

const resp = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  max_tokens: 400,
  response_format: { type: "json_object" },
  messages: [
    { role: "system", content: 'Reply with a JSON object with the keys "language" and "summary".' },
    { role: "user", content: "def add(a, b): return a + b" },
  ],
});

const choice = resp.choices[0];
if (choice.finish_reason === "length") throw new Error("Cut off by max_tokens: the JSON is incomplete");
console.log(JSON.parse(choice.message.content ?? "{}"));
```

:::

Check `finish_reason` before parsing. If it is `length`, the model hit `max_tokens` in the middle of the object and the JSON is cut off. Reasoning models spend part of that limit on thinking first, so give them more room; see [Reasoning and thinking models](/docs/reasoning).

## JSON Schema with response_format

JSON Schema output constrains the answer to a schema you supply. On Chat Completions the request looks like this (the shape is OpenAI's, checked October 2026 in [OpenAI's structured outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs) and [migration guide](https://developers.openai.com/api/docs/guides/migrate-to-responses)):

```json
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "ticket",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "severity": { "type": "string", "enum": ["low", "medium", "high"] },
          "needs_followup": { "type": "boolean" }
        },
        "required": ["title", "severity", "needs_followup"],
        "additionalProperties": false
      }
    }
  }
}
```

With `strict: true` OpenAI applies these rules to the schema, and rejects a schema that breaks them:

- The root must be an object, and it cannot be an `anyOf`.
- Every property must be listed in `required`. To make a field optional, allow `null`: `{"type": ["string", "null"]}`.
- Every object needs `"additionalProperties": false`.
- At most 5,000 properties in total and 10 levels of nesting.

Other makers support a subset of JSON Schema, and the subset differs. If a provider rejects your schema, simplify it: flat objects, simple types, `enum` for fixed choices.

:::code-tabs

```python title="Python"
import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://tokens.bd/v1", api_key=os.environ["TOKENS_API_KEY"])

schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "severity": {"type": "string", "enum": ["low", "medium", "high"]},
        "needs_followup": {"type": "boolean"},
    },
    "required": ["title", "severity", "needs_followup"],
    "additionalProperties": False,
}

resp = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=400,
    messages=[
        {"role": "user", "content": "Users report that the login page returns a 500 after the last deploy."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "ticket", "strict": True, "schema": schema},
    },
)

choice = resp.choices[0]
if getattr(choice.message, "refusal", None):
    print("The model refused:", choice.message.refusal)
elif choice.finish_reason != "stop":
    print("Incomplete answer, finish_reason =", choice.finish_reason)
else:
    ticket = json.loads(choice.message.content)
    print(ticket["severity"], ticket["title"])
```

```typescript title="Node.js"
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://tokens.bd/v1", apiKey: process.env.TOKENS_API_KEY });

const schema = {
  type: "object",
  properties: {
    title: { type: "string" },
    severity: { type: "string", enum: ["low", "medium", "high"] },
    needs_followup: { type: "boolean" },
  },
  required: ["title", "severity", "needs_followup"],
  additionalProperties: false,
};

const resp = await client.chat.completions.create({
  model: "deepseek/deepseek-v4.1-flash",
  max_tokens: 400,
  messages: [
    { role: "user", content: "Users report that the login page returns a 500 after the last deploy." },
  ],
  response_format: { type: "json_schema", json_schema: { name: "ticket", strict: true, schema } },
});

const choice = resp.choices[0];
if (choice.finish_reason !== "stop") {
  console.log("Incomplete or refused answer:", choice.finish_reason, choice.message.refusal);
} else {
  console.log(JSON.parse(choice.message.content ?? "{}"));
}
```

```bash title="cURL"
curl https://tokens.bd/v1/chat/completions \
  -H "Authorization: Bearer $TOKENS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "max_tokens": 400,
    "messages": [
      {"role": "user", "content": "Users report that the login page returns a 500 after the last deploy."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "ticket",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "severity": {"type": "string", "enum": ["low", "medium", "high"]},
            "needs_followup": {"type": "boolean"}
          },
          "required": ["title", "severity", "needs_followup"],
          "additionalProperties": false
        }
      }
    }
  }'
```

:::

Handle three outcomes before you parse: a refusal (OpenAI reports it in `message.refusal`), an incomplete answer (`finish_reason` of `length` or `content_filter`), and an answer that parses but fails your own checks. The OpenAI and Anthropic SDKs have helpers that build the schema from a Pydantic model or a Zod schema and parse the reply; they send the same JSON as above, so they work through Tokens when the model supports the feature.

:::note[Responses API]
The Responses API takes the same schema under `text.format` instead of `response_format`, with `name`, `strict` and `schema` one level up and no nested `json_schema` key. See [Responses API](/docs/responses). Support depends on the provider behind the model.
:::

## Structured output with tools

A forced tool call is the most portable method, because tool calling is supported by many more models than `response_format`, and it survives the gateway's format translation. Define one function whose `parameters` are the schema you want, force the model to call it, and read the arguments.

```python title="Chat Completions: forced function"
import json
import os
from openai import OpenAI

client = OpenAI(base_url="https://tokens.bd/v1", api_key=os.environ["TOKENS_API_KEY"])

tools = [
    {
        "type": "function",
        "function": {
            "name": "record_ticket",
            "description": "Record a support ticket extracted from the user's message.",
            "strict": True,
            "parameters": {
                "type": "object",
                "properties": {
                    "title": {"type": "string"},
                    "severity": {"type": "string", "enum": ["low", "medium", "high"]},
                },
                "required": ["title", "severity"],
                "additionalProperties": False,
            },
        },
    }
]

resp = client.chat.completions.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=400,
    messages=[{"role": "user", "content": "The login page returns a 500 after the last deploy."}],
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "record_ticket"}},
)

call = resp.choices[0].message.tool_calls[0]
ticket = json.loads(call.function.arguments)
print(ticket)
```

On Chat Completions `arguments` is a JSON string, so parse it. `strict` inside the `function` object is OpenAI's switch for exact schema adherence and has the same schema rules as above; models and providers that do not know it may ignore it. Nothing here is run by Tokens, which only carries the call. Support for `tool_choice` set to a specific function varies by model: some ignore it, some return a 400. See [Tool calling](/docs/tool-calling).

OpenAI's documentation says Chat Completions does not support tool calling with a `reasoning_effort` other than `none` on GPT-5.4 and later models. If a forced tool call fails on an OpenAI reasoning model, set `reasoning_effort` to `none` or use the Responses API.

On Messages the same idea uses Anthropic's tool format, and the arguments come back already parsed as an object in a `tool_use` block:

```python title="Messages: forced tool"
import os
import anthropic

client = anthropic.Anthropic(base_url="https://tokens.bd", api_key=os.environ["TOKENS_API_KEY"])

tools = [
    {
        "name": "record_ticket",
        "description": "Record a support ticket extracted from the user's message.",
        "input_schema": {
            "type": "object",
            "properties": {
                "title": {"type": "string"},
                "severity": {"type": "string", "enum": ["low", "medium", "high"]},
            },
            "required": ["title", "severity"],
        },
    }
]

message = client.messages.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=400,
    tools=tools,
    tool_choice={"type": "tool", "name": "record_ticket"},
    messages=[{"role": "user", "content": "The login page returns a 500 after the last deploy."}],
)

block = next(b for b in message.content if b.type == "tool_use")
print(block.input)
```

Anthropic notes that forcing a tool with `tool_choice` of `any` or `tool` cannot be combined with its manual extended thinking (`thinking.type: "enabled"`), and that a few of its newest models do not accept forced tool use even with adaptive thinking. If you need both thinking and structured output on Claude models, read [Reasoning and thinking models](/docs/reasoning) first.

## JSON Schema on Messages with output_config

Anthropic's Messages API has its own JSON Schema output, set in `output_config.format` (checked October 2026 in [Anthropic's structured outputs guide](https://platform.claude.com/docs/en/build-with-claude/structured-outputs)). The reply is text in a `text` block that holds the JSON. Anthropic also offers `strict: true` on a tool definition to guarantee that tool names and inputs match the schema.

```bash title="cURL"
curl https://tokens.bd/v1/messages \
  -H "x-api-key: $TOKENS_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "max_tokens": 400,
    "messages": [
      {"role": "user", "content": "The login page returns a 500 after the last deploy."}
    ],
    "output_config": {
      "format": {
        "type": "json_schema",
        "schema": {
          "type": "object",
          "properties": {
            "title": {"type": "string"},
            "severity": {"type": "string", "enum": ["low", "medium", "high"]}
          },
          "required": ["title", "severity"],
          "additionalProperties": false
        }
      }
    }
  }'
```

```python title="Python"
import json
import os
import anthropic

client = anthropic.Anthropic(base_url="https://tokens.bd", api_key=os.environ["TOKENS_API_KEY"])

response = client.messages.create(
    model="deepseek/deepseek-v4.1-flash",
    max_tokens=400,
    messages=[{"role": "user", "content": "The login page returns a 500 after the last deploy."}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "title": {"type": "string"},
                    "severity": {"type": "string", "enum": ["low", "medium", "high"]},
                },
                "required": ["title", "severity"],
                "additionalProperties": False,
            },
        }
    },
)

if response.stop_reason in ("refusal", "max_tokens"):
    raise RuntimeError(f"No valid JSON: stop_reason = {response.stop_reason}")
text = next(b.text for b in response.content if b.type == "text")
print(json.loads(text))
```

This needs a recent Anthropic SDK that knows `output_config`; if your SDK version rejects the argument, upgrade it. Anthropic lists the Claude models that support the feature, and it supports only a subset of JSON Schema: no recursive schemas, no numeric or string length constraints, `additionalProperties: false` on objects, and unsupported features return a 400. A `stop_reason` of `refusal` still returns a 200 and is billed, and the output may not match the schema; `max_tokens` can leave it incomplete.

Remember the limits of this path on Tokens. It works when the model is served by a provider that speaks the Anthropic format. When the gateway has to translate your Messages request to the OpenAI format, `output_config` is dropped (see above). Use the forced-tool method if you need one code path for every model.

## Streaming and structured output

Structured output works with `"stream": true`, but the JSON is only valid when the stream is complete. Concatenate the `content` deltas (or the tool-call `arguments` fragments, or Anthropic's `input_json_delta` pieces) and parse once at the end. Do not parse partial text. See [streaming](/docs/streaming).

## Cost and limits

- The schema, tool definitions and your instructions are input tokens on every request. A large schema costs on every call.
- Tokens bills the usage the provider reports, at the model's input and output prices. There is no surcharge for structured output.
- `max_tokens` bounds the whole answer. A truncated JSON object does not parse, so set it above the longest answer you expect, and add room if the model reasons first.
- The gateway reserves the worst-case cost from `max_tokens` before it forwards the request. See the reservation notes in [Chat Completions](/docs/chat-completions).

## Make it reliable

- Validate every reply against your own schema in code (Pydantic, Zod, `jsonschema`), even when you used strict mode.
- Keep schemas flat and give each field a short `description`; the model reads it.
- Set low `temperature` for extraction tasks. Some reasoning models reject or ignore it; see [Reasoning and thinking models](/docs/reasoning).
- On a parse failure, retry once with the error message appended, then fail loudly. Each retry is a new billed request.
- Test the exact model you will run in production. A schema that works on one model can be rejected by another.

---
Page: https://tokens.bd/docs/structured-output