Skip to content

How to get valid JSON from an LLM, every time

From the work/ 15 min read

Asking an LLM for “JSON only” is not enough. With structured outputs, a zod schema and a retry that sends the error back, you get a valid object or a clear error.

  • AI
  • JavaScript
In this article (8)
  1. Why "respond only in JSON" is not enough
  2. Structured outputs: give the LLM a schema
  3. Validate with zod
  4. Retry with the error and an attempt limit
  5. Full TypeScript example
  6. Common mistakes
  7. Frequently asked questions
  8. Checklist

To get valid JSON from an LLM every time, asking for it in the prompt is not enough: turn on structured outputs with a JSON schema in output_config.format, validate the reply with zod and, when it fails, retry with the exact error sent back to the model, with an attempt limit. Handle refusals and max_tokens cutoffs separately.

This guide covers what structured outputs guarantee according to Anthropic's docs and what they do not, how zod checks what the schema cannot, and a TypeScript function that retries with the error. The request shape follows the docs as they stood on November 2, 2026, and the tests use a fake client: we made no live API call. If your problem is the instructions themselves, start with how to write an AI landing page prompt.

Why "respond only in JSON" is not enough

An instruction in the prompt is a request, not a contract. The model follows it almost every time, and "almost" breaks an integration that runs thousands of times a day, such as importing supplier product sheets. Anthropic's structured outputs docs list what happens without a schema, even with careful prompting: parsing errors from invalid JSON syntax, missing required fields, inconsistent data types, and schema violations that force error handling and retries.

In practice it looks like this:

  • Prose around the JSON. "Sure, here is the product:" followed by the object. JSON.parse fails on the first character.
  • Markdown fences. The JSON arrives inside ```json. It looks fine in a chat window and breaks your code.
  • Truncation at max_tokens. The reply stops in the middle of a field.
  • Wrong types. "price": "899" instead of 899. The worst one, because JSON.parse accepts it and the bug shows up later: in JavaScript, "899" + 99 is "89999".
  • Missing or extra fields. No inStock, or a color nobody asked for.
how LLM JSON output breaks, and what catches itfailurewhat you getwith schemawhat the code doesprose around the JSONSure, here it is: {…}preventedSyntaxError, retrymarkdown fences```json {…} ```preventedSyntaxError, retrycut off at max_tokens{"price":899,"currcan happendouble max_tokenswrong type"price":"899"preventedzod error, retrymissing fieldno "inStock"preventedzod error, retryextra field"color":"black"preventedz.object drops itenum casing"mxn" instead of "MXN"can happenzod error, retrystructured outputs prevents five; not truncation or enum casingand what the API does not enforce, like .positive(), only zod checks

Scroll the figure sideways to see all of it

The fix has three layers: a schema the model cannot step outside of, validation in your code for what the schema does not cover, and a retry that carries the error when something fails. None of them is enough on its own.

Structured outputs: give the LLM a schema

Structured outputs constrain what the model generates to a JSON schema through constrained decoding. The docs sum it up in three promises: always valid, with no more JSON.parse() errors; guaranteed field types and required fields; and no retries needed for schema violations. You turn it on per request, with the schema inside output_config.format:

{
  "model": "claude-sonnet-5-5",
  "max_tokens": 1024,
  "messages": [{ "role": "user", "content": "Extract the product from this supplier sheet: …" }],
  "output_config": {
    "format": {
      "type": "json_schema",
      "schema": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "price": { "type": "number" },
          "currency": { "type": "string", "enum": ["MXN", "USD"] },
          "sizes": { "type": "array", "items": { "type": "string" } },
          "inStock": { "type": "boolean" }
        },
        "required": ["title", "price", "currency", "sizes", "inStock"],
        "additionalProperties": false
      }
    }
  }
}
  1. model and max_tokens are as usual. Per the docs, structured outputs are available on every model; the list includes claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5.
  2. messages carries the request in plain language. The schema does not replace the prompt: still say what to extract and from where.
  3. output_config.format with type: "json_schema" turns the feature on, and no beta header is needed anymore. The old parameter, output_format, is deprecated: it only works with the structured-outputs-2025-11-13 beta header, and without it the API returns a 400.
  4. schema is plain JSON Schema: required lists every field, and every object sets additionalProperties: false, which the API requires.
  5. enum limits currency to two values. Watch the casing, covered below.

Claude returns the JSON in the response's text content block. What the schema cannot use, per the limitations section: recursive schemas, numerical constraints (minimum, maximum, multipleOf), string constraints (minLength, maxLength), array constraints beyond a minItems of 0 or 1, complex types inside an enum, external $ref, and additionalProperties set to anything but false. Use any of them and you get a 400. There are per-request limits too: 20 strict tools, 24 optional parameters and 16 parameters with union types.

And this is what it does not guarantee, which is the part your code has to handle:

CaseWhat the docs sayWhat your code does
JSON that does not parseDoes not happen with the schema onCatches it anyway: it costs one line
Fields and typesGuaranteed for the fields in requiredValidates with zod and types the result
Refusalstop_reason: "refusal", a 200 status, billed, and the output may not match the schemaStops without retrying
Token limitstop_reason: "max_tokens", incomplete output; retry with a higher max_tokensRetries with twice as many
enum casingNot guaranteed: you may get "Conversation Topic 3" when the value is "Conversation topic 3", with no errorZod rejects it and the code retries
Property orderRequired properties first, then optional onesDoes not rely on order

Structured outputs are also incompatible with citations (you get a 400) and with prefilling the response. For tool inputs there is a sibling feature, strict tool use: strict: true on a tool guarantees its input follows the input_schema. Both use the same grammar pipeline and can be combined.

Validate with zod

If the schema is guaranteed, why validate? Three reasons. The table above leaves three gaps: refusals, truncation and casing. Constraints the API does not support are not enforced at all. And you want a typed object in TypeScript, not an unknown.

The TypeScript SDK, @anthropic-ai/sdk (we tested 1.4.0), ships zodOutputFormat(), which turns a zod schema into the output_config.format. The docs explain that the SDKs strip unsupported constraints from the schema and add them to the field's description. This is what it produced for price: z.number().positive() and currency: z.enum(["MXN", "USD"]):

{
  "price": {
    "type": "number",
    "description": "{exclusiveMinimum: 0}"
  },
  "currency": {
    "type": "string",
    "description": "{enum: [\"MXN\",\"USD\"]}"
  }
}
  1. price goes out as an unconstrained number: .positive() became exclusiveMinimum: 0 and moved into the description, as a hint for the model. The API will not stop a negative price.
  2. currency also goes out as a string with a description. In SDK 1.4.0 the helper moved the enum into the description instead of keeping it in the schema, even though the API does support enum. The grammar does not enforce it; only zod does.

That is why validation in your code is not optional. Product.safeParse(...) returns { success, data } or { success, error } without throwing, and z.prettifyError(result.error) turns the error into readable text, one line per issue with its path (→ at price): exactly what the model needs to fix its answer. Tested with zod 4.6.7: z.object() silently drops undeclared keys (z.strictObject() rejects them with Unrecognized key: "color"), and a missing field reads expected boolean, received undefined.

On versions: the SDK accepts zod ^3.25.0 || ^4.0.0. On zod 4, import * as z from "zod". On zod 3.25.x, import from "zod/v4", a subpath that zod's versioning page says stays available forever. With zod 3.25.76 and "zod/v4" the same four tests pass; with "zod", zodOutputFormat() throws Cannot read properties of undefined (reading 'def'). More in zod's error formatting guide and its JSON Schema page, which covers z.toJSONSchema(), the function the helper calls internally.

Retry with the error and an attempt limit

A retry that only says "try again" tends to repeat the same mistake. One that sends the model its own reply plus the exact error gives it something concrete to fix. These are the rules we follow:

from the LLM request to a validated objectclient.messages.create()model + output_config.formatres.stop_reasonrefusal: stop · max_tokens: double itJSON.parse(text)on failure: retry with its messageProduct.safeParse(json)on failure: retry with z.prettifyErrorreturn result.dataa typed object, ready to usethe error goes back to the model as a user messagedashed line: retry, 3 attempts at most; a refusal is never retried

Scroll the figure sideways to see all of it

  • Refusal (refusal): stop. Sending the same request again does not change the decision, and every attempt is billed. Throw an error so a person can look at it.
  • Cutoff (max_tokens): retry with twice the budget. That is what the docs recommend. Do not send the truncated reply back: there is nothing to correct, it just ran out of room.
  • JSON.parse or zod fails: send the error back. Append the reply as an assistant turn and a user turn with the error and an instruction to answer with only the corrected object. The last message is from the user, so this is not a prefill.
  • Attempt limit. Three by default. After that, throw with the last reason so you can log and review it, instead of a loop that burns tokens forever.

This is the message our function built when the fake client returned wrong types:

Your reply does not match the JSON schema:
✖ Invalid input: expected number, received string
  → at price
✖ Invalid option: expected one of "MXN"|"USD"
  → at currency
✖ Invalid input: expected boolean, received string
  → at inStock
Reply again with only the corrected JSON object.
  1. The first line is fixed and says what went wrong.
  2. Each ✖ is one issue from z.prettifyError, with its field on the → at line: price came as a string, currency as "mxn" and inStock as "yes".
  3. The last line asks for the object only, with no explanation around it.
the JSON error that goes back to the model1 · the model's reply (assistant)```json{"title":"Sudadera Volcán","price":899,…}```does not start with {2 · the retry (user)Your reply does not match the JSON schema:fixed headerUnexpected token '`', "```jsonJSON.parse's err.message,{""... is not valid JSONcopied as isReply again with only the corrected JSON object.asks for the object only3 · if zod fails, this goes there instead✖ Invalid input: expected number, received string→ at pricez.prettifyError(result.error)one line per issuethe exact error goes back, not a bare “try again”a reply cut off at max_tokens is not sent back: retry with twice the tokens

Scroll the figure sideways to see all of it

Full TypeScript example

All of it fits in one function, extractJson, in extract.ts. It takes the client as an argument: the SDK client in production, a fake one in tests.

import type Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import * as z from "zod";

export const Product = z.object({
  title: z.string(),
  price: z.number().positive(),
  currency: z.enum(["MXN", "USD"]),
  sizes: z.array(z.string()),
  inStock: z.boolean(),
});

type Client = {
  messages: { create(body: Anthropic.MessageCreateParamsNonStreaming): Promise<Anthropic.Message> };
};

export async function extractJson<S extends z.ZodType>(
  client: Client,
  schema: S,
  prompt: string,
  { model = "claude-sonnet-5-5", maxTokens = 1024, maxAttempts = 3 } = {},
): Promise<z.infer<S>> {
  const messages: Anthropic.MessageParam[] = [{ role: "user", content: prompt }];
  let lastError = "";

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const res = await client.messages.create({
      model,
      max_tokens: maxTokens,
      messages,
      output_config: { format: zodOutputFormat(schema) },
    });

    if (res.stop_reason === "refusal") {
      throw new Error("The model refused the request; retrying will not change that.");
    }
    if (res.stop_reason === "max_tokens") {
      lastError = `output cut off at max_tokens=${maxTokens}`;
      maxTokens *= 2;
      continue;
    }

    const text = res.content.flatMap((block) => (block.type === "text" ? [block.text] : [])).join("");
    try {
      const result = schema.safeParse(JSON.parse(text));
      if (result.success) return result.data;
      lastError = z.prettifyError(result.error);
    } catch (err) {
      lastError = (err as Error).message;
    }

    messages.push(
      { role: "assistant", content: text || "(empty)" },
      {
        role: "user",
        content: `Your reply does not match the JSON schema:\n${lastError}\nReply again with only the corrected JSON object.`,
      },
    );
  }

  throw new Error(`No valid JSON after ${maxAttempts} attempts. Last error: ${lastError}`);
}
  1. Product is the zod schema: title, a positive price, MXN or USD, sizes and stock status. The example data is illustrative.
  2. Client describes only what the function uses, messages.create. The real SDK client fits that type, and so does a fake one.
  3. extractJson takes the client, the schema, the prompt and options: model, maxTokens (1024) and maxAttempts (3). It returns z.infer<S>, the type derived from the schema.
  4. Each attempt sends the conversation so far plus output_config: { format: zodOutputFormat(schema) }. The helper also carries a parse function that never goes over the wire: JSON.stringify drops it when the request is serialized.
  5. refusal throws right away. max_tokens records the reason, doubles maxTokens and asks again without touching the conversation.
  6. It joins the text of the text blocks, parses it and validates it. If it passes, it returns result.data, already typed; if not, the reason is the z.prettifyError text or the SyntaxError message.
  7. It appends the reply and the correction message to the conversation. text || "(empty)" avoids sending an empty turn.
  8. After maxAttempts attempts it throws with the last reason.

With the SDK you call it like this. The file passes TypeScript 5.7.3's strict typecheck, but we did not run it: it needs an API key, and we sent no data to Anthropic's API.

import Anthropic from "@anthropic-ai/sdk";
import { extractJson, Product } from "./extract.ts";

const client = new Anthropic(); // reads ANTHROPIC_API_KEY
const sheet = "Sudadera Volcán, algodón 100 %, tallas S a L, $899 MXN, disponible.";
const product = await extractJson(client, Product, `Extract the product from this supplier sheet:\n${sheet}`);
console.log(product.price, product.inStock);
  1. new Anthropic() reads the key from ANTHROPIC_API_KEY, and sheet is an illustrative supplier sheet.
  2. extractJson returns an object typed from Product: product.price is a number and product.inStock a boolean, with no casts.

If you do not need retries, the SDK has client.messages.parse(), which validates and returns parsed_output. In 1.4.0 it throws an AnthropicError when the JSON or zod fails, and it never looks at stop_reason: a truncated reply shows up as one more parse error. That is why this code calls create() and validates on its own.

For the tests, a fake client records each request exactly as it would go over the wire (JSON.parse(JSON.stringify(body))) and returns five replies, in order: the JSON inside a ```json fence, prose before the JSON, wrong types, a truncated reply with stop_reason: "max_tokens" and, last, the valid JSON. We ran it on Node 22.22.3, which runs .ts files with no extra flags, with zod 4.6.7 and @anthropic-ai/sdk 1.4.0. This is the output:

$ node --test --test-reporter=spec extract.test.ts
✔ retries each failure with its error and returns the valid object (9.5142ms)
✔ sends the schema in output_config.format (0.5039ms)
✔ stops on a refusal without retrying (0.6876ms)
✔ gives up after maxAttempts (1.6116ms)
ℹ tests 4
ℹ suites 0
ℹ pass 4
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 257.4801
  1. With maxAttempts: 5, the function works through all five replies and returns the valid object. The test checks the error each retry carries, and that the fifth request raises max_tokens to 2048 without the truncated reply.
  2. The request carries output_config.format with type: "json_schema", additionalProperties: false and all five fields in required.
  3. A refusal ends in an error after a single request.
  4. Three invalid replies in a row end in No valid JSON after 3 attempts, with no fourth request.

Common mistakes

  • Relying on the prompt alone. "Respond only in JSON" works almost every time. For anything that runs unattended, use the schema.
  • "Repairing" the JSON with regexes. Stripping fences or slicing from the first { to the last } hides the problem and misses truncation. Our recommendation: treat any text that does not parse as a failure and retry with the error.
  • Retrying without the error, or without a limit. Without the error the model does not know what to fix; without a cap the loop burns tokens until someone notices.
  • Retrying a refusal, or sending a truncated reply back. The first will not change; the second needs a bigger max_tokens, not a correction.
  • Assuming the schema enforces .positive() or .min(). The API does not support those constraints: the SDK moves them into the description, and only your safeParse enforces them.
  • Assuming enum casing always matches. It is not guaranteed. Our function rejects the value and retries; to save the attempt, normalize it before validating.
  • Using zod 3 with the SDK helper. On zod 3.25.x, "zod" is version 3 and the helper fails; import from "zod/v4".
  • Staying on output_format. It is deprecated and returns a 400 without its beta header. Move it to output_config.format.
  • Putting sensitive data in the schema. Schemas are cached for up to 24 hours, separately from message content, and the docs say protected health information (PHI) must not appear in them. Keep field names and enum values generic.

Frequently asked questions

Which Claude models support structured outputs?

All of them, according to the docs as of November 2, 2026. The list includes claude-opus-5-5, claude-sonnet-5-5 and claude-haiku-5-5, plus earlier models such as claude-haiku-4-5-20251001. The feature is generally available: with output_config.format you need no beta header.

Structured outputs or strict tool use?

They solve different problems. Structured outputs (output_config.format) control Claude's response, what it says. Strict tool use (strict: true on a tool) guarantees the arguments it passes to your functions. To extract data for your own code, use structured outputs; for an agent that calls tools, use strict tool use. You can combine them in one request.

Do I still need zod if I use structured outputs?

Yes. It covers what the schema does not guarantee (truncation, refusals, enum casing), enforces what the API does not support, such as a positive price, and gives you the TypeScript type. If you switch models or providers, the validation stays the same.

Why is the first request with a schema slower?

Because the API compiles the schema into a grammar the first time it sees it. Per the docs, the grammar stays cached for 24 hours from last use, and changing the schema's structure invalidates it. Note that changing output_config.format also invalidates the prompt cache for that conversation.

Checklist

  • The request puts the schema in output_config.format with type: "json_schema", not just in the prompt.
  • Every object has additionalProperties: false, and the fields you need are in required.
  • The schema uses nothing the API does not support, or the SDK helper converts it.
  • Your code validates every reply with zod, structured outputs or not.
  • refusal stops the process; max_tokens retries with more tokens, without sending the cut-off text back.
  • A JSON.parse or zod failure goes back to the model with the exact error.
  • There is an attempt limit and a clear error when it is reached.
  • On zod 3.25.x you import from "zod/v4".
  • Tests use a fake client with broken replies and never call the API.

A schema, a validation step and a retry that carries the error turn "JSON most of the time" into a valid object or an error you can review. If the model still misreads what to extract, the problem is in the instructions: see how to build an AI landing page prompt and what context to give an AI coding agent. And when the bug is in your code rather than in the reply, here is how to report a bug to AI.

Saw this on Instagram? Comment JSON on the Reel and I'll send you the guide.

Written by

Luis Morales

Luis Morales

Frontend and Shopify developer, EcomLabs

@luismoralesdev on Instagram

Published November 3, 2026

← Blog