What if your AI could catch and fix its own output errors before they ever reach your application? Spring AI 2.0 introduces self-correcting structured output, and honestly, it changes how I think about building reliable AI systems. I broke down exactly how it works and why it matters.
Getting a language model to return valid JSON feels like it should be a solved problem by now. It really doesn't seem that hard. Ask the model to return structured data, it returns structured data, your application uses it. Done. But anyone who's shipped an LLM-backed feature knows that's not how it goes. What you actually get, at least some of the time, is something like this: Sure! Here's the JSON you asked for: {"name": "John", "age": 30} Let me know if you need anything else! Or worse, JSON that's almost valid. A trailing comma, a missing bracket, a field name that's slightly off from what your schema expects. Your deserializer throws, your error handling kicks in, and you're either surfacing a 500 to the user or silently dropping data. Neither is great. I ran into this constantly during a job recommendation feature we built a while ago. The model was GPT-4o, the task was to return a structured list of product suggestions with reasons and confidence scores, and about 8% of responses were unparseable on the first try. That's not catastrophic, but it's not acceptable either, especially when the fallback was just "no results." Spring AI 2.0 has a real answer to this: self-correcting structured output. And it's the thing I wish had existed 18 months ago. What We Were Doing Before Spring AI 2.0 Before 2.0, structured output in Spring AI was honestly pretty bare. You had BeanOutputConverter and OutputParser , which would generate a format hint from your Java type and append it to the prompt. Something like "respond with JSON matching this schema." That part worked fine. The problem was what happened when the model didn't comply. Nothing. You got an exception and it was your problem. So teams (including mine) built their own retry wrappers. The pattern looked roughly like this: public <T> T callWithRetry(String prompt, Class<T> responseType, int maxAttempts) { BeanOutputConverter<T> converter = new BeanOutputConverter<>(responseType); String fullPrompt = prompt + "\n" + converter.getFormat(); for (int attempt = 1; attempt <= maxAttempts; attempt++) { try { ChatResponse response = chatModel.call(new Prompt(fullPrompt)); String text = response.getResult().getOutput().getContent(); return converter.convert(text); } catch (Exception e) { if (attempt == maxAttempts) throw e; // just... try again with the same prompt } } throw new IllegalStateException("Should not reach here"); } See the problem? The retry sends the exact same prompt. The model has no idea what went wrong. So it either produces the same broken output or gets lucky on the second attempt. It was basically a coin flip dressed up as error handling. Some teams got more creative. I saw a pattern where the exception message got appended to a new prompt manually, something like "Your previous response was invalid: " + e.getMessage() + ". Try again." That worked better, but it was fragile. You had to parse the exception yourself, format the error message in a way the model could understand, manage conversation history to preserve context, and do all of this without any framework support. It was a lot of glue code. There was also the option of reaching for external libraries. LangChain4j (the Java port of the Python LangChain library) had more mature output parsing support at the time, and I know a few teams who switched to it specifically for this reason. That's a real trade-off: LangChain4j has a broader feature set for structured output handling but it's a heavier dependency with its own abstractions that don't always play nicely with the rest of a Spring ecosystem. And if you were already using Spring AI for other things, pulling in a second AI framework just for retry logic felt wrong. The other common approach was to skip retries entirely and make the downstream code more tolerant. Parse what you can, default the rest, never throw. That works until you have a field that genuinely can't be defaulted, like a primary key or a required category ID. None of these solutions were good. They were workarounds. What "Self-Correcting" Actually Means Here The basic idea isn't complicated. When the model returns something that doesn't match your expected schema, instead of throwing an exception and giving up, the framework sends the model a follow-up prompt. That prompt includes the original request, the bad output it produced, and the specific validation errors. Then it asks the model to try again. That's it but the implementation details matter a lot. Spring AI 2.0 does this through the BeanOutputConverter with retry support baked into the StructuredOutputConverter pipeline. The OutputParser abstraction has been around since earlier versions, but 2.0 tightened up the retry loop and added proper schema generation from your Java types using Jackson's schema tooling. So the model isn't just being told "that was wrong, try again." It's seeing its own output, the JSON schema it was supposed to match, and a precise description of where it failed. That specificity is what makes the retry actually work. Not magic. Just good context. Setting It Up Here's the basic setup. Assume you have a domain object you want the model to populate: public record ProductRecommendation( String productId, String name, String reason, @JsonProperty("confidenceScore") double confidenceScore, List<String> tags ) {} In Spring AI 2.0, you'd wire this up something like: @Service public class RecommendationService { private final ChatClient chatClient; public RecommendationService(ChatClient.Builder builder) { this.chatClient = builder.build(); } public List<ProductRecommendation> getRecommendations(String userQuery) { return chatClient.prompt() .user(u -> u.text(""" Based on this customer query, suggest 3 relevant jobs. Query: {query} """) .param("query", userQuery)) .call() .entity(new ParameterizedTypeReference<List<ProductRecommendation>>() {}); } } The .entity() call is where the magic happens. Spring AI generates the JSON schema from your type, injects it into the prompt, and handles the parse-and-retry loop if the response doesn't conform. You don't write any of that retry logic yourself. If you want to configure max retry attempts, you do that at the ChatClient level when you build it, through the RetryTemplate configuration. Default is 3 attempts. @Bean public ChatClient.Builder chatClientBuilder(ChatModel chatModel) { return ChatClient.builder(chatModel) .defaultAdvisors(new SimpleLoggerAdvisor()); } The retry behavior is handled internally by the output converter, so you're not manually wrapping calls in try-catch blocks and re-invoking the model. That was the pattern I was using before, and it was a mess. Lots of stateful stuff, hard to test, and it didn't pass along the validation errors to the model anyway, so the retries often produced the exact same bad output. Pointless. The Prompt That Gets Sent on Retry This part I found genuinely interesting to look at. When a parse failure happens, Spring AI constructs a follow-up prompt that looks roughly like this (I'm paraphrasing from the source, not quoting exactly): Your previous response failed to parse. Here was your response: [model's bad output] Here are the errors: [validation error messages] Please correct your response and return only valid JSON matching this schema: [JSON schema] The key detail is that it includes the validation errors. That's not just "try again." It's "here's specifically what was wrong." Models respond much better to that kind of feedback, at least in my testing. The 8% failure rate I mentioned earlier? With self-correcting output and 2 retries configured, it dropped to under 0.3% over about two weeks of production traffic on that recommendation endpoint. Two weeks,...