Classic RAG already retrieves. Agentic RAG is a capped retrieve-grade-rewrite graph where Java decides when the evidence is enough. A bookstore chatbot makes the three paths obvious.
Picture the chatbot on a neighborhood bookstore's site. You text it the way you would text the shop. Three messages. Anyone who has bought a book already knows the difference. Three texts to a bookstore The shop's files have Sunday hours, a shelf list, and nothing about events in other countries. Watch what a "smart" search does with that. Are you open this Sunday? The FAQ says 11 to 5. A clerk would answer once. Classic RAG should too. One retrieve. Done. Do you have Atomic Habits? Search often returns The Power of Habit because the words sit close. Those are different books. A clerk checks the title. Naive RAG says yes anyway. When is James Clear signing in Reykjavik? That date is not in this shop's files. A clerk says "I don't know." Naive RAG invents a Saturday. Agentic RAG should stop after three misses. LangChain4j already had classic RAG before I touched the agentic module. You write an Assistant interface, attach a ContentRetriever , and AiServices stuffs the top-k chunks into the prompt. That is fine for Sunday hours, when the right slice is sitting in the FAQ. It is not the same thing as deciding whether those slices are enough to speak. Session retrieval answers "what did we find this turn." It does not answer "is this the book you asked for." I have shipped the second miss on a product catalog: similar names, wrong item, and the "is this relevant" check lived in the prompt, which means it did not live at all. The bookstore texts are the same bug with less jargon. I did the expensive version next. A second model call asked whether the context was good. The model said yes. The chunk was still The Power of Habit . That is when I stopped treating grade as prose and put it in a Java record. The opinion behind that design is already up as Agentic RAG is a graph with a grade, not a fancier prompt . This piece is how I would actually wire it. What agentic RAG actually changes The idea is simpler than the papers make it sound. Instead of retrieve-then-hope, you run a small graph: retrieve, grade in Java, rewrite the query if the grade fails, stop after a cap, and synthesize only when the record says the evidence is sufficient. Sunday hours skip the loop. Atomic Habits may need a second look or an exact-title drawer. Reykjavik must fail closed. The model still writes labels and answers. Java owns the exit. Naive RAG is one sequence. The reader asks, the assistant embeds, the store returns chunks, the chat model generates anyway. That is how The Power of Habit becomes a yes. If step 5 returns the wrong book, step 6 still says we have it. That is the miss I keep paying for. Agentic RAG adds a loop and a valve. The yellow frame is loopBuilder . Synthesis is optional. The cap is Java, not the model. Same reader. Different control flow. Sunday hours exit early. Reykjavik can fail closed. That's the whole calling contract I care about. No supervisor on day one. No second cron that summarizes retrieval. The grade happens in the same run that would have sold the wrong book. Where this actually changed my thinking My first reaction to "agentic RAG" was skepticism. Letting a model retry retrieval sounded like a good way to spend 3x tokens on the same wrong neighborhood: habit , atomic , still the wrong spine. I have spent 25 years being the person who puts a budget on a loop. An unbounded rewrite is just a more expensive stuff-the-prompt. Then I tried the grade as a record with a compact constructor, and the pattern turned out to be more controllable than a grader prompt. You are not giving the model free rein over "enough." You let it propose RELEVANT , PARTIALLY_RELEVANT , or IRRELEVANT per chunk. Java recomputes sufficient . If the only hit is The Power of Habit , the boolean is false even if the model wrote otherwise. The model proposing RELEVANT is a status update. A GradeVerdict record is a check. That's the piece that flipped my opinion. The interesting part is not that an agent retrieves twice. It is that you get a clean interception point to refuse an answer exactly where the data is missing. Same lesson I keep repeating on loops: the exit lives in code you can test. How I would wire it in LangChain4j I would not start with supervisorBuilder . I would start with types I can name. Java 21 or later, LangChain4j 1.18.1 or later, and the experimental langchain4j-agentic beta that ships sequenceBuilder , loopBuilder , and conditionalBuilder . Compile with -parameters if you want to drop @V . I keep the annotation. Named keys are the graph. <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j</artifactId> <version>1.18.1</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-agentic</artifactId> <version>1.18.1-beta28</version> </dependency> Prove classic RAG first on Sunday hours. If that FAQ is already wrong, a loop will only spend tokens on the same miss. Assistant plus EmbeddingStoreContentRetriever is enough: load the hours page and the shelf list, embed segments, set maxResults and minScore , ask a question whose answer is in the store, and read the chunks, not only the sentence. public interface Assistant { String chat(String userMessage); } ContentRetriever vector = EmbeddingStoreContentRetriever.builder() .embeddingStore(store) .embeddingModel(embeddings) .maxResults(5) .minScore(0.75) .build(); Assistant faq = AiServices.builder(Assistant.class) .chatModel(chat) .contentRetriever(vector) .build(); String hours = faq.chat("Are you open this Sunday?"); contentRetriever(vector) installs naive RAG. On every chat call, LangChain4j embeds the question, searches the store, and injects the top chunks. There is still no way to stop if the chunks are junk. An exact title or ISBN belongs on keyword or SQL, not on another rewrite. That is the cheap toolbox: one drawer for the FAQ, one for the shelf list. LanguageModelQueryRouter plus two ContentRetriever implementations is still one pass. I add the loop only after the router exists. Hybrid search (vector plus trigram) is how I keep "Atomic Habits" from landing on a neighbor. The loop is a budget, not a substitute for the right drawer. ContentRetriever faq = EmbeddingStoreContentRetriever.builder() .embeddingStore(hoursAndPolicyStore) .embeddingModel(embeddings) .maxResults(3) .minScore(0.75) .build(); ContentRetriever shelf = EmbeddingStoreContentRetriever.builder() .embeddingStore(titleAndIsbnStore) .embeddingModel(embeddings) .maxResults(5) .minScore(0.80) .build(); QueryRouter router = LanguageModelQueryRouter.builder() .chatModel(chat) .retrieverToDescription(Map.of( faq, "Sunday hours, returns, and shop policy", shelf, "Book titles, authors, and ISBNs on the shelf list" )) .build(); Hops talk through AgenticScope , a shared map for one run. @Agent(outputKey = "chunks") writes. @V("query") reads. If two hops share a name, the later write wins. That is why the rewriter uses outputKey = "query" . Sequence · AgenticScope keys Print scope.state() after each hop when @V comes back null. The key names have to match. Retrieval does not need an LLM. A class with @Agent is enough. The router from the toolbox becomes a field. public final class RetrieveHop { private final QueryRouter router; public RetrieveHop(QueryRouter router) { this.router = router; } @Agent(value = "Retrieve evidence for the current query", outputKey = "chunks") public List<Content> retrieve(@V("query") String query) { Query q = Query.from(query); return router.route(q).stream() .flatMap(retriever -> retriever.retrieve(q).stream()) .toList(); } } Three labels on each chunk: RELEVANT , PARTIALLY_RELEVANT , IRRELEVANT . Drop the last bucket bef...