I've spent years deciding what state a service should persist and what it should forget. Spring AI's AutoMemoryTools flips that: the agent itself judges what's worth keeping in memory. After building agents on LangChain4j, I wanted to see how this changes the guardrails we need around orchestration. Here's what I foun...
Spring AI already had ChatMemory before this. Wire up a MessageWindowChatMemory or a JDBC-backed store, and the framework keeps the last N messages around so the model has context. That's fine for short-term continuity. It's not the same thing as memory in the sense people mean when they say "the agent should remember me." Session memory answers "what did we just talk about." It doesn't answer "what facts about this person are still true next week." If a candidate tells the assessment bot they're based in Costa Rica and prefer backend roles, that's durable information, not a chat-window fact. Most teams end up hand-rolling this: a regex pass, or a second LLM call that extracts "important" bits on a schedule and writes them to a database. I did exactly that on an early prototype for our Job Fit feature. Not proud of it. A cron job ran a summarization prompt over recent candidate messages, trying to extract structured facts with a JSON schema. It worked, sort of, but it was brittle and reacted to a clock instead of to what actually happened in the conversation. What AutoMemoryTools actually changes The idea is simpler than I expected: instead of a separate extraction pass, you give the model tools for storing and retrieving memory, and let it call them whenever it decides something is worth keeping. The model becomes the one deciding what matters, in real time, in the same turn where the user said the thing. Roughly how it looks wired into a ChatClient : @Bean ChatMemoryRepository chatMemoryRepository(JdbcTemplate jdbcTemplate) { return JdbcChatMemoryRepository.builder() .jdbcTemplate(jdbcTemplate) .build(); } @Bean ChatClient chatClient(ChatClient.Builder builder, AutoMemoryTools autoMemoryTools) { return builder .defaultTools(autoMemoryTools) .defaultSystem(""" You are an assistant helping recruiters assess candidates. Store durable facts about the candidate using your memory tools. Do not store salary expectations unless explicitly asked to. """) .build(); } AutoMemoryTools exposes something like storeMemory(String fact) and searchMemory(String query) as regular function calls. The model sees them next to whatever domain tools are registered, and decides mid-conversation whether to invoke them: public String assessCandidate(String candidateId, String message) { return chatClient.prompt() .user(message) .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, candidateId)) .call() .content(); } That's the whole calling contract. No separate extraction job, no cron, no second model call to summarize. The memory write happens as a side effect of the same generation that answered the user. Where this actually changed my thinking My first reaction was skepticism. Letting the model decide what to persist sounded like a good way to end up with a memory store full of noise, or PII you never meant to keep. I've spent 25 years being cautious about what gets written to disk in audited, regulated delivery environments, so "let the LLM decide" set off every alarm I have. Then I tried it on a throwaway prototype with a mock candidate conversation, and the tool-calling pattern turned out to be more controllable than I expected. You're not giving the model free rein over your database. You're giving it a narrow tool with a schema, and you control what that tool does when it's invoked. Want to reject anything that looks like a phone number or an email before it ever hits storage? Do it in the tool implementation, not in the prompt: @Tool(description = "Store a durable fact about the current candidate") public String storeMemory(String fact) { if (containsPii(fact)) { return "Rejected: fact appears to contain personal contact information."; } memoryStore.save(currentCandidateId(), fact); return "Stored."; } That's the piece that flipped my opinion. The interesting part isn't that the model decides what's worth remembering. It's that you get a clean interception point to enforce policy exactly where the decision gets made, instead of scrubbing a summarization prompt after the fact and hoping the instructions stick. Comparing it to what I do in LangChain4j Agent UI Editor is built on LangChain4j, not Spring AI, so I don't get AutoMemoryTools out of the box there. What I do instead is closer to the manual pattern: a dedicated memory-write node in the workflow graph, wired as its own agent step with a narrow system prompt ("extract only durable facts, ignore small talk"), running after the main response node. It works, but it's more moving parts. Two LLM calls where Spring AI's approach needs one. Latency matters here: in a quick test against a mid-size OpenRouter model, the two-call pattern added close to 900ms per turn just for the extraction step. Not huge on its own, but it compounds across a chat-heavy flow. I'm not rewriting Agent UI Editor's memory handling to copy Spring AI. LangChain4j's agentic module doesn't have a drop-in equivalent yet, and the multi-agent graph pattern I use, Sequence, Parallel, Supervisor nodes, actually makes a dedicated memory-write node more natural, since I can route it in parallel with the response node instead of serializing it. But it changed how I think about tool-based memory versus pipeline-based memory. Tool-based is cheaper and more reactive. Pipeline-based gives you more control over exactly when memory gets touched and lets you audit it as a distinct step in the graph. How often should memory actually get cleared This is the question nobody answers in the docs, and it's the one I kept circling back to once I got past the "should the model decide what to store" debate. Writing memory is only half the design. The other half is deciding when it stops being true, and that's a policy decision, not a framework feature. There are really two clocks you can attach memory expiry to, and they behave very differently. Clock-based expiry treats memory like a cache with a TTL. A fact expires N days after it was written, full stop, regardless of what happens in between. This is predictable and easy to reason about, and it's the one I'd default to for anything that touches PII, because it gives you a hard upper bound on retention without needing the application to understand its own lifecycle: @Scheduled(cron = "0 0 3 * * *") public void pruneExpiredMemory() { Instant cutoff = Instant.now().minus(90, ChronoUnit.DAYS); memoryRepository.deleteByCreatedAtBefore(cutoff); } The downside is that clock-based expiry has no idea whether the fact is still relevant to an active workflow. If a candidate is mid-pipeline on day 89, you can delete a fact right before the recruiter needs it, purely because a scheduler fired. Session-based (or event-based) clearing ties memory lifetime to something that happens in the business process instead of a calendar. On ITJobOpportunities, the natural boundary for candidate assessment memory isn't a date, it's the job application's lifecycle. Memory tied to a specific application should probably clear when that application closes (hired, rejected, withdrawn), not 90 days after the first message: @EventListener public void onApplicationClosed(JobApplicationStatusChangedEvent event) { if (event.isTerminalStatus()) { memoryRepository.deleteByApplicationId(event.getApplicationId()); } } The tradeoff is the mirror image of clock-based expiry. Event-based clearing respects what's actually happening in the workflow, but it depends on your application reliably firing the right event at the right time. If a status transition gets skipped or a webhook fails silently, that memory just sits there indefinitely, which defeats the point of having an expiry policy in the first place. The other dimension worth deciding up front is scope. Is the clock per-fact, per-conversation, or per-candidate? A fact like "prefers remote roles" arguably shouldn't expi...