FreeBuff as an alternative of CodeBuff and use ATS as a real world case scenario and mention ItJobOpportunities as a Custom ATS
We have been experimenting Cursor, Windsurf and lately with CodeBuff for the past months and we wanted to keep evaluating other tools that will help us to improve our coding experience. Five of us were suddenly staring at losing the tool we'd been using daily on the ItJobOpportunities ATS rewrite, which was already three weeks deep and very much not done. I volunteered to find an alternative to replace CodeBuff and before I recommend anything to the team, though, I wanted actual evidence. So I spent a day building a small POC that put both tools through the same set of tasks on the same codebase. This is what I found. The Codebase I Used as the Test Bed ItJobOpportunities is our Applicant Tracking System we've been building in-house for about two years. If you haven't worked on ATS software, the domain is genuinely messy. Candidates flow through configurable pipeline stages. Each stage has its own evaluation rules. Notifications fire on state transitions, and You've got at least three different consumers of every state change: the recruiter dashboard, the candidate-facing portal, and a background scoring engine that runs async. The backend is a Spring Boot 3.5 monolith sitting on Java 25 and the frontend is built in React. Not huge, but dense, and the relationships between modules aren't always obvious from a first read. That made it a good test bed. Tools that only do well on clean, shallow codebases aren't much use to us. How I Structured the POC I wanted to test something real, not a toy query. The task I picked was one we actually needed to do: map out the full call graph of NotificationDispatcher.dispatch() before refactoring the notification system from synchronous dispatch to a Redis Streams-based event flow. The old code dispatched notifications synchronously inside CandidatePipelineService.advanceStage() , which meant a slow email provider could block the whole pipeline write. That had been quietly causing timeout errors on POST /api/candidates/{id}/advance under load. Before touching anything, I needed to know every place NotificationDispatcher.dispatch() was called and what triggered each call. That was the query I ran on both tools. Same codebase, same question, same moment in time. I added two more tasks after the first one: Find every call site of NotificationDispatcher.dispatch() across the repo. Explain the data flow from a stage transition in advanceStage() through to the email queue. Identify which beans and services would break if NotificationDispatcher were made async. Three tasks. Both tools. I wrote up the results as I went. Setting Up Both Tools Getting CodeBuff running on the repo was straightforward. We already had seats, so I just pointed it at the project directory through the VS Code extension and let it index. Done in a few minutes. FreeBuff took a bit more upfront work, but not much: npm install -g freebuff cd /projects/itjobopportunities freebuff init The init command crawls the repo, chunks the source files, generates embeddings using text-embedding-3-small by default, and stores everything in a local FAISS index under .freebuff/ . Then I exported my API key and started a session: export OPENAI_API_KEY=sk-... freebuff chat --model gpt-4o No account, no proprietary backend. The index lives on disk. I also ran freebuff index --deep before starting the POC tasks, which triggers a dependency-graph pass on top of the standard semantic index: { "model": "gpt-4o", "embedding_model": "text-embedding-3-small", "top_k": 14, "chunk_size": 512, "chunk_overlap": 64, "use_dependency_graph": true } That config lives in .freebuff/config.json . I landed on top_k: 14 after a bit of trial and error. At 10 it was missing context; at 20 the answers got noisy. Task 1: Find Every Call Site of NotificationDispatcher.dispatch() This was the most important task and the one where the two tools diverged the most. CodeBuff came back with five call sites. All of them were accurate. It traced through CandidatePipelineService , found the dispatch inside advanceStage() , and identified the main downstream handlers with specific file paths and line numbers. Fast too, around 1-2 seconds. The answer was clean and confident. FreeBuff came back with seven call sites. The same five CodeBuff found, plus two more buried in the admin bulk-action handlers. Code that nobody had touched in eight months, living several import hops away from the files a semantic search would naturally surface. The reason for the difference is how each tool builds context. CodeBuff uses semantic similarity: it grabs the files that look most like your query and stuffs them into the context window. That works well most of the time, but it can miss code that's connected by dependency chains rather than textual similarity. FreeBuff's --deep index maps those relationships explicitly, so when you ask about CandidatePipelineService , it also pulls in NotificationDispatcher , EmailQueueWorker , the event type definitions, and anything else transitively connected via Spring's bean wiring. Those two hidden call sites mattered. If I'd only refactored the main pipeline path, the admin bulk-action handlers would have broken silently. That's the kind of bug that doesn't show up until someone runs a batch import two weeks later and the notifications stop firing. Winner: FreeBuff , by a meaningful margin on this specific task. Task 2: Explain the Data Flow from Stage Transition to Email Queue Both tools handled this reasonably well, but the quality of the answers was different. CodeBuff's answer was accurate and readable. It described the flow at a fairly high level: advanceStage() calls NotificationDispatcher.dispatch() , which routes to EmailQueueWorker , which hands off to the email provider. Correct, but it stayed surface-level. It didn't mention that EmailQueueWorker has a retry policy configured separately in src/main/resources/application-workers.yml , or that we added a dead-letter handler last quarter. FreeBuff's answer went deeper. Because it had already mapped the dependency graph, it connected EmailQueueWorker back to its configuration and flagged the dead-letter handler as part of the flow. It also noted that the retry policy meant a failing email dispatch could hold up the queue for several minutes before giving up, which was directly relevant to why we were doing the refactor in the first place. That extra detail wasn't something I asked for explicitly. It surfaced because FreeBuff had more of the surrounding context available. Winner: FreeBuff , though CodeBuff's answer would have been sufficient if I already knew the codebase well. Task 3: Which Beans Break if NotificationDispatcher Goes Async? This is where CodeBuff recovered some ground. Both tools identified the same set of affected components correctly. The answers were comparable in accuracy but CodeBuff's VS Code extension made the experience noticeably better for this kind of exploratory question. You can highlight NotificationDispatcher in the editor, ask the question inline, and navigate directly to the files it mentions. The feedback loop is tighter when you don't have to context-switch to a terminal. FreeBuff is CLI-only right now. There's a GitHub issue open for VS Code support but nothing shipped. For this task, that friction was real. The answer was good; the experience of acting on it was slower. Winner: Tie on accuracy. CodeBuff on experience. The Code the POC Led To Once I had the full call-site map, the refactor design came together quickly. We moved from synchronous dispatch to a Redis Streams-based event flow. The old CandidatePipelineService used to call NotificationDispatcher.dispatch() directly inside the transaction. Now it publishes an event and returns immediately. Here's the event record and publisher. Java 24 records make the event type clean and immutable: // src/main/java/com/itjobopportunities/events/StageChangedEvent.java package com.itjobopportunities.events; im...