AI writes code faster than any of us ever could, but it doesn't know what your system promised customers yesterday. Building ITJobOpportunities taught me that speed without regression discipline just means breaking things faster. Here's why I trust my test suite more than ever now, not less.
I merged a pull request last month that I didn't fully read. Not proud of it, but there it is. It was a small refactor to the resume parsing service on ITJobOpportunities, generated mostly by an AI assistant inside my IDE, and it looked clean. Passed the linter. Compiled. I skimmed it, saw nothing alarming, and hit merge before lunch. Three days later, our Job Fit scoring started returning odd results for resumes with multi-column layouts. Not broken exactly. Just off. Scores that should have landed in the 70s were coming in at 40 something. It took me almost half a day to trace it back to that "small" refactor, where the AI had quietly changed how we handled whitespace normalization in a text extraction helper. The old code trimmed and collapsed whitespace one way. The new code did something subtly different that worked fine for most resumes and quietly broke the rest. Nobody caught it in review because it wasn't a bug you'd catch by reading the diff. It was a bug you'd catch by running the thing against real data. That's basically the whole point of this article. The productivity trap Everyone's talking about how AI coding tools make you faster. And they do. I write boilerplate controllers, DTO mappers, and repository interfaces noticeably faster than I did two years ago. But speed of writing code isn't the same as confidence in code. And that gap is where things get dangerous. Here's the part nobody says out loud enough: when a human writes bad code, it's usually bad in a predictable way. Off by one errors. Forgotten null checks. Copy paste mistakes where you forgot to update a variable name. You develop a nose for it after enough years in the trenches. AI-generated code fails differently. It fails in ways that look plausible. The syntax is right. The naming is consistent, sometimes cleaner than what a tired human would write at 6pm. The logic often makes sense in isolation. But it can drift from the actual intent of the codebase in ways that are almost impossible to spot by eye, because the code isn't wrong, it's just not quite what you meant. It solved a slightly different problem than the one you had. Why this hits distributed systems harder On a single-file script, this is annoying but recoverable. You run it, you see the output is wrong, you fix it. Microservices don't work that way. On the jobs-posting backend that powers ITJobOpportunities, a change in one service can ripple through async job processing, SSE streams, Keycloak token validation, database migrations via Liquibase, and whatever downstream consumer is listening on a queue. A subtle regression in one place doesn't announce itself. It shows up as an odd support ticket three services away, two weeks later, and by then good luck remembering which commit caused it. I've seen this pattern on bigger programs too, not just at ITJobOpportunities. On a hospital microservices program I worked on a few years back, we had a rule: any AI-assisted change to a shared library needed the full regression suite before merge, no exceptions, even for "obviously safe" one liners. People grumbled about it at first. Then someone's "safe" one liner in a shared date formatting utility broke three downstream services that depended on a specific timezone assumption nobody had documented. After that, nobody grumbled anymore. What I actually do differently now I changed how I write tests after that Job Fit incident. Not dramatically, but enough that it's become a habit. I write regression tests before I let an AI touch legacy code, not after. This felt backwards at first. Usually you write the feature, then backfill tests if you have time. Now, if I'm about to ask an AI assistant to refactor something in the resume parsing pipeline, I write a handful of characterization tests first, capturing current behavior, weird edge cases included, even if that behavior isn't textbook correct. Then I run the AI-assisted change and see what breaks. @Test void extractsTextFromMultiColumnResume() { String rawText = loadFixture("multi-column-resume.pdf"); String normalized = textNormalizer.normalize(rawText); // this exact spacing pattern matters for downstream skill matching assertThat(normalized).contains("Java Spring Boot PostgreSQL"); assertThat(normalized).doesNotContain("Java\n\nSpring"); } That test would have caught my whitespace bug in seconds instead of half a day. I stopped trusting "it compiles and passes existing tests" as a real signal. Existing tests only protect the paths someone already thought to test. Here's what our suite actually looked like before the incident: @Test void extractsTextFromSingleColumnResume() { String rawText = loadFixture("single-column-resume.pdf"); String normalized = textNormalizer.normalize(rawText); assertThat(normalized).contains("Senior Java Engineer"); } @Test void extractsTextFromResumeWithBulletPoints() { String rawText = loadFixture("bulleted-resume.pdf"); String normalized = textNormalizer.normalize(rawText); assertThat(normalized).contains("Built microservices architecture"); } // Both of these were green before the AI refactor and green after it. // Neither one exercises a multi-column layout, so neither one had // any chance of catching the regression that shipped to production. That's the whole problem in one comment. If your test suite doesn't cover multi-column PDF layouts, an AI refactor can happily break multi-column PDF layouts forever and your CI pipeline will show a green checkmark the whole time. Green doesn't mean safe. It means safe according to what we remembered to check. I lean harder on snapshot testing for anything text or data transformation heavy. For the resume-to-PDF rendering pipeline, I use snapshot tests that compare generated output against known-good baselines, and I review diffs manually when something changes. It's not glamorous work, but it has caught more AI-introduced regressions for me than unit tests ever did, because it doesn't rely on me predicting what could go wrong. It just shows me what changed. @Test void resumePdfMatchesApprovedSnapshot() throws Exception { CandidateResume resume = loadFixture("candidate_senior_java.json", CandidateResume.class); byte[] pdfBytes = resumePdfRenderer.render(resume); // Approvals.verify writes a .received file on failure so I can diff // it against the .approved baseline before deciding to update it. Approvals.verify(pdfBytes, Printers.forRawBytes(".pdf")); } I ask AI tools to explain their own changes before I merge them. Sounds obvious, but I skipped this step for months because it felt slow. Now I'll literally prompt: explain what behavior changed in this diff and why. Here's a real one, lightly paraphrased, from a refactor to that same text normalizer a few weeks after the incident: Prompt: Explain what behavior changed in this diff and why. Response: The refactor replaces the manual whitespace collapsing loop with a single regex-based replaceAll call. Functionally equivalent for text with single spaces between words. Note: the new regex also collapses non-breaking spaces and tab characters into single spaces, which the original implementation treated as significant whitespace in multi-column extracted text. That second paragraph is the whole reason I keep asking this question. The AI told me exactly what it changed and exactly why it might matter, and I hadn't asked for that behavior at all. That's usually the moment I go add a test. The false sense of "it looks right" Here's an uncomfortable truth about code review in the AI era: our eyes are getting worse at catching bugs, not better, because the code is getting more readable. Sounds backwards, I know. But think about it. Ugly, inconsistent, hastily written human code triggers your suspicion instinctively. You slow down. You squint at it. Clean, well-formatted, consistently named AI code doesn't trigger that instinct at all. It looks like something a senior engineer wo...