Stop treating prompts like magic spells. I've learned the real unlock is running loops, iterate, check, adjust, repeat, until the output actually holds up. Here's why I stopped writing one shot prompts and started building feedback loops instead.
I spent about three weeks last spring writing what I thought were "good" prompts for the résumé skills extraction pipeline on ITJobOpportunities. Long ones. Carefully worded, with examples, edge cases, formatting instructions, the works. And they worked, sort of, until a résumé came in with a weird PDF export from a design tool and the whole thing fell apart. Missing skills. Garbled job titles. One candidate's "Senior Java Developer" title got mangled into something unrecognizable. That's when I stopped treating prompts like finished products and started treating them like the first iteration in a loop I hadn't built yet. I know that sounds obvious once you say it out loud. But after 25 years in Java and Spring Boot land, prompt engineering culture feels a little foreign at first. We're used to code that either compiles or doesn't, tests that pass or fail, deterministic stuff. Prompts are none of that. A prompt that works great on Monday can quietly degrade after a model update on Wednesday and nobody tells you. So the instinct to "perfect the prompt" and move on is understandable. It's also wrong, at least for anything you're shipping to real users. The prompt-as-artifact trap Here's the pattern I see a lot of engineers fall into, myself included for a while: you write a prompt, test it against three or four examples, it looks great, you ship it, and you consider the job done. The prompt becomes an artifact, like a config file. You set it and forget it. The problem is that a single LLM call is a snapshot of a probabilistic process, not a deterministic function. Run the exact same prompt against the exact same input ten times and you'll get variance, sometimes small, sometimes embarrassing. Add real-world messiness (odd PDF encodings, résumés with tables, non-English job titles, candidates who paste their LinkedIn "About" section verbatim into the skills field) and that variance compounds fast. I learned this the hard way with the Job Fit feature on the platform. Early version: one prompt, one call, parse the JSON response, show a match score. Worked in the demo. Fell apart constantly in production because the model would occasionally return prose instead of JSON, or hedge with "I cannot determine this without more context" instead of just giving a score. Classic. What "running loops" actually means The shift that fixed things wasn't a better prompt. It was accepting that the prompt is just one step in a process that needs the same discipline we'd apply to any distributed system: retries, validation, feedback, and a stopping condition. Concretely, a loop looks something like this: Call the model with an initial prompt and the input. Validate the output against a schema or a set of rules. If it fails validation, don't just retry blindly, feed the failure back into the next call so the model knows what went wrong. Cap the number of iterations (I use 3 for most flows, sometimes 2 for latency-sensitive stuff like Job Fit checks). If you hit the cap without success, fail gracefully. Log it, flag it for a human, don't show garbage to the candidate. That's it. Not fancy. But it's the difference between a demo and something you can put in front of paying customers. Here's roughly what the skill extraction loop looks like on the backend now (simplified, I'm not pasting our actual service code into a LinkedIn article): public SkillExtractionResult extractSkills(String resumeText) { int attempt = 0; String feedback = null; while (attempt < MAX_ATTEMPTS) { String prompt = buildPrompt(resumeText, feedback); String rawResponse = llmClient.complete(prompt); ValidationResult validation = skillSchemaValidator.validate(rawResponse); if (validation.isValid()) { return skillMapper.toResult(rawResponse); } feedback = validation.getErrorSummary(); attempt++; } throw new SkillExtractionFailedException(resumeText, feedback); } Nothing exotic. The interesting part isn't the code, it's the feedback variable getting passed back into buildPrompt . That's the loop actually learning something between iterations instead of hammering the same request and hoping for a different roll of the dice. Why this matters more than prompt wording I'll be honest, I used to obsess over prompt phrasing way more than I should have. "Should I say 'extract' or 'identify'? Should the examples go before or after the instructions?" Some of that matters. But it matters way less than having a validation step that catches garbage before it reaches a candidate or a recruiter. Compare two approaches: Prompt-only approach: spend hours tuning wording, add five examples, add a system message that says "always respond in valid JSON" (narrator: it will not always respond in valid JSON), ship it, hope. Loop approach: write a decent-but-not-perfect prompt, add a schema validator, wrap it in a retry loop with feedback, add a fallback for when it still fails, ship it. The second approach took longer to set up the first time. But I haven't touched the underlying prompt for the resume improvement flow in months, and it still works fine even though we've changed the underlying model routing since, from a single provider call to a primary path with a fallback provider when rate limits got annoying. The loop absorbed the change. The prompt didn't need to be rewritten because the loop was doing the heavy lifting of catching failures, not the wording. The SSE and polling thing nobody tells you about One thing that surprised me building the Job Fit check feature: loops take time, and users hate waiting on a spinner with no feedback. If your loop retries twice with a multi-second model call each time, that's real dead air on the page. We solved this with Server-Sent Events as the primary transport, with polling as a fallback for clients that don't play nice with SSE. The candidate uploads a résumé, sees a progress indicator, and gets partial updates as the loop works through validation. It's not instant. But it doesn't feel like the page is broken either. const eventSource = new EventSource(`/public/job-fit/${jobId}/stream?token=${token}`); eventSource.onmessage = (event) => { const update = JSON.parse(event.data); if (update.status === "scoring") { setProgress("Matching your skills against the role..."); } else if (update.status === "complete") { setResult(update.payload); eventSource.close(); } }; That's a simplified version, but the point stands: if your backend is running a loop instead of a single call, your frontend needs to be honest about that instead of pretending everything happens instantly. When one shot is actually fine I don't want to overcorrect and make it sound like every LLM call needs a retry loop with feedback. That's overkill for a lot of stuff, and it's also money you don't need to spend. If the output is low-stakes and a bad result just means a slightly worse suggestion, like AI-generated skill tag suggestions in our recruiter console, where a human can just edit the tags if the model gets it wrong, a single call with a sane timeout is fine. Don't build machinery you don't need. The line I use: if a bad output reaches an end user without a human checking it first, it needs a loop. If a human is going to glance at it and can easily correct it, a single call is probably fine. Candidate-facing, unreviewed output (Job Fit summary, résumé improvement text): needs a loop, needs validation, needs a fallback. Recruiter-facing, reviewed output (the candidate AI assessment on an application, which is manually triggered and always reviewed before it goes anywhere): still benefits from a loop, but the stakes of a single bad run are lower because a human is in the path. Internal tooling, low stakes (auto-suggesting skill tags for a job posting): a single call with decent error handling is usually enough. Loops cost tokens, so budget for them Here's the part I glossed over for too long: every re...