Everyone loves throwing around "design a system for 1 million notifications" in interviews, but most answers miss the point entirely. Here is my honest take on what that question actually tests, and why the obvious answer usually falls apart under real load.
I saw this system design interview question floating around again last week: how would you send a million notifications at midnight without melting your servers. It's a classic, and honestly, it's a good one. But every time I see someone answer it, they jump straight to "use a queue" like that's the whole answer. It's not. I've built pieces of this exact problem twice — once on a lending platform's batch notification service back in 2017, and again in a smaller form on ITJobOpportunities when we started sending job alert digests. So I've got opinions. Let me walk through how I'd actually think about this, not how I'd whiteboard it for an interviewer. The naive version fails for a boring reason Everybody's first instinct is: loop through a million user IDs, call the notification API for each one. That fails immediately, and not because of some exotic distributed systems reason. It fails because a synchronous loop calling an external API a million times will either time out, throw connection pool exhaustion errors, or take six hours to finish. I've watched an engineer on a past team try exactly this with a plain client and a simple for loop. The service ran fine in staging with 500 test users. In production it choked at around 40,000 and the whole batch job got killed by the orchestrator's timeout. So the real first move isn't "add a queue." It's "stop doing this synchronously, at all." That distinction matters more than people give it credit for. Why I reach for a queue first, but not just any queue Once you accept the work needs to be async, you need somewhere to put a million individual notification jobs. Kafka is the obvious answer, and it's what I'd use if the system already has a Kafka cluster running for other purposes — which, on most teams I've worked with over the last five years, it usually does. But I want to be blunt here: Kafka is overkill if this is the only thing you're doing with it. Standing up a cluster just to fan out notification jobs is a lot of operational weight for something SQS or even RabbitMQ can handle just as well. On a document ingestion pipeline I worked on years ago, we used SNS fanning out to SQS queues, and it worked beautifully for exactly this kind of "distribute work, don't care about ordering, need retries" pattern. I still think SNS-to-SQS is underrated for notification fan-out. People jump to Kafka because it's the trendy answer in interviews, but SQS with dead-letter queues gives you retry semantics almost for free. Here's roughly how I'd structure the producer side: @Service public class NotificationDispatchService { private final SqsTemplate sqsTemplate; private final UserRepository userRepository; public void dispatchCampaign(String campaignId) { userRepository.findActiveUserIdsStream() .forEach(userId -> { NotificationJob job = new NotificationJob(campaignId, userId); sqsTemplate.send("notification-jobs-queue", job); }); } } Notice I'm streaming user IDs rather than pulling a million rows into memory. That's a small thing, but it's the kind of small thing that separates "works in the demo" from "works at all night when marketing kicks off a big campaign." Rate limiting is where the interesting decisions actually live This is the part most answers skip, and it's the part I actually care about. Once you've got a queue full of jobs, the real problem isn't "can I enqueue a million messages" — SQS and Kafka both handle that without blinking. The real problem is your downstream notification provider (Twilio, SendGrid, FCM, whatever) has rate limits, and so does your own database if every notification write also updates a delivery status column. I've seen two approaches work well: Token bucket at the consumer level. Each worker pulls from the queue but only processes N jobs per second, using something like Guava's RateLimiter or a Redis-backed sliding window if you've got multiple worker instances and need a shared limit. Simple, and it's what I'd default to. Batching at the provider call. A lot of push notification providers (FCM in particular) support batch sends of up to 500 tokens per call. If you're not batching, you're making a million HTTP calls when you could make two thousand. This one is easy to miss because it's not a "system design" concept — it's just knowing the provider's API docs. Honestly, the batching one bugs me every time I see someone skip it in these interview answers. It's the highest-leverage optimization and the most concrete, yet most write-ups go straight to "add exponential backoff" instead. Backoff matters, sure, but batching changes your call count by two orders of magnitude before you even need backoff. Here's the rough shape of a rate-limited consumer, using a Redis-backed limiter since most of my recent work runs multiple worker pods behind EKS: @SqsListener("notification-jobs-queue") public void handleNotificationJob(NotificationJob job) { if (!rateLimiter.tryAcquire("provider:fcm")) { // requeue with a short delay instead of blocking the thread sqsTemplate.sendWithDelay("notification-jobs-queue", job, Duration.ofSeconds(2)); return; } notificationProvider.send(job); deliveryStatusRepository.markSent(job.getId()); } That requeue-with-delay trick isn't glamorous, but it keeps workers from spinning and burning CPU on Thread.sleep . I picked up this pattern on a hospital microservices program, where we had a similar problem with a downstream lab-results API that rate-limited us hard. Different domain, same shape of problem. The database write pattern nobody mentions until it bites them Here's something that caught us on ITJobOpportunities, and it wasn't even the notification sending itself — it was the delivery status tracking. The founder side of me (which is most of my hats these days) wanted to know: did this candidate actually get the job alert email? Simple ask. Except now every notification job does a database write to update a status column, and if you've got a million of those firing inside a short window, you've turned your notification problem into a database write-contention problem. The fix we landed on was batching status updates instead of writing them one at a time. Workers accumulate delivery results in memory (or in a Redis list, if you want durability across worker restarts) and flush every 500 records or every 5 seconds, whichever comes first. It cut our write load from thousands of tiny UPDATE statements to a manageable number of bulk upserts. On ITJobOpportunities that mattered less at our scale — we're nowhere near a million notifications a day — but the pattern holds regardless of scale. PostgreSQL, which is what I run everywhere, handles bulk upserts through ON CONFLICT a lot better than it handles a firehose of single-row updates. INSERT INTO notification_status (job_id, status, sent_at) VALUES (?, ?, ?) ON CONFLICT (job_id) DO UPDATE SET status = EXCLUDED.status, sent_at = EXCLUDED.sent_at; Batch these into groups of a few hundred and you've solved most of your write pressure without touching your queueing infrastructure at all. What happens when a worker crashes halfway through This is the question I'd expect a strong candidate to bring up unprompted, and it's the one that separates "read a blog post about queues" from "has actually operated one of these systems." If a worker dies mid-batch, what happens to the notifications it hadn't processed yet? With SQS, you get this mostly for free through visibility timeouts. If a worker picks up a message and doesn't ack it within the visibility window, the message becomes available again for another worker to grab. Set the timeout too short and you get duplicate sends. Set it too long and a genuinely stuck job sits there for ages before anyone notices. I usually start around 30 seconds for something like a notification send and tune from there based on actual provider latency, not guessw...