Queues · Lesson 3

The receive loop: long vs short polling

SQS is pull-based (Lesson 1), so unlike a push broker, you write the loop that asks for work. How you ask — how long you wait each time — decides your bill and your latency. This is the loop in runner.go and dispatcher.go, demystified.

The hidden cost of asking "anything for me?"

A consumer calls ReceiveMessage in a loop. The one knob that matters is WaitTimeSeconds — how long SQS may hold your request open before answering. Two settings, very different behaviour:

Short poll (wait = 0)Long poll (wait = 1–20s)
Answersimmediatelyas soon as a message lands, else at timeout
Servers checkeda sample of SQS's serversall of them
Empty repliesfrequent — even when messages existrare
Requests/min on an idle queuethousands~3
Why short polling stings SQS stores your messages spread across many servers. A short poll checks only a subset, so it can return "nothing here" while messages sit on a server it didn't ask — you re-poll instantly, get another empty, and spin. That's the empty-receive storm: a hot loop of requests that each cost money (you're billed per API request) and CPU, and still add latency because a waiting message might be missed for several rounds. Long polling holds one request open, checks every server, and returns the instant a message arrives.
The rule Default to long polling (wait = 20s, the SQS max). It's cheaper, lower-latency, and AWS's own recommendation. Short polling is a niche tool — you want it only when a single thread must rotate across many queues without parking on any one of them.

Conveyor's two loops — both long-poll

A correction worth making out loud Earlier I told you the per-run queues short-poll. Reading the code says otherwise — and that's exactly the muscle this course is building. The dispatcher is constructed with dispatch.Config{} (all zero values), and withDefaults() turns a zero ReceiveWaitTime into 20s; only a negative value opts into short polling. So both loops long-poll. Trust the code over anyone's summary — including mine.

The control queue (runner.go) is the simple shape: one long-poll receive per cycle, up to 10 messages, fan each out to a goroutine, ack on success.

// runner.go
const longPoll = 20 * time.Second   // "20s minimises empty receives"

for ctx.Err() == nil {
    msgs := receive(ctx, batch=10, wait=longPoll)  // one long poll
    for _, m := range msgs { go handle(m) }         // fan out
}

The per-run task queues (dispatcher.go) are the interesting case: there isn't one queue, there are hundreds (one per live run). Conveyor runs a pool of 100 goroutines; each picks a random active queue and long-polls it for up to 10 messages.

The many-queues wrinkle Long-polling one queue is obviously good. Long-polling hundreds with a fixed pool has a catch: a goroutine that long-polls an empty queue is parked on it for 20s, not helping a busy queue elsewhere. Conveyor manages that with three moves you can see in the code: random pick (no central scheduler; fair in expectation), empty-eviction (a queue empty for 10 min is dropped from the local table, so dead runs aren't polled forever), and a pre-poll gate (skip a queue whose in-flight count already hit the job's limit — don't fetch work you can't run). The pool size (100) is set well above the queue count so parked goroutines never starve the busy queues.

Batching: amortise the round trip

Both loops pass MaxNumberOfMessages = 10 (the SQS cap). One request can return up to ten messages, so you pay one round-trip's latency and cost to get ten units of work. Free throughput — always batch unless you have a reason not to.

SQS

Pull. The polling strategy is your problem — long-poll or you burn money on empty receives.

Kafka / RabbitMQ

The consumer holds an open connection and the broker streams / pushes messages as they arrive. No empty-poll tax — there's nothing to poll.

Redis Streams / NATS

Blocking reads (XREAD BLOCK, JetStream pull with expiry) are basically long polling under another name — same idea, same win.

Read this next

Primary source: AWS · Amazon SQS short and long polling — it explains the server-sampling reason short polls come back empty. Then read internal/worker/runner/runner.go top to bottom (it's ~100 lines) and the dispatchWorker / pickRandom functions in dispatcher.go.

Curious why conveyor runs its own goroutine pool instead of a Lambda SQS event source mapping, or how the random-pick fairness holds up under load? Ask me.

← Lesson 2 · Idempotency Next → DLQs, redrive & maxReceiveCount