Kafka Consumer Graceful Shutdown: Handle WakeupException and Commit Offsets Safely

In the previous article, we looked at how rebalancing works and why consumers pause during deployments.

Now let’s look at something that causes even more real production issues:

How do you gracefully shut down a Kafka consumer without losing work?

In real systems:

  • Pods restart
  • Deployments happen
  • Servers receive SIGTERM
  • Auto-scaling removes instances

If shutdown is not handled properly, you can:

  • Reprocess messages
  • Lose uncommitted offsets
  • Cause inconsistent state
  • Trigger unnecessary rebalances

Kafka does not handle this automatically for you.
You have to shut down the consumer correctly.

Why Graceful Shutdown Matters

A typical Kafka consumer runs inside a loop:

while (true) {
    ConsumerRecords<String, String> records =
        consumer.poll(Duration.ofMillis(100));

    for (ConsumerRecord<String, String> record : records) {
        process(record);
    }
}

Now imagine the application is terminated while:

  • Records are already processed
  • Offsets are not committed yet

When the consumer restarts, Kafka will deliver those records again.

That may be acceptable.

But in many systems, it creates duplicate writes, duplicate payments, or inconsistent updates.

A proper Kafka consumer graceful shutdown should:

  1. Stop polling new records
  2. Finish processing current records
  3. Commit offsets safely
  4. Close the consumer cleanly

The Main Problem: poll() Blocks

The poll() method blocks while waiting for records.

If your application receives a shutdown signal while the thread is inside poll(), how do you stop it?

You should not kill the thread.

Kafka provides a safe mechanism for this.

WakeupException in Kafka

Kafka provides this method:

consumer.wakeup();

When called from another thread, it interrupts the blocking poll() call.

Inside the consumer thread, poll() throws:

WakeupException

This is the official and safe way to interrupt a Kafka consumer.

Nothing else should be used.

Correct Kafka Consumer Graceful Shutdown Pattern

Let’s build this properly.

Step 1: Add a running flag

private volatile boolean running = true;

Step 2: Consumer implementation

public class GracefulKafkaConsumer implements Runnable {

    private final KafkaConsumer<String, String> consumer;
    private volatile boolean running = true;

    public GracefulKafkaConsumer(KafkaConsumer<String, String> consumer) {
        this.consumer = consumer;
    }

    @Override
    public void run() {
        try {
            consumer.subscribe(List.of("my-topic"));

            while (running) {
                ConsumerRecords<String, String> records =
                    consumer.poll(Duration.ofMillis(100));

                for (ConsumerRecord<String, String> record : records) {
                    process(record);
                }

                // Commit after processing
                consumer.commitSync();
            }

        } catch (WakeupException e) {
            // Expected during shutdown
            if (running) {
                throw e;
            }
        } finally {
            try {
                consumer.commitSync(); // final safety commit
            } finally {
                consumer.close(); // leave group cleanly
            }
        }
    }

    public void shutdown() {
        running = false;
        consumer.wakeup();
    }

    private void process(ConsumerRecord<String, String> record) {
        System.out.println("Processing: " + record.value());
    }
}

What Is Happening Here?

  • running controls the loop.
  • consumer.wakeup() interrupts poll().
  • WakeupException is expected during shutdown.
  • Offsets are committed after processing.
  • consumer.close() triggers proper group leave.

This ensures offsets are committed safely before exit.

Why Not Just Kill the Thread?

If you forcefully stop the thread:

  • Offsets may not be committed
  • The consumer may not leave the group cleanly
  • Rebalances may be delayed
  • Duplicate processing may increase

Always use consumer.wakeup() for graceful shutdown.

Manual Commit vs Auto Commit

If you are using:

enable.auto.commit=true

Offsets are committed periodically.

But auto commit does not guarantee that processing is finished before commit.

For production systems, it is usually safer to use:

enable.auto.commit=false

And commit offsets only after successful processing.

If you want a deeper explanation of offset commits, I covered it here:

👉 Kafka Auto Commit Explained (At-Least-Once Processing)

Offset management and graceful shutdown are closely connected.

A Quick Note on Long Processing

If your message processing takes a long time, Kafka may trigger a rebalance even if the consumer is still running.

This usually relates to how frequently poll() is called and certain consumer configurations.

This topic deserves a separate discussion because it directly impacts stability in high-throughput systems.

We’ll explore it properly in a dedicated article.

Connecting Shutdown to JVM Exit

You can connect shutdown logic like this:

GracefulKafkaConsumer consumerRunnable =
    new GracefulKafkaConsumer(consumer);

Thread consumerThread = new Thread(consumerRunnable);
consumerThread.start();
Thread mainThread = Thread.currentThread();

Runtime.getRuntime().addShutdownHook(new Thread(() -> {
    consumerRunnable.shutdown();

    try {
            mainThread.join();
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
}));

When the application receives a termination signal, the consumer exits safely.

How This Relates to Rebalancing

In the previous article(👉 Read: Kafka Eager vs Cooperative Rebalancing Explained), we discussed eager vs cooperative rebalancing.

Improper shutdown can trigger unnecessary rebalances.

A clean shutdown:

  • Reduces duplicate processing
  • Leaves the consumer group smoothly
  • Minimizes disruption during deployments

Graceful shutdown is not optional in production systems.

It is part of building reliable Kafka consumers.

Closing Thoughts

Kafka consumer graceful shutdown is simple in concept, but easy to get wrong.

The correct approach is:

  • Stop polling
  • Finish processing
  • Commit offsets safely
  • Close the consumer properly

If you skip any of these steps, you increase the chance of duplicate processing or inconsistent state.

What’s Next?

Next, we’ll explore Kafka producer internals — specifically how acks, retries, and idempotent producers impact reliability and delivery guarantees in production systems.

If you found this useful and want to share your thoughts, this article is also published on Dev.to where discussions are more active. You can read it there and leave a comment if you’d like:

https://dev.to/rajeev_a954661bb78eb9797f/kafka-consumer-graceful-shutdown-handle-wakeupexception-and-commit-offsets-safely-2a93

I always appreciate feedback and different perspectives.