Subject: Process for changing producer partition assignment strategy


I've set up a POC using KafkaStreams with microservices consuming and
producing from/to topics. In the beginning I hadn't thought about
partition strategy, and so I was using the DefaultPartitioner for producer
partition assignments. My messages have keys (I use these for
forking/joining), and the keys are time based UUIDs, this causes some
rather uneven distribution on my topics. I looked around google and
stumbled on KIP-369 (Alternative Partitioner to Support "Always
Round-Robin" Selection) and figured that would be what I needed, so since
2.4 isn't out yet I borrowed the class from the PR on github, added it to
my project and added the property to my config, like so:

streamProperties.put(ProducerConfig.PARTITIONER_CLASS_CONFIG,
RoundRobinPartitioner.class.getCanonicalName());
And the round robin strategy works on a newly added topic, spreading
messages evenly over 4 partitions. But, and I'm finally getting to my
question, it doesn't seem to have any effect on existing topics, in other
words, it seems to be continuing to use the DefaultPartitioner for topics
created before I added the RoundRobinPartioner class to my
project/properties.

Is it me that just hasn't understood that it is impossible to change
strategy for an existing partition or do I have to do something specific
apart from re-deploying the Microservice containing the producer?

Thanks
Mikkel

--