Subject: Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams


Hello Jason,

I agree with you that for range / round-robin it makes less sense to be
compatible with cooperative rebalance protocol.

As for StickyAssignor, however, I think it would still be possible to make
the current implementation to be compatible with cooperative rebalance. So
after pondering on different options at hand I'm now proposing this
approach as listed in the upgrade section:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol#KIP-429:KafkaConsumerIncrementalRebalanceProtocol-CompatibilityandUpgradePath

The idea is to let assignors specify which protocols it would work with,
associating with a different name; then the upgrade path would involve a
"compatible" protocol which actually still use eager behavior while
encoding two assignors if possible. In "Rejected Section" (just to clarify
I'm not finalizing it as rejected, just putting it there for now, and if we
like this one instead we can always switch them) I listed the other
approach we once discussed about, and arguing its cons of duplicated class
seems overwhelm the pros of saving the  "rebalance.protocol" config.

Let me know WDYT.

Guozhang

On Fri, Apr 12, 2019 at 6:08 PM Jason Gustafson <[EMAIL PROTECTED]> wrote: