Subject: Re: [DISCUSS] KIP-429 : Smooth Auto-Scaling for Kafka Streams

  Matthias J. Sax 2019-05-09, 13:21
  Guozhang Wang 2019-05-09, 23:11
  Guozhang Wang 2019-05-09, 01:37
  Guozhang Wang 2019-05-09, 01:36
  Guozhang Wang 2019-04-12, 00:53
  Guozhang Wang 2019-04-12, 00:23
  Guozhang Wang 2019-04-15, 01:21
Hello Matthias,

On Sat, Apr 13, 2019 at 3:24 PM Matthias J. Sax <[EMAIL PROTECTED]>
But for this case I agree with you after a second thinking. Will update
request hence trigger a new rebalance; on other members, they will not send
join-group but just relying on a single member to make that decision --- it
is to avoid split brain: the assignment logic residing on the leader is
dependent on the leader's metadata; this metadata could be different with
other members. I'm concerned to let "anyone" triggering a rebalance and use
its own metadata to decide what topic-partitions are still available, and
would prefer to just leave it to a single member, i.e. the leader to make
that decision.

On the other hand, if 1) a non-member think a partition no long exist but
leader think it still exist, it may still get it assigned, which is fine
since it will not be able to fetch anything with no metadata anyways; 2) if
a non-member a partition still exist but the leader think it does not exist
anymore, leader will just "give it back" to the non-member, which is also
fine since it will not be able to fetch anything if the metadata did not
include this partition. The bottom line is, the metadata of all clients
will be up-to-date eventually so I'm leaning towards this simplicity over
efficiencies with potential optimization.
since a rebalance "may" have been missed and this client is no longer in
the group.
this section as well.

upgrading, will reply to Jason in this thread.

  Jason Gustafson 2019-04-10, 23:03
  Jason Gustafson 2019-04-13, 01:08
  Matthias J. Sax 2019-04-13, 22:24
  Boyang Chen 2019-03-26, 17:34
  Guozhang Wang 2019-03-30, 01:20
  Guozhang Wang 2019-03-21, 01:23
  Matthias J. Sax 2019-04-08, 23:47
  Boyang Chen 2019-04-09, 04:39
  Boyang Chen 2019-05-01, 23:51