Subject: [DISCUSS] KIP-578: Add configuration to limit number of partitions

Thanks for the KIP, Gokul!

I like the overall premise - I think it's more user-friendly to have
configs for this than to have users implement their own config policy -> so
unless it's very complex to implement, it seems worth it.
I agree that having the topic policy on the CreatePartitions path makes
sense as well.

Multi-tenancy was a good point. It would be interesting to see how easy it
is to extend the max partition limit to a per-user basis. Perhaps this can
be done in a follow-up KIP, as a natural extension of the feature.

I'm wondering whether there's a need to enforce this on internal topics,
though. Given they're internal and critical to the function of Kafka, I
believe we'd rather always ensure they're created, regardless if over some
user-set limit. It brings up the question of forward compatibility - what
happens if a user's cluster is at the maximum partition capacity, yet a new
release of Kafka introduces a new topic (e.g KIP-500)?


On Fri, Apr 24, 2020 at 2:39 PM Gokul Ramanan Subramanian <