Subject: [DISCUSS] KIP-317 - Add end-to-end data encryption functionality to Apache Kafka


Adam, I agree, seems reasonable to limit the broker's responsibility to
encrypting only data at rest. I guess whole segment files could be
encrypted with the same key, and rotating keys would just involve
re-encrypting entire segments. Maybe a key rotation would involve closing
all affected segments and kicking off a background task to re-encrypt them.
Certainly that would not impede ingestion of new records, and seems
consumers could use the old segments until they are replaced with the newly
encrypted ones.

Seems that could still get us per-topic keys (vs encrypting the entire
volume), which would be my main requirement.

Not really "end-to-end", but combined with TLS or something, seems
reasonable.

Ryanne

On Sat, May 9, 2020, 11:00 AM Adam Bellemare <[EMAIL PROTECTED]>
wrote: