Adam, I agree, seems reasonable to limit the broker's responsibility to
encrypting only data at rest. I guess whole segment files could be
encrypted with the same key, and rotating keys would just involve
re-encrypting entire segments. Maybe a key rotation would involve closing
all affected segments and kicking off a background task to re-encrypt them.
Certainly that would not impede ingestion of new records, and seems
consumers could use the old segments until they are replaced with the newly
Seems that could still get us per-topic keys (vs encrypting the entire
volume), which would be my main requirement.
Not really "end-to-end", but combined with TLS or something, seems
On Sat, May 9, 2020, 11:00 AM Adam Bellemare <[EMAIL PROTECTED]>