Subject: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions


For store updates, records are first serialized and afterwards put into
the store and written into the changelog topic.

In the current implementation, if the send() into the changelog topic
produces an error and the handler skips over it, the local store content
and the changelog topic diverge. This seems to be a correctness issue.

For serialization error, it would not happen that store and changelog
diverge, because serialization happens before and put/send. Thus, with
this KIP we could skip both put() and send(). However, I am still
wondering, if it would be ok to skip a store update for this case? (Btw:
the current PR does not address this atm, and a serialization error for
a store write would not be covered but kill the instance).

IIRC, the original idea of the KIP was to allow skipping over record for
output topics only. That's why I am wondering if it's ok to allow
skipper over record in repartitions topics, too.

In the end, it's some data loss for all 3 cases, so maybe it's ok to
allow skipping for all 3 cases. However, we should not allow that local
store and changelog topic diverge IMHO (what might been an orthogonal
bug thought).

I also don't have an answer or preference. Just think, it's important to
touch on those cases and get input how people think about it.
-Matthias

On 12/11/18 11:43 AM, Kamal Chandraprakash wrote:
Comment: GPGTools - https://gpgtools.org

iQIzBAEBCgAdFiEESn/iOv2tmCkcP0KLVp2sL37kObwFAlwSWxYACgkQVp2sL37k
ObxQug//UvzvFPGWd58zgbIqtOe860utqu3SzcG3XanEl6xjwAysRLZUyfHpAz0/
pWFS2l1pk69xKegp9H0mKfy+oLdncgqxjby0lXiIeREEEkgxn9rUpi7yjkIK8PUH
n8bqAMqJBWGZvybSibOueyW5bfTjz6x3R9O+/bUDZZ8JSJkRKYmxtxSQZU6fMa+D
30iQUyCJnbPsSX6bu6UIvOgSAoa4Ger4mSETJUE/4C1+2Mi6LDdB3jLUN5fy8ZKM
UuXf6S47wtN4NbJUsYNDE7iYx3iFGXMtXpezI0l8lIwnKvTHkuNgNDwC98n5q4EN
ZAE8rkJjAMs8EANFp9vvFm8v7rWLHPVklgunM812qp42vg5Q02TmhXoNFVqhxUqN
VfabTOG37X5JH1di47C88tVpRCOHoQ2CTfaOyLYpzUAQn2ak0vaNfPLrVQ4A7K+J
51mdVWUyl2M2VSaVJPFABQ4OTJSNgAhD66DQacmR/LHUvb6kZrBw7idLDbzlHWGA
SlR3M6rpMk/3fEJNLJ0C+46x4zuMvxEOGSzkojifEeKvl8Jgjf0wrp0W4H3eGw16
Ts8hy+tn31X8bTNHBwAtUVfIeA7pVbnsfZRYypxvlkVGI//9tpTiVoBxvBM/rSYt
1KlQ78eTmusFHIx5gZkSduaC0cTlu5+sFt6dJ4aFNrhjTuNNv1s=
=rsV/