Subject: [DISCUSS] KIP-399: Extend ProductionExceptionHandler to cover serialization exceptions


Thanks for picking up this KIP.

My personal take about repartition topics is, that it seems to be ok to
apply the handler for those, too (in addition to output topics). It
seems to be more flexible and also simplifies the code. In the end, the
topic name is passed via `ProducerRecord` into the handler and thus
users can decide on a per-topic basis what to do.

About stores and changelogs: yes, serialization happens first. Hence,
when we put() into RocksDB and also send() to the changelog topic (in
that case we use `ByteArraySerializer`) no serialization error should
happen (if there would have been a problem, it would have happened earlier).

However, in KIP-210, we did not consider the case that a send() might
fail for a changelog topic while the put() into the store was already
successfully applies. Hence, it's possible atm, to skip a failed write
into the changelog topic, even if the put() into the store was
successful. This seems to be a bug to me and we might want to create a
separate Jira for it -- it's related to this KIP but should not be
mangled into the KIP IMHO.

For the KIP itself, what we _could_ do is, to apply the handler to the
serialization that happens before the data is put() into the store.
However, I am not sure if we should allow this -- atm, I tend to think
we should not allow it and exclude store serialization for the handler.

-Matthias

On 9/10/19 1:46 AM, Alaa Zbair wrote:
Comment: GPGTools - https://gpgtools.org

iQIzBAEBCgAdFiEE8osu2CcCCF5douGQu8PBaGu5w1EFAl2Be2QACgkQu8PBaGu5
w1Ef/hAAqPZ2Qju1sLhyWKTW5ko1FsRu8czC+Abk6B4txWLtryZhZ7hyarkFZF0X
/PU/iWpE7MQKQy3j0iKQdB3Op+/eI3NzX84FHP5IY61gttIlM1fUVYvu8lTgp4cP
cH/dN2WHoRZm2Zu+yd6PNjcUPVI8a+Cl18fnZG65j0Il64A31cKqRkXAlWHoCpEP
Vz0oD/yG+Ed/x0qLqzDlKw4YpqpV6lK5ctNx7p67EoWakiDY6BjUI9lwcYxvm0W8
5k/P370vh+6CgNR6I9t+Fvs9+7AdhJU+o6hGEcqV77Pqm8pZw2FwJ5TJTxa9uD4v
xKlJ9QBQ+ObvD+eiGTEEt3JlftBoSOYMoe0Wbd1cnrKtQPoNFxMtSiLvM/E7RjKl
PJiQ5Yujv2d3XMg0R6VWxUWDJo3QfD8iXb/n+eT+upvAhZ6I1vShhgeG/R2G8bpR
wZHv1yCZxzY/Lw+AsiTvVIbfsGHORgLZVFF+HJsm7MkeJvUjoRW0CAf8UGKvpmuu
bWyFxS4/dktJaevj/aMBJ/CIDDDCkC8ttNN7+6QPuMxSKOQ8AvtT7CA44Zyd2FOi
kaj1eAz4YDyzmo6cmjpIAKnN9G0HpU9BAXswYN3uy2y4oGAyvh1lIkJ1u0ODnpNY
GFki3SaZbZo7sd0RNUFg3f2Tbr7OvHQrDlfZq0XD5MXA6XvFsJo=
=Rw+/