When back pressure is enabled (default) the upstream operators are blocked
till space is freed up by downstream operators consuming data.
Since bufferserver also provides fault recovery functionality it cannot
immediately clear out the data when it is consumed by the downstream
operators and needs to keep it around till next checkpoints thoughout the
dag and the spillover to disk can come into play if the amount of data
between checkpoints is greater than the in memory buffer capacity.
On Wed, Jun 20, 2018 at 4:41 PM Mateusz Zakarczemny <[EMAIL PROTECTED]>