I'd like to know if anyone has a magic method to avoid such messages in
2015-08-30 19:30:44 ERROR LiveListenerBus:75 - SparkListenerBus has already
stopped! Dropping event
After further investigations, I understand that LiveListenerBus extends
AsynchronousListenerBus. And thus, at some point, .stop() method is called.
Then, messages that might be sent/received will be dropped and remain
unprocessed. Basically, some SparkListenerExecutorMetricsUpdate messages
are unfortunately not received yet, and once they are, they become dropped
This doesn't look critical since SparkListenerExecutorMetricsUpdate only
corresponds to Periodic updates from executors.
What is embarrassing is that I absolutely don't understand why this happens
and nothings refers to this issue. Note that this is totally
non-deterministic and I can't reproduce this, probably due to the
asynchronous nature and my lack of understand on how/when stop() is
supposed to be called.
Head of Backend/Infrastructure
50, avenue Montaigne - 75008 Paris