Subject: Re: Access to live data of cached dataFrame

  Tomas Bartalos 2019-05-19, 19:33
When you cache a dataframe, you actually cache a logical plan. That's why
re-creating the dataframe doesn't work: Spark finds out the logical plan is
cached and picks the cached data.

You need to uncache the dataframe, or go back to the SQL way:
spark.table("abc").cache() // returns latest data.
spark.table("abc").show // returns cached data.
On Mon, May 20, 2019 at 3:33 AM Tomas Bartalos <[EMAIL PROTECTED]>
  Tomas Bartalos 2019-05-17, 18:44
  Sean Owen 2019-05-17, 18:57