Subject: Spark structured streaming: periodically refresh static data frame

1. Just loop like this.
def startQuery(): Streaming Query = {
   // Define the dataframes and start the query

// call this on main thread
while (notShutdown) {
   val query = startQuery()
   // refresh static data
2. Yes, stream-stream joins in 2.3.0, soon to be released. RC3 is available
if you want to test it right now -

On Wed, Feb 14, 2018 at 3:34 AM, Appu K <[EMAIL PROTECTED]> wrote: