clear query| facets| time Search criteria: author:"Tathagata Das".   Results from 1 to 8 from 8 (0.0s).
Loading phrases to help you
refine your search...
Announcing Delta Lake 0.2.0 - Spark - [mail # user]
...@ayan guha  @Gourav SenguptaDelta Lake is OSS currently does not support defining tables in Hivemetastore using DDL commands. We are hoping to add the necessarycompatibility fixes in Ap...
   Author: Tathagata Das , 2019-06-21, 10:03
How to execute non-timestamp-based aggregations in spark structured streaming? - Spark - [mail # user]
...SQL windows with the 'over' syntax does not work in Structured Streaming.It is very hard to incrementalize that in the general case. Hence non-timewindows are not supported.On Sat, Apr 20, 2...
   Author: Tathagata Das , 2019-04-22, 17:11
Spark streaming join on yarn - Spark - [mail # user]
...How many tasks in the stage 2? How long do they take? If there are 200tasks taking 1 second each (so many "rounds" of tasks on available corestaking 13 seconds), then you can reduce the numb...
   Author: Tathagata Das , 2018-11-29, 01:32
Announcing Delta Lake 0.3.0 - Spark - [mail # user]
...Hello everyone,We are excited to announce the availability of Delta Lake 0.3.0 whichintroduces new programmatic APIs for manipulating and managing data inDelta Lake tables.Here are the main ...
   Author: Tathagata Das , 2019-08-02, 01:45
Structured Streaming: How to add a listener for when a batch is complete - Spark - [mail # user]
...https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#reporting-metrics-programmatically-using-asynchronous-apisOn Tue, Sep 3, 2019, 3:26 PM Natalie Ruiz wrot...
   Author: Tathagata Das , 2019-09-04, 01:28
[expand - 1 more] - Structured Streaming Dataframe Size - Spark - [mail # user]
...https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#basic-concepts*Note that Structured Streaming does not materialize the entire table*. It> reads the latest...
   Author: Tathagata Das , 2019-08-27, 22:42
how can I dynamic parse json in kafka when using Structured Streaming - Spark - [mail # user]
...You can use *from_json* built-in SQL function to parse json.https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/functions.html#from_json-org.apache.spark.sql.Column-org.apache...
   Author: Tathagata Das , 2019-09-17, 08:13