1d 1w 1m 1y all
clear query| facets| time Search criteria: author:"Tathagata Das".   Results from 41 to 50 from 1165 (0.0s).
Loading phrases to help you
refine your search...
[expand - 1 more] - Nullpointerexception error when in repartition - Spark - [mail # user]
...It's not very surprising that doing this sort of RDD to DF conversioninside DStream.foreachRDD has weird corner cases like this. In fact, youare going to have additional problems with partia...
   Author: Tathagata Das , 2018-04-12, 07:09
Structured Streaming on Kubernetes - Spark - [mail # user]
...Structured streaming is stable in production! At Databricks, we and ourcustomers collectively process almost 100s of billions of records per dayusing SS. However, we are not using kubernetes...
   Author: Tathagata Das , 2018-04-13, 08:27
Does partition by and order by works only in stateful case? - Spark - [mail # user]
...The traditional SQL windows with `over` is not supported in streaming. Onlytime-based windows, that is, `window("timestamp", "10 minutes")` issupported in streaming.On Thu, Apr 12, 2018 at 7...
   Author: Tathagata Das , 2018-04-13, 02:53
can we use mapGroupsWithState in raw sql? - Spark - [mail # user]
...Unfortunately no. Honestly it does not make sense as for type-awareoperations like map, mapGroups, etc., you have to provide an actual JVMfunction. That does not fit in with the SQL language...
   Author: Tathagata Das , 2018-04-17, 03:33
How to read json data from kafka and store to hdfs with spark structued streaming? - Spark - [mail # user]
...Are you writing multiple streaming query output to the same location? Ifso, I can see this error occurring. Multiple streaming queries writing tothe same directory is not supported.On Tue, J...
   Author: Tathagata Das , 2018-07-27, 00:39
Exceptions with simplest Structured Streaming example - Spark - [mail # user]
...Unfortunately, your output is not visible in the email that we see. Was itan image that some got removed?Maybe best to copy the output text (i.e. the error message) into the email.On Thu, Ju...
   Author: Tathagata Das , 2018-07-26, 22:49
How to execute non-timestamp-based aggregations in spark structured streaming? - Spark - [mail # user]
...SQL windows with the 'over' syntax does not work in Structured Streaming.It is very hard to incrementalize that in the general case. Hence non-timewindows are not supported.On Sat, Apr 20, 2...
   Author: Tathagata Das , 2019-04-22, 17:11
Spark streaming join on yarn - Spark - [mail # user]
...How many tasks in the stage 2? How long do they take? If there are 200tasks taking 1 second each (so many "rounds" of tasks on available corestaking 13 seconds), then you can reduce the numb...
   Author: Tathagata Das , 2018-11-29, 01:32
Spark Streaming ZeroMQ Java Example - Spark - [mail # user]
...Unfortunately there isnt one right now. But it is probably too hard to start with the JavaNetworkWordCount, and use the ZeroMQUtils in the same way as the Scala ZeroMQWordCount example. Basi...
   Author: Tathagata Das , 2014-03-26, 01:29
[expand - 1 more] - [bug?] streaming window unexpected behaviour - Spark - [mail # user]
...You can probably do it in a simpler but sort of hacky way!  If your window size is W and sliding interval S, you can do some math to figure out how many of the first windows are actuall...
   Author: Tathagata Das , 2014-03-26, 02:15

Products

Services

About

Contact

© Sematext Group. All rights reserved
Terms Of Service · Privacy Policy

Apache Lucene, Apache Solr and their respective logos are trademarks of the Apache Software Foundation. Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. Sematext Group, Inc. is not affiliated with Elasticsearch BV. Some of the icons, illustrations, and pictures used on this website are made by Flaticon, Flickr, and Freepik and their contributors. They are licensed by CC 3.0 BY and are used as originals or their derivatives.