clear query| facets| time Search criteria: .   Results from 1 to 10 from 23311 (0.0s).
Loading phrases to help you
refine your search...
[expand - 1 more] - [Announcement] Analytics Zoo 0.7.0 release - Spark - [mail # user]
...Fixed one typo below: should be TensorFlow 1.15 support in tfpark :-)On Tue, Jan 21, 2020 at 7:52 AM Jason Dai  wrote:> Hi all,>>>> We are happy to announce the 0.7.0 rel...
   Author: Jason Dai , 2020-01-21, 02:06
[expand - 2 more] - Extract value from streaming Dataframe to a variable - Spark - [mail # user]
...Hi,you can try out foreachBatch to apply the batch query operation to the eachoutput of micro-batch:http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#using-fore...
   Author: Jungtaek Lim , Nick Dawes , ... , 2020-01-20, 22:09
[expand - 4 more] - RESTful Operations - Spark - [mail # user]
...I see. Thanks for the advice! Jan 20, 2020, 09:47 by [EMAIL PROTECTED]:> Livy has both statement based(scala) as well as batch processing(code jar packaged). I think first statement based...
   Author: hamishberridge@... , hemant singh , ... , 2020-01-20, 10:41
[expand - 2 more] - Record count query parallel processing in databricks spark delta lake - Spark - [mail # user]
...Thank you Farhan so much for the help.please help me on the design approach of this problem.what is the best wayto achieve this code to get the results better.I have some clarification on th...
   Author: anbutech , Farhan Misarwala , ... , 2020-01-20, 09:24
[expand - 3 more] - Does explode lead to more usage of memory - Spark - [mail # user]
...Depends on the use case, if you have to join, you're saving a join and ashuffle from having it already in an array.If you explode, at least sort within partitions to get you predicatepushdow...
   Author: Chris Teoh , Jörn Franke , ... , 2020-01-19, 11:49
[expand - 1 more] - How to implement "getPreferredLocations" in Data source v2? - Spark - [mail # user]
...https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/InputPartition.javaSee InputPartition which had a preferred location parameter you ...
   Author: Russell Spitzer , kineret M , ... , 2020-01-18, 21:37
How to prevent and track data loss/dropped due to watermark during structure streaming aggregation - Spark - [mail # user]
...We have a scenario to group raw records by correlation id every 3 minutes andappend groupped result to some HDFS store, below is an example of our query    val df= records.readStre...
   Author: stevech.hu , 2020-01-18, 08:57
[expand - 7 more] - Spark Executor OOMs when writing Parquet - Spark - [mail # user]
...Yes. Disk spill can be a huge performance hit, with smaller partitions youmay avoid this and possibly complete your job faster. I hope you don't getOOM.On Sat, 18 Jan 2020 at 10:06, Arwin Ti...
   Author: Chris Teoh , Arwin Tio , ... , 2020-01-18, 02:15
Is there a way to get the final web URL from an active Spark context - Spark - [mail # user]
...Given a session/context, we can get the UI web URL like this:sparkSession.sparkContext.uiWebUrlThis gives me something like http://node-name.cluster-name:4040.  Ifopening this from outs...
   Author: Jeff Evans , 2020-01-17, 22:10
unsubscribe - Spark - [mail # user]
...    ---------------------------------------------------------------------To unsubscribe e-mail: [EMAIL PROTECTED]...
   Author: Bruno S. de Barros , 2020-01-17, 19:39