clear query| facets| time Search criteria: .   Results from 1 to 10 from 23877 (0.0s).
Loading phrases to help you
refine your search...
[expand - 1 more] - [SPARK-SQL] How to return GenericInternalRow from spark udf - Spark - [mail # user]
...The UDF should return the result value you want, not a whole Row. InScala it figures out the schema of the UDF's result from thesignature.On Thu, Aug 6, 2020 at 7:56 AM Amit Joshi  wrot...
   Author: Sean Owen , Amit Joshi , ... , 2020-08-06, 14:48
join doesn't work - Spark - [mail # user]
...I've using streamline pulsar connector, each dataset receives the dataproperly but cannot make join to be workingDataset datasetPolicyWithWtm =datasetPolicy.withWatermark("__publishTime", "5...
   Author: nt , 2020-08-06, 11:54
[expand - 3 more] - S3 read/write from PySpark - Spark - [mail # user]
...Hi,Thanks for your help. Problem solved, but I thought I should add somethingin case this problem is encountered by others.Both responses are correct; BasicAWSCredentialsProvider is gone, bu...
   Author: Daniel Stojanov , Stephen Coy , ... , 2020-08-06, 08:07
Understanding Spark execution plans - Spark - [mail # user]
...Hi,When an execution plan is printed it lists the tree of operations that willbe completed when the job is run. The tasks have somewhat cryptic names ofthe sort: BroadcastHashJoin, Project, ...
   Author: Daniel Stojanov , 2020-08-06, 02:51
Multi insert with join in Spark SQL - Spark - [mail # user]
...Hi,I am trying to migrate Hive SQL to Spark SQL. When I execute the Multiinsert with join statement, Spark SQL will scan the same table multipletimes, while Hive SQL will only scan once. In ...
   Author: moqi , 2020-08-06, 02:08
[expand - 5 more] - Tab delimited csv import and empty columns - Spark - [mail # user]
...Hi Sean, German and others,Setting the “nullValue” option (for parsing CSV at least) seems to be an exercise in futility.When parsing the file, com.univocity.parsers.common.input.AbstractCha...
   Author: Stephen Coy , Vladimir Ryzhov , ... , 2020-08-06, 01:00
[expand - 1 more] - Comments conventions in Spark distribution official examples - Spark - [mail # user]
...These only matter to our documentation, which includes the source ofthese examples inline in the docs. For brevity, the examples don'tneed to show all the imports that are otherwise necessar...
   Author: Sean Owen , Fuad Efendi , ... , 2020-08-06, 00:29
Async API to save RDDs? - Spark - [mail # user]
...Hi,The RDD API provides async variants of a few RDD methods, which let theuser execute the corresponding jobs asynchronously. This makes itpossible to cancel the jobs for instance:https://sp...
   Author: Antonin Delpeuch , 2020-08-05, 11:27
file importing / hibernate - Spark - [mail # user]
...1. I need to import csv files with a entity resolution logic, spark couldhelp me to process rows in parallelDo you think is a good approach ?2. I've quite complex database structure and eage...
   Author: nt , 2020-08-05, 10:18
[expand - 2 more] - Renaming a DataFrame column makes Spark lose partitioning information - Spark - [mail # user]
...Well that's great ! Thank you very much :)AntoineOn Tue, Aug 4, 2020 at 11:22 PM Terry Kim  wrote:> This is fixed in Spark 3.0 by https://github.com/apache/spark/pull/26943:>> ...
   Author: Antoine Wendlinger , Terry Kim , ... , 2020-08-05, 07:53