clear query| facets| time Search criteria: .   Results from 1 to 10 from 3899 (0.0s).
Loading phrases to help you
refine your search...
Starting work on last Scala 2.13 updates - Spark - [mail # dev]
...I call attention to https://github.com/apache/spark/pull/28971 whichrepresents part 1 of several changes that are the last large changefor Scala 2.13, except for REPL updates: dealing with t...
   Author: Sean Owen , 2020-07-11, 18:41
Strange WholeStageCodegen UI values - Spark - [mail # user]
...It sounds like you have huge data skew?On Thu, Jul 9, 2020 at 4:15 PM Bobby Evans  wrote:>> Sadly there isn't a lot you can do to fix this.  All of the operations take iterat...
   Author: Sean Owen , 2020-07-09, 21:47
[Spark 3.0 Kubernetes] Does Spark 3.0 support production deployment - Spark - [mail # user]
...I haven't used the K8S scheduler personally, but, just based on thatcomment I wouldn't worry too much. It's been around for severalversions and AFAIK works fine in general. We sometimes aren...
   Author: Sean Owen , 2020-07-09, 20:20
com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0 - Spark - [mail # user]
...You have a Jackson version conflict somewhere. It might be from otherlibraries you include in your application.I am not sure Spark 2.3 works with Hadoop 3.1, so this may be theissue. Make su...
   Author: Sean Owen , 2020-07-09, 13:30
Issue on SPARK 3.0.0 loading MultilayerPerceptronClassificationModel - Spark - [mail # dev]
...Yeah that's a bug, I can reproduce it. Can you open a JIRA?It works in Scala, so must be an issue with the Python wrapper. Theserialized model is fine; it's loading it back.I think it's beca...
   Author: Sean Owen , 2020-07-08, 15:45
When does SparkContext.defaultParallelism have the correct value? - Spark - [mail # user]
...If not set explicitly with spark.default.parallelism, it will defaultto the number of cores currently available (minimum 2). At the verystart, some executors haven't completed registering, w...
   Author: Sean Owen , 2020-07-07, 14:07
Is it possible to use Hadoop 3.x and Hive 3.x using spark 2.4? - Spark - [mail # user]
...2.4 works with Hadoop 3 (optionally) and Hive 1. I doubt it will workconnecting to Hadoop 3 / Hive 3; it's possible in a few cases.It's also possible some vendor distributions support this c...
   Author: Sean Owen , 2020-07-06, 13:16
XmlReader not Parsing the Nested elements in XML properly - Spark - [mail # user]
...This is more a question about spark-xml, which is not part of Spark.You can ask at https://github.com/databricks/spark-xml/ but if you doplease show some example of the XML input and schema ...
   Author: Sean Owen , 2020-06-30, 16:43
[expand - 2 more] - When is a Bigint a long and when is a long a long - Spark - [mail # user]
...'bigint' is a long, not a Java BigInteger.On Sun, Jun 28, 2020 at 5:52 AM Anwar AliKhan  wrote:>> I wish to draw your attention for your consideration to  this  approach...
   Author: Sean Owen , 2020-06-28, 13:23
Setting spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version=1 and Doc issue - Spark - [mail # dev]
...I think is a Hadoop property that is just passed through? if thedefault is different in Hadoop 3 we could mention that in the docs. idon't know if we want to always set it to 1 as a Spark de...
   Author: Sean Owen , 2020-06-25, 20:56