clear query| facets| time Search criteria: .   Results from 21 to 30 from 556 (0.0s).
Loading phrases to help you
refine your search...
Should python-2 be supported in Spark 3.0? - Spark - [mail # user]
...As Reynold pointed out, we don't have to drop Python 2 support right offthe bat. We can just deprecate it with Spark 3.0, which would allow us toactually drop it at a later 3.x release.On Sa...
   Author: Nicholas Chammas , 2018-09-15, 18:18
Python friendly API for Spark 3.0 - Spark - [mail # dev]
...Do we need to ditch Python 2 support to provide type hints? I don’t thinkso.Python lets you specify typing stubs that provide the same benefit withoutforcing Python 3.2018년 9월 14일 (금) 오후 8:0...
   Author: Nicholas Chammas , 2018-09-15, 00:38
[SPARK-18084] write.partitionBy() does not recognize nested columns that select() can access - Spark - [issue]
...Here's a simple repro in the PySpark shell:from pyspark.sql import Rowrdd = spark.sparkContext.parallelize([Row(a=Row(b=5))])df = spark.createDataFrame(rdd)df.printSchema()df.select('a.b').s...
http://issues.apache.org/jira/browse/SPARK-18084    Author: Nicholas Chammas , 2018-09-11, 14:19
Joining DataFrames derived from the same source yields confusing/incorrect results - Spark - [mail # dev]
...Dunno if I made a silly mistake, but I wanted to bring some attention tothis issue in case there was something serious going on here that mightaffect the upcoming release.https://issues.apac...
   Author: Nicholas Chammas , 2018-08-29, 16:44
Review notification bot - Spark - [mail # dev]
...On this topic, I just stumbled on a GitHub feature called CODEOWNERS. It lets you specifyowners of specific areas of the repository using the same syntax that.gitignore uses. Here is CPython...
   Author: Nicholas Chammas , 2018-07-23, 02:05
[HADOOP-15559] Clarity on Spark compatibility with hadoop-aws - Hadoop - [issue]
...I'm the maintainer of Flintrock, a command-line tool for launching Apache Spark clusters on AWS. One of the things I try to do for my users is make it straightforward to use Spark with s3a:/...
http://issues.apache.org/jira/browse/HADOOP-15559    Author: Nicholas Chammas , 2018-06-28, 14:16
[expand - 3 more] - [VOTE] Spark 2.3.1 (RC4) - Spark - [mail # dev]
...I'll give that a try, but I'll still have to figure out what to do if noneof the release builds work with hadoop-aws, since Flintrock deploys Sparkrelease builds to set up a cluster. Buildin...
   Author: Nicholas Chammas , 2018-06-02, 23:53
[expand - 1 more] - Documenting the various DataFrame/SQL join types - Spark - [mail # dev]
...OK great, I’m happy to take this on.Does it make sense to approach this by adding an example for each join typehere(and perhaps also in the matching areas for Scala, Java, and R), and thenre...
   Author: Nicholas Chammas , 2018-05-09, 02:53
[expand - 2 more] - Identifying specific persisted DataFrames via getPersistentRDDs() - Spark - [mail # dev]
...That’s correct. I probably would have done better to title this threadsomething like “How to effectively track and release persisted DataFrames”.I jumped the gun in my initial email by refer...
   Author: Nicholas Chammas , 2018-05-09, 02:37
eager execution and debuggability - Spark - [mail # dev]
...This may be technically impractical, but it would be fantastic if we couldmake it easier to debug Spark programs without needing to rely on eagerexecution. Sprinkling .count() and .checkpoin...
   Author: Nicholas Chammas , 2018-05-09, 02:22