| clear
query|
facets|
time |
Search criteria: .
Results from 1 to 10 from
940 (0.0s).
|
|
|
Loading phrases to help you
refine your search...
|
|
[SPARK-18818] Window...orderBy() should accept an 'ascending' parameter just like DataFrame.orderBy() -
Spark - [issue]
|
| ...It seems inconsistent that Window...orderBy() does not accept an ascending parameter, when DataFrame.orderBy() does.It's also slightly inconvenient since to specify a descending sort order y... |
|
|
|
|
|
|
|
[SPARK-25150] Joining DataFrames derived from the same source yields confusing/incorrect results -
Spark - [issue]
|
| ...I have two DataFrames, A and B. From B, I have derived two additional DataFrames, B1 and B2. When joining A to B1 and B2, I'm getting a very confusing error:Join condition is missing or triv... |
|
|
|
|
|
|
|
[expand - 4 more]
[collapse]
-
Documentation of boolean column operators missing? -
Spark - [mail # dev]
|
| ...On Tue, 23 Oct 2018 at 21:32, Sean Owen wrote:>>> The comments say that it is not possible to overload 'and' and 'or',>> which would have been more natural.>>> ... |
|
|
|
|
| |
|
Documentation of boolean column operators missing? - Spark - [mail # dev]
|
| ...Also, to clarify something for folks who don't work with PySpark: Theboolean column operators in PySpark are completely different from those inScala, and non-obvious to boot (since they over... |
|
|
|
|
|
Documentation of boolean column operators missing? - Spark - [mail # dev]
|
| ...So it appears then that the equivalent operators for PySpark are completelymissing from the docs, right? That’s surprising. And if there are columnfunction equivalents for |, &, and ~, then ... |
|
|
|
|
|
Documentation of boolean column operators missing? - Spark - [mail # dev]
|
| ...Nope, that’s different. I’m talking about the operators on DataFramecolumns in PySpark, not SQL functions.For example:(df .where(~col('is_exiled') & (col('age') > 60)) ... |
|
|
|
|
|
Documentation of boolean column operators missing? - Spark - [mail # dev]
|
| ...I can’t seem to find any documentation of the &, |, and ~ operators forPySpark DataFrame columns. I assume that should be in our docs somewhere.Was it always missing? Am I just missing somet... |
|
|
|
|
|
|
|
[VOTE] SPARK 2.4.0 (RC3) -
Spark - [mail # dev]
|
| ...FYI I believe we have an open correctness issue here:https://issues.apache.org/jira/browse/SPARK-25150However, it needs review by another person to confirm whether it is indeeda correctness ... |
|
|
|
|
|
|
|
[VOTE] SPARK 2.4.0 (RC1) -
Spark - [mail # dev]
|
| ...I believe -1 votes are merited only for correctness bugs and regressionssince the previous release.Does SPARK-23200 count as either?2018년 9월 17일 (월) 오전 9:40, Stavros Kontopoulos <[EMAIL P... |
|
|
|
|
|
|
|
Should python-2 be supported in Spark 3.0? -
Spark - [mail # user]
|
| ...As Reynold pointed out, we don't have to drop Python 2 support right offthe bat. We can just deprecate it with Spark 3.0, which would allow us toactually drop it at a later 3.x release.On Sa... |
|
|
|
|
|
|
|
Python friendly API for Spark 3.0 -
Spark - [mail # dev]
|
| ...Do we need to ditch Python 2 support to provide type hints? I don’t thinkso.Python lets you specify typing stubs that provide the same benefit withoutforcing Python 3.2018년 9월 14일 (금) 오후 8:0... |
|
|
|
|
|
|
|
[SPARK-18084] write.partitionBy() does not recognize nested columns that select() can access -
Spark - [issue]
|
| ...Here's a simple repro in the PySpark shell:from pyspark.sql import Rowrdd = spark.sparkContext.parallelize([Row(a=Row(b=5))])df = spark.createDataFrame(rdd)df.printSchema()df.select('a.b').s... |
|
|
|
|
|
|
|
Joining DataFrames derived from the same source yields confusing/incorrect results -
Spark - [mail # dev]
|
| ...Dunno if I made a silly mistake, but I wanted to bring some attention tothis issue in case there was something serious going on here that mightaffect the upcoming release.https://issues.apac... |
|
|
|
|
|
|
|
[SPARK-17025] Cannot persist PySpark ML Pipeline model that includes custom Transformer -
Spark - [issue]
|
| ...Following the example in this Databricks blog post under "Python tuning", I'm trying to save an ML Pipeline model.This pipeline, however, includes a custom transformer. When I try to save th... |
|
|
|
|
|
|
|
|