clear query| facets| time Search criteria: author:"Josh Rosen".   Results from 1 to 10 from 11 (0.0s).
Loading phrases to help you
refine your search...
[SPARK-28427] Support more Postgres JSON functions - Spark - [issue]
...Postgres features a number of JSON functions that are missing in Spark:'s JSON functions (    Author: Josh Rosen , 2019-12-29, 06:07
[SPARK-28007] Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift - Spark - [issue]
...The expression expr1 ^ expr2 has different meanings in Spark and Postgres: In Postgres and Redshift , this returns expr1 raised to the exponent expr2 (additionally, the Postgres docs explici...    Author: Josh Rosen , 2019-12-29, 06:36
[SPARK-30414] Optimizations for arrays and maps in ParquetRowConverter - Spark - [issue]
...Similar to SPARK-30338, there are multiple opportunities to optimize ParquetRowConverter performance for maps and arrays, plus room for more general constant-factor shaving in a few other pl...    Author: Josh Rosen , 2020-01-07, 05:31
[SPARK-30338] Avoid unnecessary InternalRow copies in ParquetRowConverter - Spark - [issue]
...ParquetRowConverter calls InternalRow.copy() in cases where the copy is unnecessary; this can severely harm performance when reading deeply-nested Parquet.It looks like this copying was orig...    Author: Josh Rosen , 2020-01-07, 09:18
[SPARK-27915] Update logical Filter's output nullability based on IsNotNull conditions - Spark - [issue]
...The physical `FilterExec` operator has logic to update its output nullability based on IsNotNull expressions; it might make sense to add similar logic to the logical Filter operator....    Author: Josh Rosen , 2019-12-30, 00:06
[SPARK-27846] Eagerly compute in sc.hadoopConfiguration - Spark - [issue]
...Hadoop Configuration has an internal properties map which is lazily initialized. Initialization of this field, done in the private Configuration.getProps() method, is rather expensive becaus...    Author: Josh Rosen , 2019-12-30, 03:20
[SPARK-30413] Avoid unnecessary WrappedArray roundtrip in GenericArrayData constructor - Spark - [issue]
...GenericArrayData has a constructor which accepts a seqOrArray: Any parameter. This constructor was originally added for use in situations where we don't know the actual type at compile-time ...    Author: Josh Rosen , 2020-01-20, 03:15
[SPARK-28478] Optimizer rule to remove unnecessary explicit null checks for null-intolerant expressions (e.g. if(x is null, x, f(x))) - Spark - [issue]
...I ran across a family of expressions likeif(x is null, x, substring(x, 0, 1024))or when($"x".isNull, $"x", substring($"x", 0, 1024))that were written this way because the query author was un...    Author: Josh Rosen , 2020-01-22, 00:45
[SPARK-27619] MapType should be prohibited in hash expressions - Spark - [issue]
...Spark currently allows MapType expressions to be used as input to hash expressions, but I think that this should be prohibited because Spark SQL does not support map equality.Currently, Spar...    Author: Josh Rosen , 2020-01-22, 20:51
[SPARK-29419] Seq.toDS / spark.createDataset(Seq) is not thread-safe - Spark - [issue]
...The Seq.toDS and spark.createDataset(Seq) code is not thread-safe: if the caller-supplied Encoder is used in multiple threads then createDataset's usage of the encoder may lead to incorrect ...    Author: Josh Rosen , 2020-01-22, 20:51