clear query| facets| time Search criteria: .   Results from 1 to 10 from 20 (0.0s).
Loading phrases to help you
refine your search...
[SPARK-22870] Dynamic allocation should allow 0 idle time - Spark - [issue]
...As discussed in SPARK-22765, with SPARK-21656, an executor will not idle out when there are pending tasks to run. When there is no task to run, an executor will die out after spark.dynamicAl...    Author: Xuefu Zhang , 2018-01-13, 16:02
[SPARK-22765] Create a new executor allocation scheme based on that of MR - Spark - [issue]
...Many users migrating their workload from MR to Spark find a significant resource consumption hike (i.e, SPARK-22683). While this might not be a concern for users that are more performance ce...    Author: Xuefu Zhang , 2017-12-22, 04:43
feedback about SPARK-22683 needed - Spark - [mail # dev]
...At Uber we have observed the same resource efficiency issue with dynamicallocation. Our workload is migrated from Hive on MR to Hive on Spark. Wesaw significant performance improvement (>...
   Author: Xuefu Zhang , 2017-12-11, 17:57
Futures timeout exception in executor logs - Spark - [mail # dev]
...I saw this quite often in our clusters. we have increasedspark.executor.heartbeatIntervalto 60s from the default value, which should help. The problem seems due topoor Spark driver performan...
   Author: Xuefu Zhang , 2017-09-14, 20:39
[SPARK-20662] Block jobs that have greater than a configured number of tasks - Spark - [issue]
...In a shared cluster, it's desirable for an admin to block large Spark jobs. While there might not be a single metrics defining the size of a job, the number of tasks is usually a good indica...    Author: Xuefu Zhang , 2017-06-09, 17:53
[SPARK-4921] TaskSetManager mistakenly returns PROCESS_LOCAL for NO_PREF tasks - Spark - [issue]
...During research for HIVE-9153, we found that TaskSetManager returns PROCESS_LOCAL for NO_PREF tasks, which may caused performance degradation. Changing the return value to NO_PREF, as demons...    Author: Xuefu Zhang , 2017-05-23, 16:28
welcoming Takuya Ueshin as a new Apache Spark committer - Spark - [mail # dev]
...Congratulations, Takuya!On Mon, Feb 13, 2017 at 11:25 AM, Xiao Li  wrote:...
   Author: Xuefu Zhang , 2017-02-13, 19:29
[SPARK-2421] Spark should treat writable as serializable for keys - Spark - [issue]
...It seems that Spark requires the key be serializable (class implement Serializable interface). In Hadoop world, Writable interface is used for the same purpose. A lot of existing classes, wh...    Author: Xuefu Zhang , 2016-02-16, 15:10
Pros and cons -Saving spark data in hive - Spark - [mail # user]
...You might want to consider Hive on Spark where you can work directly withHive and your query execution is powered by Spark as an engine.--XuefuOn Tue, Dec 15, 2015 at 6:04 PM, Divya Gehlot w...
   Author: Xuefu Zhang , 2015-12-16, 04:01
[expand - 1 more] - Building spark 1.3 from source code to work with Hive 1.2.1 - Spark - [mail # user]
...As mentioned a few times in the past, you need to build yourspark-assembly.jar without Hive and copied it to Hive's /lib directory. Youshouldn't use the one you downloaded from Spark install...
   Author: Xuefu Zhang , 2015-12-03, 18:42