clear query| facets| time Search criteria: .   Results from 1 to 10 from 26 (0.0s).
Loading phrases to help you
refine your search...
Spark In Memory Shuffle / 5403 - Spark - [mail # user]
...I would be very interested in the initial question here:is there a production level implementation for memory only shuffle andconfigurable (similar to  MEMORY_ONLY storage level,  ...
   Author: Peter Liu , 2018-10-18, 15:07
configure yarn to use more vcores as the node provides? - Spark - [mail # dev]
...Hi Khaled,the 50-2-3g I mentioned below is meant for the --conf spark.executor.*config, in particular,  spark.executor.instances=50,  spark.executor.cores=2and  spark.executor...
   Author: Peter Liu , 2018-10-16, 16:38
[expand - 1 more] - overcommit: cpus / vcores - Spark - [mail # dev]
...Hi Khaled,I have attached the spark streaming config below in (a).In case of the 100vcore run (see the initial email), I used 50 executorswhere each executor has 2 vcores and 3g memory. For ...
   Author: Peter Liu , 2018-10-16, 01:06
re: yarn resource overcommit: cpu / vcores - Spark - [mail # user]
...Hi there,is there any best practice guideline on yarn resource overcommit with cpu /vcores, such as yarn config options, candidate cases ideal forovercommiting vcores etc.?this slide below (...
   Author: Peter Liu , 2018-10-11, 19:35
[AVRO-1702] Add LogicalType support to c++ library - Avro - [issue]
...I'd like to port the logicaltype support to c++ library...
http://issues.apache.org/jira/browse/AVRO-1702    Author: peter liu , 2018-10-08, 16:22
re: should we dump a warning if we drop batches due to window move? - Spark - [mail # dev]
...Hello there,I have a quick question for the following case:situation:a spark consumer is able to process 5 batches in 10 sec (where the batchinterval is zero by default - correct me if this ...
   Author: Peter Liu , 2018-08-03, 20:10
[expand - 1 more] - [External Sender] re: streaming, batch / spark 2.2.1 - Spark - [mail # user]
...thanks for the clarification.the processing time on both systems seems to be fine :(a) based on thepattern of batch processing time chart, i.e the  batch processing time isnot becoming ...
   Author: Peter Liu , 2018-08-02, 20:48
[expand - 3 more] - spark 2.3.1 with kafka spark-streaming-kafka-0-10 (java.lang.AbstractMethodError) - Spark - [mail # user]
...Thank you, Ryan!I'll keep you guys updated.Cheers,PeterOn Tue, Jul 3, 2018 at 4:22 PM, Shixiong(Ryan) Zhu wrote:> -dev +user>> Hey Peter,>> You should upgrade "org.apache.spar...
   Author: Peter Liu , 2018-07-03, 23:32
re: streaming - kafka partition transition time from (stage change logger) - Spark - [mail # user]
...Hi there,Working on the streaming processing latency time based on timestamps fromKafka, I have two quick general questions triggered by looking at the kafkastage change log file:(a) the par...
   Author: Peter Liu , 2018-06-11, 14:51
[expand - 1 more] - help with streaming batch interval question needed - Spark - [mail # user]
...Hi Jacek,This is exact what i'm looking for. Thanks!!Also thanks for the link. I just noticed that I can unfold the link oftrigger and see the examples in java and scala languages - what a g...
   Author: Peter Liu , 2018-05-25, 14:27