clear query| facets| time Search criteria: .   Results from 1 to 10 from 180 (0.0s).
Loading phrases to help you
refine your search...
How to debug Spark job - Spark - [mail # user]
...Hi Might sound like a dumb advice. But try to break apart your process.Sounds like youAre doing ETLstart basic with just ET. and do the changes that results in issuesIf no problem add t...
   Author: Marco Mistroni , 2018-09-08, 17:44
Reading multiple files in Spark / which pattern to use - Spark - [mail # user]
...hi all i have mutliple files stored in S3 in the following patternYYYY-MM-DD-securities.txtI want to read multiple files at the same time..I am attempting to use this pattern, for examp...
   Author: Marco Mistroni , 2018-07-12, 21:05
spark-shell gets stuck in ACCEPTED state forever when ran in YARN client mode. - Spark - [mail # user]
...You running on emr? You checked the emr logs?Was in similar situation where job was stuck in accepted and then itdied..turned out to be an issue w. My code when running g with hugedata.perha...
   Author: Marco Mistroni , 2018-07-08, 14:20
Error submitting Spark Job in yarn-cluster mode on EMR - Spark - [mail # user]
...Did you by any chances left a   sparkSession.setMaster("local") lurking inyour code?Last time i checked, to run on yarn you have to package a 'fat jar'. couldyou make sure the spark dep...
   Author: Marco Mistroni , 2018-05-08, 20:14
RV: Unintelligible warning arose out of the blue. - Spark - [mail # user]
...Hi  i think it has to do with spark configuration,  dont think the standardconfiguration is geared up to be running in local mode on windowsyour dataframe is ok, you can check out ...
   Author: Marco Mistroni , 2018-05-04, 20:54
[expand - 1 more] - Problem in persisting file in S3 using Spark: xxx file does not exist Exception - Spark - [mail # user]
...Hi Sorted ..I just replaced s3 with s3a....I think I recall similar issues inthe past with aws libraries.Thx anyway for getting backKrOn Wed, May 2, 2018, 4:57 PM Paul Tremblay  wr...
   Author: Marco Mistroni , 2018-05-02, 19:35
A naive ML question - Spark - [mail # user]
...Maybe not necessarily what you want but you could, based on transattributes, find out initial state and end state and give it to a decisiontree to figure out if you if based on these attribu...
   Author: Marco Mistroni , 2018-04-29, 10:21
Dataframe vs dataset - Spark - [mail # user]
...Imho .neither..I see datasets as typed df and therefore ds are enhanced dfFeel free to disagree..KrOn Sat, Apr 28, 2018, 2:24 PM Michael Artz  wrote:> Hi,>> I use Spark everyda...
   Author: Marco Mistroni , 2018-04-28, 13:34
Live Stream Code Reviews :) - Spark - [mail # user]
...PST  I believe....like last timeWorks out 9pm bst & 10 pm cet if I m correctOn Thu, Apr 12, 2018, 8:47 PM Matteo Olivi  wrote:> Hi,> 11 am in which timezone?>> Il gio 1...
   Author: Marco Mistroni , 2018-04-12, 20:01
Best active groups, forums or contacts for Spark ? - Spark - [mail # user]
...Hi From personal experience....and I might be asking u obvious question1. Does it work in standalone (no cluster)2. Can u break down app in pieces and try to see at which step the codeg...
   Author: Marco Mistroni , 2018-01-26, 14:08