clear query| facets| time Search criteria: .   Results from 1 to 3 from 3 (0.0s).
Loading phrases to help you
refine your search...
Mocking pyspark read writes - Spark - [mail # user]
...Hi everyone,I have a function which reads and writes a parquet file from HDFS. When I'mwriting a unit test for this function, I want to mock this read & write.How do you achieve this?Any hel...
   Author: Dark Crusader , 2020-07-07, 18:07
Add python library with native code - Spark - [mail # user]
...Hi Stone,Have you looked into this article?https://medium.com/@SSKahani/pyspark-applications-dependencies-99415e0df987I haven't tried it with .so files however I did use the approach herecom...
   Author: Dark Crusader , 2020-06-05, 08:02
[expand - 3 more] - Spark dataframe hdfs vs s3 - Spark - [mail # user]
...Thanks all for the replies.I am switching to hdfs since it seems like an easier solution.To answer some of your questions, my hdfs space is a part of my nodes I usefor computation on spark.F...
   Author: Dark Crusader , 2020-05-30, 07:29