clear query| facets| time Search criteria: .   Results from 1 to 10 from 12 (0.0s).
Loading phrases to help you
refine your search...
Spark Sparser library - Spark - [mail # user]
...Hi Team,Please let me know the spark Sparser library to use while submitting thespark application to use below mentioned format,val df = spark.read.format("*edu.stanford.sparser.json*")When ...
   Author: umargeek , 2018-08-10, 05:48
[expand - 2 more] - How to validate orc vectorization is working within spark application? - Spark - [mail # user]
...Hello Jorn,I am unable to post the entire code due to some data sharing related issues.Use Case: I am performing aggregations after reading data from HDFS fileevery min would like to underst...
   Author: umargeek , 2018-07-12, 19:15
Pyspark Structured Streaming Error - Spark - [mail # user]
...Hi All,I am trying to test structured streaming using pyspark mentioned below sparksubmit commands and packages used*pyspark2 --master=yarn --packagesorg.apache.spark:spark-sql-kafka-0-10_2....
   Author: umargeek , 2018-07-12, 18:54
Spark DF to Hive table with both Partition and Bucketing not working - Spark - [mail # user]
...Hi Folks,I am trying to save a spark data frame after reading from ORC file and addtwo new columns and finally trying to save it to hive table with bothpartition and bucketing feature.Using ...
   Author: umargeek , 2018-06-20, 03:41
testing frameworks - Spark - [mail # user]
...Hi Steve,you can try out pytest-spark plugin if your writing programs using pyspark,please find below link for reference.https://github.com/malexer/pytest-spark  Thanks,Umar--Sent from:...
   Author: umargeek , 2018-05-23, 05:07
Alternative for numpy in Spark Mlib - Spark - [mail # user]
...Hi Folks,I am planning to rewrite one of my python module written for entropycalculation using numpy into Spark Mlib so that it can be processed indistributed manner.Can you please advise on...
   Author: umargeek , 2018-05-23, 05:04
Streaming Analytics/BI tool to connect Spark SQL - Spark - [mail # user]
...Hi All,We are currently looking for real-time streaming analytics of data stored asSpark SQL tables is there any external connectivity available to connectwith BI tools(Pentaho/Arcadia).curr...
   Author: umargeek , 2017-12-07, 18:27
How to write dataframe to kafka topic in spark streaming application using pyspark other than collect? - Spark - [mail # user]
...Hi Team,Can someone please advise me on the above post since because of this I havewritten data file to HDFS location. So as of now am just passing the filename into Kafka topic and not util...
   Author: umargeek , 2017-12-07, 18:25
Suggestions on using scala/python for Spark Streaming - Spark - [mail # user]
...We are building a spark streaming application which is process and timeintensive and currently using python API but looking forward for suggestionswhether to use Scala over python such as pr...
   Author: umargeek , 2017-10-26, 16:22
How to write dataframe to kafka topic in spark streaming application using pyspark? - Spark - [mail # user]
...Can anyone provide me code snippet/ steps to write a data frame to Kafkatopic in a spark streaming application using pyspark with spark 2.1.1 andKafka 0.8 (Direct Stream Approach)?Thanks,Uma...
   Author: umargeek , 2017-09-25, 10:40