clear query| facets| time Search criteria: author:"Mich Talebzadeh".   Results from 1 to 10 from 46 (0.0s).
Loading phrases to help you
refine your search...
[expand - 1 more] - Spark on yarn - application hangs - Spark - [mail # user]
...sure NP.I meant these topics[image: image.png]Have a look at this article of minehttps://www.linkedin.com/pulse/real-time-processing-trade-data-kafka-flume-spark-talebzadeh-ph-d-/under secti...
   Author: Mich Talebzadeh , 2019-05-10, 17:28
Reading JSON RDD in Spark Streaming - Spark - [mail # user]
...Hi,I have prices coming through Kafka in the following formatkey,{JSON data}The key is needed as part of data post to NoSQL database like Aerospike.The following is record of topic from Kafk...
   Author: Mich Talebzadeh , 2019-06-18, 21:42
Load Time from HDFS - Spark - [mail # user]
...Have you tried looking at Spark GUI to see the time it takes to load fromHDFS?Spark GUI by default runs on port 4040. However, you can set in spark-submit${SPARK_HOME}/bin/spark-submit  ...
   Author: Mich Talebzadeh , 2019-04-10, 19:32
Window function range between - Spark - [mail # user]
...Hi,This works for mescala> val wSpec3 =Window.partitionBy('priceInfo.getItem("ticker")).orderBy('priceInfo.getItem("timeissued").desc).rangeBetween(0,5)wSpec3: org.apache.spark.sql.expres...
   Author: Mich Talebzadeh , 2019-03-25, 21:29
[expand - 1 more] - Streaming data out of spark to a Kafka topic - Spark - [mail # user]
...Hi,In a traditional we get data via Kafka into Spark streaming, do some workand write to a NoSQL database like Mongo, Hbase or Aerospike.That part can be done below and is best explained by ...
   Author: Mich Talebzadeh , 2019-03-27, 19:47
[expand - 1 more] - Writing to Aerospike from Spark with bulk write with user authentication fails - Spark - [mail # user]
...Aerospike Enterprise version if anyone has worked with user authentication!As far as I know one can create a client with aerospike authentication asfollows that works for single putimport co...
   Author: Mich Talebzadeh , 2019-04-21, 10:30
writing into oracle database is very slow - Spark - [mail # user]
...I don't believe there is inherently anything slow writing to Oracle.This piece of code works fine for me ready a csv file from HDFS and writingit to Oracle tableimport java.sql.DriverManager...
   Author: Mich Talebzadeh , 2019-04-19, 21:36
Using Spark as an ETL tool for moving data from Hive tables to BigQuery - Spark - [mail # user]
...Hi,To move data from Hive to Google BigQuery, one needs to create a stagingtable in Hive in a storage format that can be read in BigQuery. Both AVROand ORC file format in Hive work but the f...
   Author: Mich Talebzadeh , 2019-01-03, 10:02
Where does the Driver run? - Spark - [mail # user]
...Hi,I have explained this in my following Linkedlin article "The OperationalAdvantages of Spark as a Distributed Processing Framework"An extract*2) YARN Deployment Modes*The term D*eployment ...
   Author: Mich Talebzadeh , 2019-03-28, 09:28
[expand - 1 more] - How to retrieve multiple columns values (in one row) to variables in Spark Scala method - Spark - [mail # user]
...Hi,Pretty basic question.I use Spark on Hbase to retrieve the last  14 prices average and standarddeviation  for a security (ticker) from an Hbase table.However, the call is expens...
   Author: Mich Talebzadeh , 2019-04-05, 19:38