| clear
query|
facets|
time |
Search criteria: author:"Mich Talebzadeh".
Results from 1 to 10 from
46 (0.0s).
|
|
|
Loading phrases to help you
refine your search...
|
|
[expand - 1 more]
[collapse]
-
Spark on yarn - application hangs -
Spark - [mail # user]
|
| ...sure NP.I meant these topics[image: image.png]Have a look at this article of minehttps://www.linkedin.com/pulse/real-time-processing-trade-data-kafka-flume-spark-talebzadeh-ph-d-/under secti... |
|
|
|
|
| |
|
Spark on yarn - application hangs - Spark - [mail # user]
|
| ...Hi,Have you checked matrices from Spark UI by any chance? What is stagnant?HTHDr Mich TalebzadehLinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw*ht... |
|
|
|
|
|
|
|
Reading JSON RDD in Spark Streaming -
Spark - [mail # user]
|
| ...Hi,I have prices coming through Kafka in the following formatkey,{JSON data}The key is needed as part of data post to NoSQL database like Aerospike.The following is record of topic from Kafk... |
|
|
|
|
|
|
|
Load Time from HDFS -
Spark - [mail # user]
|
| ...Have you tried looking at Spark GUI to see the time it takes to load fromHDFS?Spark GUI by default runs on port 4040. However, you can set in spark-submit${SPARK_HOME}/bin/spark-submit ... |
|
|
|
|
|
|
|
Window function range between -
Spark - [mail # user]
|
| ...Hi,This works for mescala> val wSpec3 =Window.partitionBy('priceInfo.getItem("ticker")).orderBy('priceInfo.getItem("timeissued").desc).rangeBetween(0,5)wSpec3: org.apache.spark.sql.expres... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
Streaming data out of spark to a Kafka topic -
Spark - [mail # user]
|
| ...Hi,In a traditional we get data via Kafka into Spark streaming, do some workand write to a NoSQL database like Mongo, Hbase or Aerospike.That part can be done below and is best explained by ... |
|
|
|
|
| |
|
Streaming data out of spark to a Kafka topic - Spark - [mail # user]
|
| ...Hi Gabor,I will look at the link and see what it provides.Thanks,Dr Mich TalebzadehLinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw*http://talebzad... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
Writing to Aerospike from Spark with bulk write with user authentication fails -
Spark - [mail # user]
|
| ...Aerospike Enterprise version if anyone has worked with user authentication!As far as I know one can create a client with aerospike authentication asfollows that works for single putimport co... |
|
|
|
|
|
|
|
writing into oracle database is very slow -
Spark - [mail # user]
|
| ...I don't believe there is inherently anything slow writing to Oracle.This piece of code works fine for me ready a csv file from HDFS and writingit to Oracle tableimport java.sql.DriverManager... |
|
|
|
|
|
|
|
Using Spark as an ETL tool for moving data from Hive tables to BigQuery -
Spark - [mail # user]
|
| ...Hi,To move data from Hive to Google BigQuery, one needs to create a stagingtable in Hive in a storage format that can be read in BigQuery. Both AVROand ORC file format in Hive work but the f... |
|
|
|
|
|
|
|
Where does the Driver run? -
Spark - [mail # user]
|
| ...Hi,I have explained this in my following Linkedlin article "The OperationalAdvantages of Spark as a Distributed Processing Framework"An extract*2) YARN Deployment Modes*The term D*eployment ... |
|
|
|
|
|
|
|
[expand - 1 more]
[collapse]
-
How to retrieve multiple columns values (in one row) to variables in Spark Scala method -
Spark - [mail # user]
|
| ...Hi,Pretty basic question.I use Spark on Hbase to retrieve the last 14 prices average and standarddeviation for a security (ticker) from an Hbase table.However, the call is expens... |
|
|
|
|
|
|
|
|