clear query| facets| time Search criteria: author:"Mich Talebzadeh".   Results from 21 to 30 from 43 (0.0s).
Loading phrases to help you
refine your search...
Sharing ideas on using Databricks Delta Lake - Spark - [mail # user]
...I upgraded my Spark to 2.4.3 that allows using the storage layer Delta Lake . Actually, I wishDatabricks would have chosen a different name for it :)Anyhow although most example of stor...
   Author: Mich Talebzadeh , 2019-08-07, 21:06
[expand - 1 more] - Control Sqoop job from Spark job - Spark - [mail # user]
...Spark is an excellent ETL tool to lift data from source and put it intarget. Spark uses JDBC connection similar to Sqoop. I don't see the needfor Sqoop with Spark here.Where is the source (O...
   Author: Mich Talebzadeh , 2019-08-31, 06:47
Questions for platform to choose - Spark - [mail # user]
...Hi,What is the definition of real time here?The engineering definition of real time is roughly fast enough to beinteractive. However, I put a stronger definition. In real time applicationor ...
   Author: Mich Talebzadeh , 2019-08-21, 09:12
org.apache.spark.util.SparkUncaughtExceptionHandler - Spark - [mail # user]
...Hi Nimmi,Can you send us the spark parameters with overhead. assuming you arerunning with yarnExample[4] - 864GB--num-executors 32--executor-memory 21G--executor-cores 4--conf spark.yarn.exe...
   Author: Mich Talebzadeh , 2019-10-10, 21:10
Google Cloud and Spark in the docker consideration for rreal time streaming data - Spark - [mail # user]
...Hi,We have designed an efficient trading system that runs on cluster of threenodes in Google Cloud Platform (GCP).Each node runs one Zookeeper and one Kafka broker. So in total we havethree ...
   Author: Mich Talebzadeh , 2019-09-23, 19:05
[expand - 1 more] - OrderBy Year and Month is not displaying correctly - Spark - [mail # user]
...*The distinct transformation does not preserve order, you need to distinctfirst, then orderby.*Thanks Enrico. You are correct. Worked fine!joint_accounts.      select(year(col...
   Author: Mich Talebzadeh , 2020-01-06, 12:05
Spark streaming when a node or nodes go down - Spark - [mail # user]
...Hi,I know this is a basic question but someone enquired about it and I justwanted to fill my knowledge gap so to speak.Within the context of Spark streaming, the RDD is created from the inco...
   Author: Mich Talebzadeh , 2019-12-11, 10:28