clear query| facets| time Search criteria: .   Results from 41 to 50 from 1066 (0.0s).
Loading phrases to help you
refine your search...
[expand - 1 more] - Control Sqoop job from Spark job - Spark - [mail # user]
...Hi,Just to clarify, JDBC connection to RDBMS from Spark is slow?This one read from an Oracle table with 4 connections in parallel to Oracletable assuming there is a primary key on the Oracle...
   Author: Mich Talebzadeh , 2019-09-02, 18:06
Questions for platform to choose - Spark - [mail # user]
...Hi,What is the definition of real time here?The engineering definition of real time is roughly fast enough to beinteractive. However, I put a stronger definition. In real time applicationor ...
   Author: Mich Talebzadeh , 2019-08-21, 09:12
Unable to write data from Spark into a Hive Managed table - Spark - [mail # user]
...Check your permissioning.Can you do insert select from external table into Hive managed tablecreated by spark?//// Need to create and populate target ORC table transactioncodes_ll indatabase...
   Author: Mich Talebzadeh , 2019-08-09, 16:51
Spark SQL reads all leaf directories on a partitioned Hive table - Spark - [mail # user]
...also need others as well using soft link ls -lcd $SPARK_HOME/confhive-site.xml -> ${HIVE_HOME/conf/hive-site.xmlcore-site.xml -> ${HADOOP_HOME}/etc/hadoop/core-site.xmlhdfs-site.xml -&...
   Author: Mich Talebzadeh , 2019-08-08, 19:05
Sharing ideas on using Databricks Delta Lake - Spark - [mail # user]
...I upgraded my Spark to 2.4.3 that allows using the storage layer Delta Lake . Actually, I wishDatabricks would have chosen a different name for it :)Anyhow although most example of stor...
   Author: Mich Talebzadeh , 2019-08-07, 21:06
[expand - 3 more] - Hive external table not working in sparkSQL when subdirectories are present - Hive - [mail # user]
...Have you updated partition statistics by any chance?I assume you can access the table and data though Hive itself?HTHDr Mich TalebzadehLinkedIn * https://www.linkedin.com/profile/view?id=AAE...
   Author: Mich Talebzadeh , 2019-08-07, 20:51
How to read configuration file parameters in Spark without mapping each parameter - Spark - [mail # user]
...Hi,Assume that I have a configuration file as below with static parameterssome Strings, Integer and Double:md_AerospikeAerospike {dbHost = "rhes75"dbPort = "3000"dbConnection = "trading_user...
   Author: Mich Talebzadeh , 2019-08-06, 21:34
[expand - 2 more] - Reading configuration file in Spark Scala throws error - Spark - [mail # user]
...OK it turned up pretty simple once I established where to read the conffile in yarn mode and get the values for different keys  val dbHost = conf.getString("dbHost")  val dbPort = ...
   Author: Mich Talebzadeh , 2019-08-05, 00:01
[expand - 2 more] - Spark SaveMode - Spark - [mail # user]
...I dug some of my old stuff using Spark as ETL.Regarding the question"Any reason why Spark's SaveMode doesn't have mode that ignore any PrimaryKey/Unique constraint violations?"There is no wa...
   Author: Mich Talebzadeh , 2019-07-21, 13:53
[expand - 2 more] - Spark dataset to explode json string - Spark - [mail # user]
...You can try to split the {"foo": "val1", "bar": "val2"} as below./*This is an example of output!(c1003d93-5157-4092-86cf-0607157291d8,{"rowkey":"c1003d93-5157-4092-86cf-0607157291d8","ticker...
   Author: Mich Talebzadeh , 2019-07-20, 03:54