clear query| facets| time Search criteria: .   Results from 1 to 10 from 39 (0.0s).
Loading phrases to help you
refine your search...
[expand - 3 more] - Spark reading from Hbase throws java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods - Spark - [mail # user]
...I stripped everything from the jar list. This is all I havesspark-shell --jars shc-core-1.1.1-2.1-s_2.11.jar, \              json4s-native_2.11-3.5.3.jar, ...
   Author: Mich Talebzadeh , 2020-02-17, 22:27
[expand - 5 more] - How to insert Json records from Flume into Hbase with Kafka source - Flume - [mail # user]
...Hi,This regex seems to work*JsonAgent.sinks.Hbase-sink.serializer.regex=[^_]*"(.+).{1},(.+),(.+),(.+).{1}*Remember we were getting the below as ROW (incorrect) beforehand{"rowkey":"eff0bdc7-...
   Author: Mich Talebzadeh , 2020-02-16, 20:26
[expand - 2 more] - How to insert Json records from Flume into Hbase table - HBase - [mail # user]
...Thanks Pedro. The mention of*org.apache.flume.sink.hbase.RegexHbaseEventSerializer* was very useful.This works# Describing/Configuring the sinkJsonAgent.channels.hdfs-channel-1.type = memory...
   Author: Mich Talebzadeh , 2020-02-14, 20:25
[expand - 1 more] - OrderBy Year and Month is not displaying correctly - Spark - [mail # user]
...*The distinct transformation does not preserve order, you need to distinctfirst, then orderby.*Thanks Enrico. You are correct. Worked fine!joint_accounts.      select(year(col...
   Author: Mich Talebzadeh , 2020-01-06, 12:05
Spark streaming when a node or nodes go down - Spark - [mail # user]
...Hi,I know this is a basic question but someone enquired about it and I justwanted to fill my knowledge gap so to speak.Within the context of Spark streaming, the RDD is created from the inco...
   Author: Mich Talebzadeh , 2019-12-11, 10:28
kafka connection from docker - Kafka - [mail # user]
...I do not understand this.You have on a physical host running zookeeper locally and a broker runningas well. These are using default physical host portslike 2181, 9092 etc.Then you have insta...
   Author: Mich Talebzadeh , 2019-10-17, 21:36
org.apache.spark.util.SparkUncaughtExceptionHandler - Spark - [mail # user]
...Hi Nimmi,Can you send us the spark parameters with overhead. assuming you arerunning with yarnExample[4] - 864GB--num-executors 32--executor-memory 21G--executor-cores 4--conf spark.yarn.exe...
   Author: Mich Talebzadeh , 2019-10-10, 21:10
Google Cloud and Spark in the docker consideration for rreal time streaming data - Spark - [mail # user]
...Hi,We have designed an efficient trading system that runs on cluster of threenodes in Google Cloud Platform (GCP).Each node runs one Zookeeper and one Kafka broker. So in total we havethree ...
   Author: Mich Talebzadeh , 2019-09-23, 19:05
[expand - 1 more] - Control Sqoop job from Spark job - Spark - [mail # user]
...Hi,Just to clarify, JDBC connection to RDBMS from Spark is slow?This one read from an Oracle table with 4 connections in parallel to Oracletable assuming there is a primary key on the Oracle...
   Author: Mich Talebzadeh , 2019-09-02, 18:06
Questions for platform to choose - Spark - [mail # user]
...Hi,What is the definition of real time here?The engineering definition of real time is roughly fast enough to beinteractive. However, I put a stronger definition. In real time applicationor ...
   Author: Mich Talebzadeh , 2019-08-21, 09:12