clear query| facets| time Search criteria: .   Results from 1 to 10 from 695 (0.0s).
Loading phrases to help you
refine your search...
SparkR issue - Spark - [mail # user]
...HiWe are seeing some weird behaviour in Spark R.We created a R Dataframe with 600K records and 29 columns. Then we tried toconvert R DF to SparkDF usingdf <- SparkR::createDataFrame(rdf)f...
   Author: ayan guha , 2018-10-09, 06:21
Pyspark Partitioning - Spark - [mail # user]
...HiThere are a set pf finction which can be used with the constructOver (partition by col order by col).You search for rank and window functions in spark documentation.On Mon, 1 Oct 2018 at 5...
   Author: ayan guha , 2018-09-30, 21:49
Time-Series Forecasting - Spark - [mail # user]
...HiI work mostly in data engineering and trying to promote use of sparkRwithin the company I recently joined. Some of the users are working aroundforecasting a bunch of things and want to use...
   Author: ayan guha , 2018-09-19, 23:13
Drawing Big Data tech diagrams using Pen Tablets - Spark - [mail # user]
...FWIW...I use draw.io and it is pretty neat........On Thu, Sep 13, 2018 at 6:46 AM, Gourav Sengupta wrote:> well, it may be possible to use just excel shapes ( and advanced shapes> inst...
   Author: ayan guha , 2018-09-13, 00:38
Big Burst of Streaming Changes - Spark - [mail # user]
...HiWe have a situation where we are ingesting high volume streaming ingestcoming from a Oracle table.The requirementWhenever there is a change in Oracle table, a CDC process will write outthe...
   Author: ayan guha , 2018-07-29, 23:54
the best tool to interact with Spark - Spark - [mail # user]
...Depends on what are you trying to do. I found zeppelin an excellent optionto interactively run queries and codeOn Tue, Jun 26, 2018 at 10:21 PM, Donni Khan <[EMAIL PROTECTED]lid> wrote...
   Author: ayan guha , 2018-06-26, 13:11
Spark-Mongodb connector issue - Spark - [mail # user]
...Hi GuysI have a large mongodb collection with complex document structure. I anfacing an issue when I am getting error asCan not cast Array to Struct. Value:BsonArray([])The target column is ...
   Author: ayan guha , 2018-06-18, 23:07
Append In-Place to S3 - Spark - [mail # user]
...I do not use anti join semantics, but you can use left outer join and thenfilter out nulls from right side. Your data may have dups on the columnsseparately but it should not have dups on th...
   Author: ayan guha , 2018-06-03, 21:47
Bulk / Fast Read and Write with MSSQL Server and Spark - Spark - [mail # user]
...Curious question: what is the reason of using spark here? Why not simplesql-based ETL?On Thu, May 24, 2018 at 5:09 AM, Ajay  wrote:> Do you worry about spark overloading the SQL serv...
   Author: ayan guha , 2018-05-23, 19:38
How to skip nonexistent file when read files with spark? - Spark - [mail # user]
...A relatively naive solution will be:0. Create a dummy blank dataframe1. Loop through the list of paths.2. Try to create the dataframe from the path. If success then union itcumulatively.3. I...
   Author: ayan guha , 2018-05-22, 02:33