clear query| facets| time Search criteria: .   Results from 1 to 10 from 115 (0.0s).
Loading phrases to help you
refine your search...
Spark Structured Streaming: “earliest” as “startingOffsets” is not working - Spark - [mail # user]
...My Spark Structured Streaming job works fine when I set "startingOffsets"to "latest". When I simply change it to "earliest" & specify a new "checkpoint directory", the job doesn't work. The ...
   Author: Something Something , 2020-06-26, 21:12
[expand - 12 more] - Using Spark Accumulators with Structured Streaming - Spark - [mail # user]
...*Honestly, I don't know how to do this in Scala.* I tried something likethis...*.mapGroupsWithState(GroupStateTimeout.ProcessingTimeTimeout())(  newStateUpdater(myAcc))*StateUpdater is ...
   Author: Something Something , 2020-06-09, 01:54
[expand - 1 more] - RecordTooLargeException in Spark *Structured* Streaming - Spark - [mail # user]
...Thanks. Missed that part of documentation. Appreciate your help. Regards.On Mon, May 25, 2020 at 10:42 PM Jungtaek Lim wrote:> Hi,>> You need to add the prefix "kafka." for the conf...
   Author: Something Something , 2020-05-26, 15:13
[expand - 1 more] - No. of active states? - Spark - [mail # user]
...No. We are already capturing these metrics (e.g. numInputRows,inputRowsPerSecond).I am talking about "No. of States" in the memory at any given time.On Thu, May 7, 2020 at 4:31 PM Jungtaek L...
   Author: Something Something , 2020-05-08, 02:30
Dynamically changing maxOffsetsPerTrigger - Spark - [mail # user]
...Is there a way to dynamically modify value of 'maxOffsetsPerTrigger' whilea Stateful Structured Streaming job is running?We are thinking of auto-scaling our Spark cluster but if we don't mod...
   Author: Something Something , 2020-05-07, 18:43
Writing Integration test with Embedded Kafka - Kafka - [mail # user]
...I am trying to write an integration test using Embedded Kafka but Ikeep getting NullPointerException. My test case is very simple. It hasfollowing steps:   1. Read a JSON file & wr...
   Author: Something Something , 2020-03-07, 02:38
Spark not able to read from an Embedded Kafka Topic - Spark - [mail # user]
...I am trying to write an integration test using Embedded Kafka but I keepgetting NullPointerException. My test case is very simple. It has followingsteps:   1. Read a JSON file & wr...
   Author: Something Something , 2020-03-07, 02:36
[expand - 1 more] - Stateful Structured Spark Streaming: Timeout is not getting triggered - Spark - [mail # user]
...Yes.... that was it! It seems it only works if input data is continuouslyflowing. I had stopped the input job because I had enough data but it seemstimeouts work only if the data is continuo...
   Author: Something Something , 2020-03-05, 17:19
[expand - 1 more] - Stateful Spark Streaming: Required attribute 'value' not found - Spark - [mail # user]
...By simply adding 'toJSON' before 'writeStream' the problem was fixed. Maybeit will help someone.On Tue, Mar 3, 2020 at 6:02 PM Something Something wrote:> In a Stateful Spark Streaming ap...
   Author: Something Something , 2020-03-04, 22:48
[expand - 2 more] - Integrating Kafka with Stateful Spark Streaming - Kafka - [mail # user]
...By simply adding 'toJSON' before 'writeStream' the problem was fixed. Maybeit will help someone.On Wed, Mar 4, 2020 at 10:38 AM Something Something <[EMAIL PROTECTED]> wrote:> Yes, ...
   Author: Something Something , 2020-03-04, 22:47