Subject: About KafkaConsumer and WM'ing and EventTime charactersitics


Thank you. This though is a little different.

The producer of the kafka message attaches a time stamp
https://issues.apache.org/jira/browse/KAFKA-2511.  I do not see how I can
get to that timestamp through a any stream abstraction over
FlinkKafkaConsumer  API even though it is available here
https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/ConsumerRecord.html
being used here
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java#L141

All I want to do is this

* Pull from kafka topic . This topic is been written too with a time stamp
on each kafka record.
* Write to hdfs using StreamingSink BUT make buckets that * honor ingestion
time's  water mark. *

Questions is,

*If  we have TimeCharacteristic as IngestionTime,  does the context's
watermark  in   getBucketId(KafkaRecord element, Context context)
in BucketAssigner.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.7/api/java/org/apache/flink/streaming/api/functions/sink/filesystem/BucketAssigner.html>
 reflect the kafka record time stamp in
https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/ConsumerRecord.html
<https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/ConsumerRecord.html>
given this "*automatic timestamp assignment and automatic watermark
generation." is done if  *TimeCharacteristic is **IngestionTime*  (* here
<https://ci.apache.org/projects/flink/flink-docs-stable/dev/event_time.html>)*
Regards.


On Tue, Jan 29, 2019 at 8:42 PM Congxian Qiu <[EMAIL PROTECTED]> wrote: