Subject: kafka-producer-perf-test.sh transaction support?


I am trying to test a 3 node Kafka cluster using the producer and consumer test perf scripts that ship with the binaries. When I am trying to enable transaction in the producer script it runs into errors. For example:

# bin/kafka-producer-perf-test.sh --topic testtopic1 --num-records 10000 --throughput -1 --record-size 256 --producer-props bootstrap.servers=172.18.0.4:9092,172.18.0.5:9092,172.18.0.6:9092 acks=all transactional.id=1111 enable.idempotence=true
Exception in thread "main" java.lang.IllegalStateException: Cannot perform a 'send' before completing a call to initTransactions when transactions are enabled.
at org.apache.kafka.clients.producer.internals.TransactionManager.failIfNotReadyForSend(TransactionManager.java:362)
at org.apache.kafka.clients.producer.internals.TransactionManager.maybeAddPartitionToTransaction(TransactionManager.java:341)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:915)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856)
at org.apache.kafka.tools.ProducerPerformance.main(ProducerPerformance.java:143)

A non-transactional test succeeds though:

bin/kafka-producer-perf-test.sh --topic testtopic1 --num-records 10000 --throughput -1 --record-size 256 --producer-props bootstrap.servers=172.18.0.4:9092,172.18.0.5:9092,172.18.0.6:9092 acks=all
10000 records sent, 6997.900630 records/sec (1.71 MB/sec), 112.32 ms avg latency, 499.00 ms max latency, 101 ms 50th, 206 ms 95th, 213 ms 99th, 214 ms 99.9th.

The version I am using is: kafka_2.12-2.3.0

Does the producer perf test script support transactions? If it does, how do we run the producer with transaction support?

Sincerely,
Anindya Haldar
Oracle Responsys