My team and I are working on a containerized Zookeeper service on top of
DC/OS. We're running into an issue with Kerberos in the following scenario.
Simplified, we have a zk server with the DNS address zk-server.dcos (e.g.
the dns address of the ZK task) and actual hostname zk-server.aws
(Shortened here, but really a standard resolvable AWS private dns address)
and a kafka broker, kafka.dcos.
We can easily setup our Zookeeper and Kafka services to work together,
until we try to enable Kerberos. ZK itself works just fine with Kerberos,
but the Kafka broker is not able to connect to the ZK server:
0. kafka.dcos is started with a zk server list of zk-server.dcos.
1. kafka.dcos starts up, initializing its ZK client.
2. kafka.dcos then attempts to retrieve a ticket from the KDC in order to
talk to zk-server.aws, however the only zk principal known to the kdc is
From reading the source (https://github.com/apache/zookeeper/blob/master/src/java/main/org/apache/zookeeper/client/StaticHostProvider.java#L112
it appears that the zk client is winding up with the actual hostname of the
Being new to the codebase, is this because of a client reverse lookup? Or
because the zk server is telling the client about its hostname? It appears
to be the former.