Subject: [VOTE] Merge HDFS-12943 branch to trunk - Consistent Reads from Standby

Hi Hadoop developers,

I would like to propose to merge to trunk the feature branch HDFS-12943 for
Consistent Reads from Standby Node. The feature is intended to scale read
RPC workloads. On large clusters reads comprise 95% of all RPCs to the
NameNode. We should be able to accommodate higher overall RPC workloads (up
to 4x by some estimates) by adding multiple ObserverNodes.

The main functionality has been implemented see sub-tasks of HDFS-12943.
We followed up with the test plan. Testing was done on two independent
clusters (see HDFS-14058 and HDFS-14059) with security enabled.
We ran standard HDFS commands, MR jobs, admin commands including manual
failover.
We know of one cluster running this feature in production.

There are a few outstanding issues:
1. Need to provide proper documentation - a user guide for the new feature
2. Need to fix automatic failover with ZKFC. Currently it does not doesn't
know about ObserverNodes trying to convert them to SBNs.
3. Scale testing and performance fine-tuning
4. As testing progresses, we continue fixing non-critical bugs like
HDFS-14116.

I attached a unified patch to the umbrella jira for the review and Jenkins
build.
Please vote on this thread. The vote will run for 7 days until Wed Dec 12.

Thanks,
--Konstantin
  Zhe Zhang 2018-12-06, 04:08
  Konstantin Shvachko 2018-12-06, 19:23
  Daryn Sharp 2018-12-06, 18:38
  Anu Engineer 2018-12-06, 19:08
  Konstantin Shvachko 2018-12-13, 18:10
  Konstantin Shvachko 2018-12-13, 18:24
  Vinod Kumar Vavilapalli 2018-12-13, 18:29
  Yongjun Zhang 2018-12-13, 06:36
  Chen Liang 2018-12-06, 22:41
  Yongjun Zhang 2018-12-06, 07:24
  Konstantin Shvachko 2018-12-08, 00:10