Our hbase cluster 1.3.2 crashed due to network switch reset. It’s expected.
We restarted the cluster afterwards.
What's not expevted is the application service using hbase failed even
after hbase cluster was back. The reason is because data that should’ve
been deleted reappeared after cluster restarted.
Those deletes were created a couple months ago and I didn’t found them from
a snapshot that I took a month ago.
We didn’t turn off major compaction so I expected deletes would have been
I found this issue, https://issues.apache.org/jira/browse/HBASE-13473
which discussed tombstone markers lost after Stripe compaction. But, we
didn't specify compaction policy meaning Stripe compaction is not used so i
don't think it applies.
So, has this ever occurred to anyone? I'm at a loss here. Any comment would
be appreciated. Thanks.