Subject: [prometheus-developers] Accepting features into TSDB required by other users (but not Prometheus)


On Fri, 31 Jan 2020 at 21:51, Goutham Veeramachaneni <[EMAIL PROTECTED]>
wrote:

It was a single function that only Thanos used, and there were already
(unrelated) plans for them to move to a different way of doing things. This
was one that iirc they had asked really really nicely for as there was no
sane alternative.

Here we have Thanos/Cortex not merely asking for an accessor or otherwise
to make some internals public (which we're generally fine with), but to add
a whole first-class feature that Prometheus itself doesn't need. Having to
build up and maintain code infrastructure for a 3rd party feature is messy,
and it'll also have a performance impact on Prometheus itself (we'd be
adding contention on series creation).  We'd also now to implement
Thanos/Cortex specific design decisions in the tsdb. What happens when a
WAL replay hits the limit? What happens when a scrape hits the limit or
otherwise fails? You can't rollback a series creation, only sample
ingestion.

Basically I don't think adding this sort of feature to TSDB is viable
purely for the sake of 3rd parties, it'd have to be justified and designed
to be added to Prometheus too. Given the above implementation
considerations, I don't think it's a sane feature for Prometheus itself.

Complexity isn't free. This sort of thing tends to be death by a thousand
cuts, until it's no longer practical to change the TSDB as it's just too
complicated to safely/sanely change (and it's already quite complicated
already).

Brian

Brian Brazil
www.robustperception.io