Subject: ram estimate for docvalues is incorrect


A couple major versions ago, Lucene required tons of heap memory to keep a
reader open, e.g. norms were on heap and so on. To my knowledge, the only
thing that is now kept in memory and is a function of maxDoc is live docs,
all other codec components require very little memory. I'm actually
wondering that we should remove memory accounting on readers. When Lucene
used tons of memory we could focus on the main contributors to memory usage
and be mostly correct. But now given how little memory Lucene is using it's
quite hard to figure out what are the main contributing factors to memory
usage. And it's probably not that useful either, why is it important to
know how much memory something is using if it's not much?

So I'd be curious to know more about your use-case for reader caching.
Would we break your use-case if we removed memory accounting on readers?
Given the lines that you are pointing out, I believe that you either have
many fields or many segments if these maps are using lots of memory?
On Wed, May 27, 2020 at 9:52 PM John Wang <[EMAIL PROTECTED]> wrote:
--
Adrien