dawn breaks 2018-01-11, 09:22
  Uwe Schindler 2018-01-11, 08:32
  dawn breaks 2018-01-11, 02:40
  dawn breaks 2018-01-11, 14:59
Hi lubin,

the terms dictionary is using the "tim" and "tip" files. It should be approximately in the dimension of the FST.

Do you have all indexes running in the same JVM or is it 10 servers? Because then the numbers look correct. If you really want to have such an large index in a single machine using a single JVM, you should plan for more heap space. I'd start with 12 GiB of heap space to run this index.

A last recommendation: If you update your index during runtime, make sure that you correctly close the outdated IndexReaders/IndexSearchers (e.g. using SearcherManager), so you don't have orphaned instances of IndexReader consuming heap space and disk space, because the files can't be fully deleted as long as those are open!


Uwe Schindler
Achterdiek 19, D-28357 Bremen