And I PUT that index template to ES. After that, I delete the filebeat-* indexes so they get recreated with the new template. The new index gets created, but even though I turned on debug logging and see logstash flushing data to ES, the index just sits there with 0 documents
GET /_cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size [...] yellow open filebeat-6.1.0-2018.02.14 Zz8Jkf1XTyKxor-mL9fT-A 3 1 0 0 699b 699b
If I delete the index template, the logs start flowing to the index again ` yellow open filebeat-6.1.0-2018.02.14 3aR0_cDHQIeYU6UR0CC0ww 5 1 6919 0 3.5mb 3.5mb`
Any idea what's happening here? I'm testing this on a single node ES cluster while I take ELK for a spin.
What does you Elasticsearch output look like in the Logstash config? Are you be any chance setting an incorrect document type that clashes with your index template (Elasticsearch 6.x can only have 1 type per index)? Is there anything in the Elasticsearch logs?
I think the default type used by the Elasticsearch output plugin might be `logs`. Check what is set in an index using the old template and then update other `document_type` in the plugin or the index template accordingly.
Oh thank you thank you thank you! That was exactly my problem. I changed that property from "doc" to "logs" and it worked perfectly! Sorry it took me so long to understand what you were suggesting, it all makes much more sense now.