Video and Presentation: Indexing and Searching Logs with Elasticsearch or Solr

Interested in log indexing using Elasticsearch or Solr?  Also interested in searching and analyzing logs in real time?

This topic really hits home for us since we released our log analytics tool, Logsene and we also offer consulting services for logging infrastructure.  If you are reading this and looking for a new opportunity then you might be interested to hear that we are hiring worldwide.

If you are into logging like we are, then you will want to check out this presentation delivered by Sematext’s own Radu Gheorghe to the NYC Search, Discovery and Analytics Meetup held recently at Pivotal Labs.  For the purposes of this presentation the term “logs” ranges from server logs and application events to metrics and even social media information.

The presentation has three parts:

  1. Overview of logging tools that play nicely with Elasticseach and Solr (like Logstash, Apache Flume or rsyslog)
  2. Performance tuning and scaling Elasticsearch and Solr
  3. Demo of an end-to-end solution

Here you go – enjoy!

Presentation: Introduction to Elasticsearch

During the New York Search, Discovery and Analytics group in November, we did a presentation on Introduction to Elasticsearch. It was about how you can use Elasticsearch to store, search and analyze your data in near-realtime. It included a demo on the basic functionality (index, retrieve, update, delete, search, facet) as well as scaling, and some tips for performance tuning and monitoring your Elasticsearch cluster.

Please tweet about Presentation: Introduction to Elasticsearch

If you attended, you might want to see Gilt’s blog post about it, which includes some nice pictures. If you didn’t attend, of if you just want to refresh your memory, you can find the video (courtesy of g33ktalk) and the slides below.

Presentation: Intro to HBase

Last week we had our inaugural HBase NYC meetup.  About 30 people turned up – not bad for the first meetup.  Etsy, Sematext old customer and Brooklyn neighbours, provided the space, AV equipment and help, as well as their fridge with beer – thanks!  Alex gave two talks, first the Introduction to HBase whose slides (audio) are below and then Intro to HBase Internals and Schema Design.

We’re hiring people who want to work WITH and ON HBase and other Big Data technologies.  See jobs @ sematext.

Announcement: HBase NYC Meetup

We make so much use of HBase at Sematext, we felt the need to start a New York City HBase Meetup – HBase NYC.  We just created the meetup, but there are already over 40 members.  We do not have any meetups scheduled just yet, but have a few initial ideas.  If you are in New York and have interest in HBase, this is the meetup to join.  If you would like to host the meetup or if you would like to present please let us know.  Presentation suggestions are very welcome, too!



UIMA Talk at NY Search & Discovery Meetup

This coming February, the NY Search & Discovery meetup will be hosting Dr. Pablo Duboue from IBM Research and his talk about UIMA.

This following is an “excerpt from the blurb” about Pablo’s talk:

“… In this talk, I will briefly present UIMA basics before discussing full UIMA systems I have been involved in the past (including our Expert Search system in TREC Enterprise Track 2007). I will be talking about how UIMA supported the construction of our custom NLP tools. I will also sketch the new characteristics of the UIMA Asynchronous Scaleout (UIMA AS) subproject that enable UIMA to run Analysis Engines in thousands of machines… “

The talk/presentation will be on February 24, 2010.  Sign up (free) at .

Sponsoring NYC Search & Discovery Meetup

We were busy in 2009, and so was Otis, who organized the NYC Search & Discovery Meetup, and which we happily sponsor.  So far a few interesting talks/presentations have happened and more are scheduled for 2010 (presenters for the next few months are already known).  Also, Otis and Daniel gave Faceted Search presentation (slides) at the NY CTO Club in December 2009, and Otis presented Lucene (slides and video) at the Semantic Web Meetup in October 2009.  More to come in 2010!