Retrieval-Augmented Generation Consulting

Retrieval-Augmented Generation (RAG) is about using results from a search engine as context for a large language model (LLM) so that it has more domain-specific knowledge when answering a question.

A good RAG implementation involves a lot more than sending the top N documents to ChatGPT. Tweaks can be made at every step:

• Retrieval: this is where search relevance is important, because the context will only be as good as the search results. Larger documents need to be chunked with the best strategy for the use-case, to make sure that the provided text is relevant to the question, while fitting in the LLM's conversation limits.
• Augmenting: a pipeline normally processes the question and builds a query out of it. Here is where questions can be validated or classified and transformed into a structured request that the search engine works well with.
• Generation: besides choosing the right LLM for the use-case, the generative step can be improved via prompt engineering as well as evaluating the quality of generated content.

Though RAG consulting, Sematext can help at every step of the way to implement RAG on top of Elasticsearch, Solr, or OpenSearch.

Professional Services

• Choose the right LLM for the use-case, balancing quality, cost and latency.
• Build and maintain a search pipeline that transforms questions into queries. It can use LLM features (e.g. OpenAI function calling) or an independent set of functions and models.
• Select the right chunking method and integrate it in the indexing pipeline.
• Tweak search relevance using hybrid search.
• Build and maintain a pipeline that makes a context out of search results. This can involve prompt engineering, cutting off irrelevant results, etc.
• Develop a test harness to evaluate and monitor the quality of RAG results.