Retrieval Augmented Generation (RAG) represents a significant step in improving the generative capabilities of Large Language Models (LLMs). Indeed, RAG is an innovative paradigm that is based on the integration of two crucial components: the extraction of relevant information from a knowledge base and the ability of a generative model to reframe this information into a coherent text. In the first stage, a search model, or Retriever, is used to index and extract fragments of text. These fragments are then provided to an LLM, which uses this information base to provide relevant and accurate answers.

The power of this approach allows an LLM to limit incorrect or misleading information, thereby reducing hallucinations, i.e., the tendency of generative models to produce false information supported by seemingly solid reasoning. While the internal knowledge of an LLM is extensive, it is necessarily generic or limited because of the slowness and cost of training. The use of an external knowledge base provides a more accurate guide to text generation, overall improving the reliability and consistency of the generated responses.

After an introduction on the basic concepts behind the RAG, the purpose of the presentation is to build a RAG through the use of Haystack, an open-source framework widely used in Natural Language Processing (NLP). Haystack has many features that make it an optimal choice for implementing a RAG, such as flexible document management, the ability to index heterogeneous data, and ease of integration with recent LLMs.

As of August 2023, Haystack has released a version still under development of a major update that will lead to the release of Haystack 2.0 or v2. During the presentation, we will see how new logic in building Pipelines for text retrieval and generation makes Haystack an even more robust tool for LLM-based applications.

Through a practical example, it will be shown how to implement a RAG with Haystack’s methods and functions and what changes in the new v2 version. Pipeline architecture and configuration choices will be examined to understand how to translate the theoretical concepts exposed in the first part of the presentation into an efficient and reproducible scheme. The presentation will conclude with a discussion of the performance and limitations of RAG when put into production within a software product.