Table of Contents

1. Setting up the Environment

   • This section guides you on how to set up the environment by creating an account with the NVIDIA API Catalog and installing necessary tools like LangChain, NVIDIA AI Endpoints, and FAISS.

2. Loading Relevant Documents

   • Learn how to load a specific document (e.g., NVIDIA publication on multi-modal LLMs) for the examples in this tutorial. It also highlights the importance of selecting the right chunk size for RAG performance.

3. Creating a Basic Retriever

   • Create a basic retriever using the loaded document to search for the most relevant chunks based on a simple retrieval algorithm. This section demonstrates how to output the top 45 relevant chunks for a given query.

4. Adding a Re-Ranking Step

   • Integrate a re-ranking step using the NeMo Retriever reranking NIM to further enhance the relevance of search results. Explore how re-ranking can optimize the order of results based on their relevance to the query.

5. Connecting to a RAG Pipeline

   • Explore how to connect the re-ranking step to a RAG (Retriever-Augmented Generation) pipeline to ensure the most relevant chunks are used for augmenting the original query. This section emphasizes the importance of optimizing chunk size and selecting suitable LLMs for an effective RAG pipeline.

6. Additional Resources

   • Find more information about using re-ranking, RAG pipelines, and leveraging NVIDIA AI LangChain endpoints to enhance search and generation tasks.

---

1. Setting up the Environment

To begin, create a free account with the NVIDIA API Catalog and follow steps to install LangChain, NVIDIA AI Endpoints, and FAISS for setting up the environment.

2. Loading Relevant Documents

Load the NVIDIA publication on multi-modal LLMs, VILA: On Pre-training for Visual Language Models, for examples in this tutorial. Experiment with chunk sizes for optimal RAG performance.

3. Creating a Basic Retriever

Develop a basic retriever using the loaded document to search for and output the top 45 relevant chunks based on a simple retrieval algorithm.

4. Adding a Re-Ranking Step

Integrate the NeMo Retriever reranking NIM to re-rank search results for enhanced relevance based on query semantics and multiple data sources.

5. Connecting to a RAG Pipeline

Connect the re-ranking step to a RAG pipeline to maximize relevance and ensure the most relevant chunks are utilized for query augmentation.

6. Additional Resources

Discover more about leveraging re-ranking, RAG pipelines, and NVIDIA AI LangChain endpoints for advanced search and generation tasks.

---