In the Chat with documentation system using Generative AI demo, the combination of RAG, Langchain, and LLM models allows users to ask queries in layman's terms, retrieve relevant information from the Vector store, and generate accurate and concise answers based on the retrieved data. This integration of retrieval-based and generative-based approaches provides a powerful tool for extracting knowledge from structured sources and delivering user-friendly responses.
\n",
+ "
In the Chat with documentation system using Generative AI demo, the combination of RAG, Langchain, and LLM models allows users to ask queries in layman's terms, retrieve relevant information from the Vector store, and generate accurate and concise answers based on the retrieved data. This integration of retrieval-based and generative-based approaches provides a powerful tool for extracting knowledge from structured sources and delivering user-friendly responses.
\n",
"\n",
- "
In this demo we will build Chatbot using Panel (for chat UI), LangChain, a powerful library for working with LLMs like GPT-3.5, GPT-4, Bloom, etc. and JumpStart in ClearScape notebooks, a system is built where users can ask business questions in natural English and receive answers with data drawn from the relevant databases.
\n",
+ "
In this demo we will build Chatbot using Panel (for chat UI), LangChain, a powerful library for working with LLMs like GPT-3.5, GPT-4, Bloom, etc. and JumpStart in ClearScape notebooks, a system is built where users can ask business questions in natural English and receive answers with data drawn from the relevant databases.
\n",
"\n",
- "
The following diagram illustrates the architecture.
\n",
+ "
The following diagram illustrates the architecture.
\n",
"\n",
"
\n",
"\n",
"\n",
" \n",
- "
Before going any farther, let's get a better understanding of RAG, LangChain, and LLM.
\n",
+ "
Before going any farther, let's get a better understanding of RAG, LangChain, and LLM.
\n",
"\n",
- "
Retrieval-Augmented Generation (RAG):
\n",
- "
RAG is a framework that combines the strengths of retrieval-based and generative-based approaches in question-answering systems.It utilizes both a retrieval model and a generative model to generate high-quality answers to user queries. The retrieval model is responsible for retrieving relevant information from a knowledge source, such as a database or documents. The generative model then takes the retrieved information as input and generates concise and accurate answers in natural language.
\n",
+ "
Retrieval-Augmented Generation (RAG):
\n",
+ "
RAG is a framework that combines the strengths of retrieval-based and generative-based approaches in question-answering systems.It utilizes both a retrieval model and a generative model to generate high-quality answers to user queries. The retrieval model is responsible for retrieving relevant information from a knowledge source, such as a database or documents. The generative model then takes the retrieved information as input and generates concise and accurate answers in natural language.
\n",
"\n",
"\n",
- "
A typical RAG (Retrieval-and-Generation) application has two main components:
\n",
+ "
A typical RAG (Retrieval-and-Generation) application has two main components:
\n",
"\n",
- "
Indexing: a pipeline for ingesting data from a source and indexing it. This usually happens offline. The indexing process involves several steps, including loading the data, splitting it into smaller chunks, and storing and indexing the splits. This is often done using a VectorStore and Embeddings model.
\n",
+ "
Indexing: a pipeline for ingesting data from a source and indexing it. This usually happens offline. The indexing process involves several steps, including loading the data, splitting it into smaller chunks, and storing and indexing the splits. This is often done using a VectorStore and Embeddings model.
\n",
" \n",
- "
Retrieval and generation: the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model. The retrieval process involves searching the index for the most relevant data based on the user query, and then passing that data to the model for generation.
\n",
+ "
Retrieval and generation: the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model. The retrieval process involves searching the index for the most relevant data based on the user query, and then passing that data to the model for generation.
\n",
"\n",
- "
The most common full sequence from raw data to answer looks like:
\n",
- "
Indexing
\n",
- "
\n",
+ "
The most common full sequence from raw data to answer looks like:
\n",
+ "
Indexing
\n",
+ "
\n",
"
Load: Load: First we need to load our data. We'll use PyPDFLoader for this.
\n",
"
Split: Text splitters break large Documents into smaller chunks. This is useful both for indexing data and for passing it in to a model, since large chunks are harder to search over and won't in a model's finite context window. Here, our pdf document will be splits into pages.
\n",
"
Store: We need somewhere to store and index our splits, so that they can later be searched over. This is often done using a VectorStore and Embeddings model
\n",
"
\n",
"\n",
- "
The following diagram illustrates the architecture of load, split and store.
\n",
+ "
The following diagram illustrates the architecture of load, split and store.
Retrieval: At run time, the user enters a query, which is passed to the model for retrieval. The model searches the index for the most relevant data based on the user query, and returns the results.
\n",
"
Generation: Finally, the model generates an answer based on the retrieved data. The answer is then presented to the user.
\n",
"
\n",
" \n",
- "
The following diagram illustrates the architecture of retrieval and generation.
\n",
+ "
The following diagram illustrates the architecture of retrieval and generation.
LangChain is a framework that facilitates the integration and chaining of large language models with other tools and sources to build more sophisticated AI applications. LangChain does not serve its own LLMs; instead, it provides a standard way of communicating with a variety of LLMs, including those from OpenAI and HuggingFace. LangChain accelerates the development of AI applications with building blocks. We learn the leverage the following building blocks in this notebook:
\n",
+ "
Langchain:
\n",
+ "
LangChain is a framework that facilitates the integration and chaining of large language models with other tools and sources to build more sophisticated AI applications. LangChain does not serve its own LLMs; instead, it provides a standard way of communicating with a variety of LLMs, including those from OpenAI and HuggingFace. LangChain accelerates the development of AI applications with building blocks. We learn the leverage the following building blocks in this notebook:
\n",
" \n",
- "\n",
+ "\n",
"
LLMs – LangChain's llm class is designed to provide a standard interface for all LLM it supports.
\n",
"
PromptTemplate - LangChain’s PromptTemplate class are predefined structures for generating prompts for LLM’s. They can be reused across different LLM's.
\n",
"
Chains – When we build complex AI applications, we may need to combine multiple calls to LLM’s and to other components LangChain’s chain class allows us to link calls to LLM’s and components. The most common type of chaining in any LLM application is combining a prompt template with an LLM and optionally an output parser.
\n",
"\n",
"\n",
- "
LLM Models (Large Language Models):
\n",
+ "
LLM Models (Large Language Models):
\n",
"\n",
- "
LLM models refer to the large-scale language models that are trained on vast amounts of text data.\n",
+ "
LLM models refer to the large-scale language models that are trained on vast amounts of text data.\n",
"These models, such as GPT-3 (Generative Pre-trained Transformer 3), GPT-3.5, GPT-4, HuggingFace BLOOM, LLaMA, Google's FLAN-T5, etc. are capable of generating human-like text responses. LLM models have been pre-trained on diverse sources of text data, enabling them to learn patterns, grammar, and context from a wide range of topics. They can be fine-tuned for specific tasks, such as question-answering, natural language understanding, and text generation.\n",
"LLM models have achieved impressive results in various natural language processing tasks and are widely used in AI applications for generating human-like text responses.
Note: The above statements will install the required libraries to run this demo. Be sure to restart the kernel after executing the above lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: 0 0
\n",
+ "
Note: The above statements will install the required libraries to run this demo. Be sure to restart the kernel after executing the above lines to bring the installed libraries into memory. The simplest way to restart the Kernel is by typing zero zero: 0 0
\n",
"
\n",
" \n",
"
\n",
- "
Note: To ensure that the Chatbot interface reflects the latest changes, please reload the page by clicking the 'Reload' button or pressing F5 on your keyboard for first-time only This will update the notebook with the latest modifications, and you'll be able to interact with the Chatbot using the new libraries.
\n",
+ "
Note: To ensure that the Chatbot interface reflects the latest changes, please reload the page by clicking the 'Reload' button or pressing F5 on your keyboard for first-time only This will update the notebook with the latest modifications, and you'll be able to interact with the Chatbot using the new libraries.
In order to utilize this demo, you will need an OpenAI API key. If you do not have one, please refer to the instructions provided in this guide to obtain your OpenAI API key:
\n",
+ "
In order to utilize this demo, you will need an OpenAI API key. If you do not have one, please refer to the instructions provided in this guide to obtain your OpenAI API key:
The Chat with documentation demo aims to demonstrate how users can interact with documents such as insurance policy wordings, invoices, and other similar documents through a conversational interface.
\n",
+ "
The Chat with documentation demo aims to demonstrate how users can interact with documents such as insurance policy wordings, invoices, and other similar documents through a conversational interface.
\n",
"\n",
- "
The Traveler Easy Single Trip - International insurance policy is a comprehensive travel insurance plan that provides cover for a wide range of risks, including medical expenses, trip cancellation, loss of luggage, and personal accident. The policy is designed to be affordable and flexible, and it can be purchased online or over the phone.
"
+ "
The Traveler Easy Single Trip - International insurance policy is a comprehensive travel insurance plan that provides cover for a wide range of risks, including medical expenses, trip cancellation, loss of luggage, and personal accident. The policy is designed to be affordable and flexible, and it can be purchased online or over the phone.
In OpenAI's language models, the temperature parameter controls the randomness of the generated text. It affects the diversity and creativity of the model's responses. It is always a number between 0 and 1. A temperature of 0 means the responses will be very straightforward, almost deterministic (meaning you almost always get the same response to a given prompt). A temperature of 1 means the responses can vary wildly.
\n",
+ "
In OpenAI's language models, the temperature parameter controls the randomness of the generated text. It affects the diversity and creativity of the model's responses. It is always a number between 0 and 1. A temperature of 0 means the responses will be very straightforward, almost deterministic (meaning you almost always get the same response to a given prompt). A temperature of 1 means the responses can vary wildly.
\n",
"\n",
"\n",
- "
A higher temperature value, such as 1.0, increases the randomness and diversity of the generated output. This can lead to more varied and surprising responses, but it may also result in less coherence and occasional nonsensical outputs. A higher temperature means that the model might select a word with slightly lower probability, leading to more variation, randomness and creativity. A very high temperature therefore increases the risk of hallucination, meaning that the model starts selecting words that will make no sense or be off-topic.
\n",
+ "
A higher temperature value, such as 1.0, increases the randomness and diversity of the generated output. This can lead to more varied and surprising responses, but it may also result in less coherence and occasional nonsensical outputs. A higher temperature means that the model might select a word with slightly lower probability, leading to more variation, randomness and creativity. A very high temperature therefore increases the risk of hallucination, meaning that the model starts selecting words that will make no sense or be off-topic.
\n",
"\n",
- "
On the other hand, a lower temperature value, such as 0.2 or below, reduces randomness and makes the model's output more focused and deterministic. The generated text is likely to be more conservative, sticking closely to patterns observed in the training data. A temperature of 0 means roughly that the model will always select the highest probability word.
\n",
+ "
On the other hand, a lower temperature value, such as 0.2 or below, reduces randomness and makes the model's output more focused and deterministic. The generated text is likely to be more conservative, sticking closely to patterns observed in the training data. A temperature of 0 means roughly that the model will always select the highest probability word.
\n",
"\n",
- "
Choosing an appropriate temperature value depends on the desired output. Higher temperatures can be useful for creative tasks or brainstorming, while lower temperatures are preferred when you need more control over the output, such as when generating specific responses or following a particular style.
"
+ "
Choosing an appropriate temperature value depends on the desired output. Higher temperatures can be useful for creative tasks or brainstorming, while lower temperatures are preferred when you need more control over the output, such as when generating specific responses or following a particular style.
Vector stores are a powerful tool for storing and searching unstructured data in Language Large Models (LLMs). By embedding unstructured data into numerical vectors, vector stores allow you to easily store and search through vast amounts of data, such as text documents, images, or videos.
\n",
- "
One of the key benefits of using a vector store in LLM applications is that it allows you to perform near-instant searches. This means you can quickly find the information you need, without having to manually sift through each piece of data.
\n",
+ "
Vector stores are a powerful tool for storing and searching unstructured data in Language Large Models (LLMs). By embedding unstructured data into numerical vectors, vector stores allow you to easily store and search through vast amounts of data, such as text documents, images, or videos.
\n",
+ "
One of the key benefits of using a vector store in LLM applications is that it allows you to perform near-instant searches. This means you can quickly find the information you need, without having to manually sift through each piece of data.
\n",
"\n",
- "
Vector stores also use advanced similarity algorithms to retrieve the most relevant embedding vectors for your query. This means you can be sure that you're getting the most relevant results, even when dealing with large amounts of data.But that's not all. Vector stores also allow you to store and search through unstructured data in a way that is highly scalable. This means you can easily handle large amounts of data, without having to worry about performance issues.
\n",
+ "
Vector stores also use advanced similarity algorithms to retrieve the most relevant embedding vectors for your query. This means you can be sure that you're getting the most relevant results, even when dealing with large amounts of data.But that's not all. Vector stores also allow you to store and search through unstructured data in a way that is highly scalable. This means you can easily handle large amounts of data, without having to worry about performance issues.
\n",
"\n",
- "
One of the key aspects of working with vector stores is creating the vectors to put in them. This is usually done via embeddings, which are numerical representations of text or other unstructured data. Therefore, it is recommended that you familiarize yourself with the text embedding model interfaces before diving into this.
\n",
+ "
One of the key aspects of working with vector stores is creating the vectors to put in them. This is usually done via embeddings, which are numerical representations of text or other unstructured data. Therefore, it is recommended that you familiarize yourself with the text embedding model interfaces before diving into this.
\n",
"\n",
- "
In this demo, we will use the OpenAI embeddings method to generate the embeddings. OpenAI embeddings are a type of word embedding that can be used to represent products in a way that captures their semantic meaning. To generate embeddings for a insurance document, we will use the pages collected in the previous step by PyPDFLoader. We will use the OpenAI Embeddings API to generate embeddings for entire document. Please refer to the Embeddings documentation for more information about embeddings and types of models available.
\n",
+ "
In this demo, we will use the OpenAI embeddings method to generate the embeddings. OpenAI embeddings are a type of word embedding that can be used to represent products in a way that captures their semantic meaning. To generate embeddings for a insurance document, we will use the pages collected in the previous step by PyPDFLoader. We will use the OpenAI Embeddings API to generate embeddings for entire document. Please refer to the Embeddings documentation for more information about embeddings and types of models available.
\n",
"\n",
- "
The OpenAI Embeddings API takes a text string as input and returns a vector of numbers that represent the embedding. The length of the vector depends on the model that you are using. For example, the text-embedding-ada-002 model returns a vector of 1536 numbers.
"
+ "
The OpenAI Embeddings API takes a text string as input and returns a vector of numbers that represent the embedding. The length of the vector depends on the model that you are using. For example, the text-embedding-ada-002 model returns a vector of 1536 numbers.
A retriever is an interface that allows you to retrieve documents based on an unstructured query. It is more general than a vector store, as it does not need to store documents, only retrieve them. Vector stores can be used as the backbone of a retriever, but there are other types of retrievers as well.One of the most common types of search used in vector stores is similarity search, which compares the similarity between the query and the stored documents to retrieve the most relevant results. This is the type of search used in our vector store retriever.
"
+ "
A retriever is an interface that allows you to retrieve documents based on an unstructured query. It is more general than a vector store, as it does not need to store documents, only retrieve them. Vector stores can be used as the backbone of a retriever, but there are other types of retrievers as well.One of the most common types of search used in vector stores is similarity search, which compares the similarity between the query and the stored documents to retrieve the most relevant results. This is the type of search used in our vector store retriever.
PromptTemplate in Language Large Models (LLMs) are pre-defined templates that guide the user in generating prompts for the model. These templates provide a structure for the user to input specific information, such as topic, tone, and style, to help the model generate more accurate and relevant responses. By using prompt templates, users can create more effective prompts and improve the quality of the model's responses. A template may include instructions, few-shot examples, and specific context and questions appropriate for a given task.
"
+ "
PromptTemplate in Language Large Models (LLMs) are pre-defined templates that guide the user in generating prompts for the model. These templates provide a structure for the user to input specific information, such as topic, tone, and style, to help the model generate more accurate and relevant responses. By using prompt templates, users can create more effective prompts and improve the quality of the model's responses. A template may include instructions, few-shot examples, and specific context and questions appropriate for a given task.
We have designed this demo to handle follow-up questions with ease, even when they contain references to past chat history.
\n",
+ "
We have designed this demo to handle follow-up questions with ease, even when they contain references to past chat history.
\n",
" \n",
- "
For example, if you ask \"Does this policy cover Loss of or Damage to the Insured’s Articles?\" and then follow up with \"what is the reimbursement limit per Baggage?\", our system will understand the context and provide an appropriate response.
\n",
+ "
For example, if you ask \"Does this policy cover Loss of or Damage to the Insured’s Articles?\" and then follow up with \"what is the reimbursement limit per Baggage?\", our system will understand the context and provide an appropriate response.
\n",
" \n",
- "
Here's how it works: we add an additional step prior to retrieval that combines the chat history and the question into a standalone question. This allows us to perform the standard retrieval steps of looking up relevant documents from the retriever and passing those documents and the question into a question answering chain to return a response.
"
+ "
Here's how it works: we add an additional step prior to retrieval that combines the chat history and the question into a standalone question. This allows us to perform the standard retrieval steps of looking up relevant documents from the retriever and passing those documents and the question into a question answering chain to return a response.
If the chatbot didn't work when you pressed ENTER, on your first time using this demo on your environment, did you use F5 to reload the site? See instructions at the top of the notebook.\n",
+ "
If the chatbot didn't work when you pressed ENTER, on your first time using this demo on your environment, did you use F5 to reload the site? See instructions at the top of the notebook.\n",
"If you asked a question and got no response after a few minutes, it is possible that you will need to type 0 0 to restart the kernel and re-run the demo. Questions outside the model seem to confuse the chatbot.
In conclusion, the Chat with documentation demo is important because it showcases the potential of conversational interfaces to facilitate user interaction with various types of documents. This technology can make it easier for users to access and understand complex information, such as insurance policy wordings and invoices.
"
+ "\n",
+ "
In conclusion, the Chat with documentation demo is important because it showcases the potential of conversational interfaces to facilitate user interaction with various types of documents. This technology can make it easier for users to access and understand complex information, such as insurance policy wordings and invoices.
![\"architecture\"](\"images/header2.png\")
![\"rag](\"images/rag_load_store.png\")
![\"retrieval](\"images/rag_retrieval_generation.png\")