RetrievalQA Chain

Developing a production-grade LLM application requires many refinements, but tracking multiple versions of prompts, models, and other components can be cumbersome. The LangChain Hub offers a centralized registry to manage and version your LLM artifacts efficiently. It even lets you interact with these artifacts directly in the browser to facilitate easier collaboration with non-technical team members.

In its initial release (08/05/2023), the hub is limited to prompt management, but we plan to add support for other artifacts soon.

In this walkthrough, you will get started using the hub to manage prompts for a retrieval QA chain. You will go through the following steps:

1. Load prompt from Hub
2. Initialize Chain
3. Run Chain
4. Commit any new changes to the hub

Prerequsites

a. Set up your LangSmith account

While you can access public prompts without an account, pushing new prompts to the hub requires a LangSmith account. Create your account at https://smith.langchain.com and log in.

Next, navigate to the hub home. If you haven't already created a "handle", you will be prompted to do so. Your prompts and other artifacts will be saved within the namespace '<handle>/prompt-name', so choose one that you are comfortable sharing.

b. Configure environment

To use the hub, you'll want to use a recent version of LangChain and the langchainhub package. Install them with the following command:

%pip install -U langchain langchainhub --quiet

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

Finally, generate an API Key from your "personal" organization by navigating to the LangSmith dashboard, and then set it in the cell below.

Note: Currently (08/04/2023), only API keys from your 'personal' organization are supported! If you see a '403' error at any point in this walkthrough, please confirm you've set a valid API key.

import os

# Update with your API URL if using a hosted instance of Langsmith.
os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com"
os.environ["LANGCHAIN_API_KEY"] = "YOUR API KEY"  # Update with your API key
# Update with your API URL if using a hosted instance of Langsmith.
os.environ["LANGCHAIN_HUB_API_URL"] = "https://api.hub.langchain.com"
os.environ["LANGCHAIN_HUB_API_KEY"] = "YOUR API KEY"  # Update with your Hub API key

1. Load prompt

Now it's time to load the prompt from the hub. We will use the latest version of this retrieval QA prompt and later initialize the chain with it.

# RAG prompt
from langchain import hub

# Loads the latest version
prompt = hub.pull("rlm/rag-prompt", api_url="https://api.hub.langchain.com")

# To load a specific version, specify the version hash
# prompt = hub.pull("rlm/rag-prompt:50442af1")

2. Create the QA chain

Now that we've selected our prompt, initialize the chain. For this example, we will create a basic RetrievalQA over a vectorstore retriever.

Loading the data requires some amount of boilerplate, which we will run below. While the specifics aren't important to this tutorial, you can learn more about Q&A in LangChain by visiting the docs.

# %pip install chromadb --quiet

# Load docs
from langchain.document_loaders import WebBaseLoader

loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
data = loader.load()

# Split
from langchain.text_splitter import RecursiveCharacterTextSplitter

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

# Store splits
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma

vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

# LLM
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

Initialize the chain. With the data added to the vectorstore, we can initialize the chain. We will pass the prompt in via the chain_type_kwargs argument.

# RetrievalQA
from langchain.chains import RetrievalQA

qa_chain = RetrievalQA.from_chain_type(
    llm, retriever=vectorstore.as_retriever(), chain_type_kwargs={"prompt": prompt}
)

3. Run Chain

Now that the chain is initialized, you can run it just like you would normally.

question = "What are the approaches to Task Decomposition?"
result = qa_chain({"query": question})
result["result"]

'The approaches to task decomposition include using LLM with simple prompting, task-specific instructions, and human inputs.'

4. (Optional) Commit any new changes to the hub

After debugging, evaluating, or monitoring your chain in some deployment, you may want to make some changes to the prompt. You can do so by adding this prompt under your handle's namespace.

Note: If you receive a '403' forbidden error, you may need to set your LANGCHAIN_HUB_API_KEY to a personal API key.

handle = "YOUR HUB HANDLE"  # Replace with your handle!
hub.push(f"{handle}/rag-prompt", prompt)

'https://smith.langchain.com/hub/infra-guy/rag-prompt/597661da'

Now you can view your prompt in the hub. It should look something like this:

Let's say you've tried this prompt out and have derived a better one for your use case. You can push the updated prompt to the same key to "commit" a new version of the prompt.

For instance, let's add a system message to the prompt:

# You may try making other changes and saving them in a new commit.
from langchain import schema

prompt.messages.insert(
    0,
    schema.SystemMessage(
        content="You are a precise, autoregressive question-answering system."
    ),
)

# Pushing to the same prompt "repo" will create a new commit
hub.push(f"{handle}/rag-prompt", prompt)

'https://smith.langchain.com/hub/infra-guy/rag-prompt/e0569108'

Now the newest version of the prompt is saved as the latest version. It should look something like this:

You can view all saved versions by navigating to the "commits" tab.

Conclusion

In this tutorial, you learned how to use the hub to manage prompts for a retrieval QA chain. The hub is a centralized location to manage, version, and share your prompts (and later, other artifacts).

For more information, check out the docs or reach out to support@langchain.dev.