📘
By the way, hallucination in LLMs refers to the generation of information or content that wasn't present or implied in the input data. It's as if the model is dreaming up details, hence the term "hallucination". This can sometimes lead to creative and unexpected outputs but can also result in inaccuracies or inconsistencies in the generated text.
```python
# Initialize ChatOpenAI model
model = ChatOpenAI(model_name=language_model, temperature=0.2)
# We set gpt-3.5-turbo by default in the env variables.
# Use gpt-4 for better and more accurate responses.
```
### Creating the Conversational Retrieval Chain
Let's start with the Conversational Retrieval Chain. This is where the magic happens! The Conversational Retrieval Chain takes our question, retrieves the most relevant vectors from our database, and uses them as context to generate an answer from our language model.
```python
# Initialize ConversationalRetrievalChain
qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)
# Initialize chat history
chat_history = []
```
### Helper functions for user interaction
This is where we set up how the user can ask questions and how the answers will be displayed.
We'll have two functions, `get_user_input()` and `print_answer()`. These functions take user input, process it, and provide a response.
```python
def get_user_input():
"""Get user input and handle 'quit' command."""
question = input("\nPlease enter your question (or 'quit' to stop): ")
if question.lower() == 'quit':
return None
return question
def print_answer(question, answer):
"""Format and print question and answer."""
print(f"\nQuestion: {question}\nAnswer: {answer}\n")
```
The `get_user_input()` function is the gateway for user interaction. It prompts the user to enter a question with the message "Please enter your question (or 'quit' to stop): ". This function also handles a special command, 'quit'. If the user types 'quit', the function will return None, signaling the program to stop taking further inputs. This is an elegant way to exit the program instead of using `cmd` + `c`; otherwise, it returns the question entered by the user.
The `print_answer(question, answer)` function, on the other hand, is responsible for presenting the response to the user. It takes two parameters: the original question asked by the user and the answer generated by the system. It then formats these two pieces of information in a readable manner and prints them out. This function ensures the user can clearly see their original question and the corresponding answer, providing a smooth and understandable user experience.
## Running the Program
Finally, we're ready to run our program! We'll create a loop that prompts the user for a question, uses our Conversational Retrieval Chain to generate an answer, and then prints the answer. We'll also print the token usage for each question, which can help us keep track of our API usage.
```python
def main():
"""Main program loop."""
while True:
question = get_user_input()
if question is None: # User has quit
break
# Display token usage and approximate costs
with get_openai_callback() as tokens_usage:
result = qa({"question": question, "chat_history": chat_history})
chat_history.append((question, result['answer']))
print_answer(question, result['answer'])
print(tokens_usage)
if __name__ == "__main__":
main()
```
Now you just need to run it with `python3 chat.py` and ask some questions! Here is an example of a few interactions:
> Question: can you summarize what this episode of the podcast is about?
>
> Answer: This episode of the podcast is mainly focused on web3 security, specifically discussing a faucet that was recently drained for almost 200 in gorillaith. The speakers talk about how it happened, why it's important to take precautionary measures, and the need for usability and mainstream adoption in the web3 space. They also mention previous episodes where they discussed AI adventures and the chain sector plugin.
>
> Question: who are the speakers?
>
> Answer: The speakers of this episode of the podcast are Tabasco (Ethan), Ache (Director of Developer Experience), and David (Developer Advocate).
And it will look like this in the terminal:
```python
Question: who are the speakers?
Answer: The speakers of this episode of the podcast are Tabasco (Ethan), Ache (Director of Developer Experience), and David (Developer Advocate).
Tokens Used: 2577
Prompt Tokens: 2537
Completion Tokens: 40
Successful Requests: 2
Total Cost (USD): $0.005154000000000001
```
We can see that the responses are extremely relevant and well formatted; considering how simple the application is, this is a great starting point.
But you will notice some details are partially wrong, and this is not a problem with the LLM or our app. Since we rely on the YouTube transcription of the audio from the video, things like names, especially, might be transcripted wrong; for example, 'gorillaith' should be 'Goerli ETH', and the name of the Chainstack's Director of Developer Experience is 'Ake' and not "Ache'. We can easily fix these details when scraping the data we need.
> Go check the podcast on YouTube to see how accurate our QA bot is!
>
> [**FAFO & Chill Ep3**](https://www.youtube.com/watch?v=nFFA0lFswSA)
This code interacts with a local vector DB, but all you have to do to use a cloud instance is to change the path in the environment variables to your cloud vector DB, just like we did in the previous article explaining how to use Activeloop.
And you can use the same principle to interact with a Pinecone vector DB; check the article dedicated to Pinecone.
* [The ultimate LangChain series — Pinecone vector database](https://blog.davideai.dev/the-ultimate-langchain-series-pinecone-vector-database)
## Full code
```python
import os
from dotenv import load_dotenv
from langchain.vectorstores import DeepLake
from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.embeddings import OpenAIEmbeddings
from langchain.callbacks import get_openai_callback
# Load environment variables from .env file
load_dotenv()
# Set environment variables
os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY')
os.environ['ACTIVELOOP_TOKEN'] = os.getenv('ACTIVELOOP_TOKEN')
language_model = os.getenv('LANGUAGE_MODEL')
# Set DeepLake dataset path
DEEPLAKE_PATH = os.getenv('DATASET_PATH')
# Initialize OpenAI embeddings and disallow special tokens
EMBEDDINGS = OpenAIEmbeddings(disallowed_special=())
# Initialize DeepLake vector store with OpenAI embeddings
deep_lake = DeepLake(
dataset_path=DEEPLAKE_PATH,
read_only=True,
embedding_function=EMBEDDINGS,
)
# Initialize retriever and set search parameters
retriever = deep_lake.as_retriever()
retriever.search_kwargs.update({
'distance_metric': 'cos',
'fetch_k': 100,
'maximal_marginal_relevance': True,
'k': 10,
})
# Initialize ChatOpenAI model
model = ChatOpenAI(model_name=language_model, temperature=0.2) # gpt-3.5-turbo by default. Use gpt-4 for better and more accurate responses
# Initialize ConversationalRetrievalChain
qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)
# Initialize chat history
chat_history = []
def get_user_input():
"""Get user input and handle 'quit' command."""
question = input("\nPlease enter your question (or 'quit' to stop): ")
if question.lower() == 'quit':
return None
return question
def print_answer(question, answer):
"""Format and print question and answer."""
print(f"\nQuestion: {question}\nAnswer: {answer}\n")
def main():
"""Main program loop."""
while True:
question = get_user_input()
if question is None: # User has quit
break
# Display token usage and approximate costs
with get_openai_callback() as tokens_usage:
result = qa({"question": question, "chat_history": chat_history})
chat_history.append((question, result['answer']))
print_answer(question, result['answer'])
print(tokens_usage)
if __name__ == "__main__":
main()
```
## Conclusion
And there you have it! You've built a powerful question-answering system using LangChain, OpenAI, and DeepLake. This system can take a user's question, search a vector database for relevant context, and generate a detailed answer. It's like having your very own AI-powered oracle! In the next articles I'll show you some other use cases and applications!