Curated Resource ( ? )
Create your Document ChatBot with GPT-3 and Langchain
Curated:
13/04/2023 from
medium.com/@jeremyarancio/create-your-document-chatbot-with-gpt-3-and-langchain-8eeb66b98656
my notes ( ? )
"build a chatbot capable of extracting any kind of information from a set of documents ... your own Document Assistant from scratch, using GPT-3 and Langchain"
Good explainer of using embeddings to get around token limits "using LangChain, which is an open-source library designed to simplify the utilization of LLMs with Chain of Thoughts". Detailed instructions are provided for the following steps:
- "Documents content is extracted using the proper Python library ...
- The text is divided into chunks... Each chunk is embedded into a vector, the same for the user’s question;
- The most similar chunks to the question are picked;
- The question and extracted text chunks are fed as input prompts into GPT-3 (fine-tuning with a custom prompt);
- An answer is provided."
Because the prompt specifies that the question should only be based on the provided chunks, hallucinations are avoided. For the same reason the temperature is set to zero, making GPT choose "the next token with the highest probability based on the previous ones... reduces the likelihood of less plausible tokens being generated."
Read the Full Post
The above notes were curated from the full post medium.com/@jeremyarancio/create-your-document-chatbot-with-gpt-3-and-langchain-8eeb66b98656.Related reading
More Stuff I Like
More Stuff tagged ai , guide , gpt-x , llm , langchain , embedding
See also: Digital Transformation , Innovation Strategy , Science&Technology , Large language models