I Built an AI Chatbot Based On My Favorite Podcast

This piece captures and then explores exactly what I've always thought about the future direction of myhub.ai: imagine what would happen if you had your own personal AI assistant operating across your content - your public posts, private library (including content shared from friends, stuff in your reading queue, etc.) and the wider web, emphasising the sources you favour (based firstly on your Priority Sources, and then on the number of times you've curated their content).

He sets out the stakes from line one: "In the future, any time you look up information ... you’re not going to dig through your second brain. Instead, you’re going to ask a chatbot that sits on top of all of your notes, and the chatbot will return the right answer to you... I just built a demo over the weekend... Huberman Lab chatbot using GPT-3 ... can already answer questions plausibly well... [although] still gets things subtly wrong ... sometimes not specific enough to answer the question, and I have to ask follow-ups".

How? "When a user asks a question, it searches" a library he created of all transcripts from the Huberman Lab podcasts, finds relevant sections and sends them to GPT-3 with a carefully designed prompt - "something like 'Answer the question as truthfully as possible using the provided context, and if the answer is not contained within the text below, say "I don't know." // [ relevant sections of Huberman Lab transcripts ] // Q: [user question]' ... It took probably a weekend of effort."

Of course there are problems, but this is literally within the first month, and is constrained by OpenAI's API. A few developments would improve it:

• giving ChatGPT better data: podcast transcripts are not perfect input. This means the notes I write in my hub should be written with an eye to feeding the chatbot
• giving ChatGPT access to the internet
• scripts which "chain GPT-3 prompts together in order to check and refine the answer" allowing GPT-3 to correct itself
• citations: "If every time it answered a question it told me its source... it wouldn't matter as much if the answer was vague or slightly wrong because I could check its work"

Implications

Content monetisation

"there’s a new way to monetize any existing set of intellectual property... anything that’s used as a reference should become a chat bot... [if you can0t code] buy the rights to turn this information into chatbots and sell them later to a bidder who can."

Organisational knowledge management

The job of company librarians will evolve into chatbot training... until the bots can manage and update the knowledge themselves: "a chatbot that answers questions by sourcing information from the right person* or document, makes sure documents are up to date, and proactively records tacit knowledge into living documents by periodically interviewing key people".

My note: (*) This will need finetuning. Imagine a working environment where getting info was as easy as asking a bot, but where that bot then asked the right people for you using chat. We'd all be submerged with requests from bots.

"Where does power sit in an ecosystem like this?"

"who’s going to win? ... I think power will settle in at least four places..."

• operating system & browser layers both "sit between the user and any other interaction with the internet" and can access your offline data, so you'll be asking Windows or OS rather than visiting websites. On the other hand these models will be serving millions of users so they'll probably need "self-imposed limits about what kinds of tasks ... and results they’re willing to [perform for and] return to the user"
• This will leave opportunities for other players "willing to return answers ... that are riskier (legally, morally, or in terms of brand alignment)"
• copyright holders, particularly large ones "will fight back in the same way the record industry did against Napster.