June 8, 2023
News in the natural language space continues to march on, from fundraising to best practices. There are some really good resources this time around, but if you only click on two links make them the first two. You won’t be disappointed.
Also, if you’re in Paris on June 27, come join me and others at The Jam by Scaleway. We’ll be discussing prompt engineering, and it promises to be a good time.
(As always, if you have a link I should check out, email me at d@dcoates.com.)
Building with LLMs; What Could Be Difficult?
In this post, Honeycomb talks about their experience building a tool to craft queries for their logging tool via natural language.
They ran into many issues, which aligns very well with what I’ve heard from others when we discuss what it means to create a product on top of LLMs. Namely:
Getting enough information in the context window
The slowness of LLMs
“Prompt engineering is weird”
A tension between “correctness” and “usefulness”
Prompt injection
Legal and compliance concerns
I won’t summarize much, because it’s worth reading the whole thing. The big takeaway for me, though, is how much this reinforces that we’re in just the beginning of building products on top of LLMs and with prompt engineering. There’s a need for tools to make us more efficient and more confident.
I’m exploring something in this space, so if you are building a product on top of LLMs and with prompt engineering, please reach out at d@dcoates.com and let’s chat.
More from Andrej Karpathy
Andrej Karpathy is one of the few people in ML whose videos I will watch instantly, and without reservation. Karpathy walks us through the state of GPT in a way that’s easy to follow without a Ph.D. It’s a real skill.
If you like that video, he’s got a whole set of videos going hands-on to the very core of ML, including one on building a GPT.
More Fundraising
Here’s three straight newsletter with a major fundraising announcement. Last time it was Anthropic with a $450 million Series C and Pinecone and Weaviate the newsletter before that.
This time it’s Cohere’s turn, with $270 million raised in a Series C with money raised from industry players like NVIDIA, Oracle, and Salesforce Ventures. This part is interesting as Google has invested heavily in Anthropic and, of course, Microsoft with OpenAI. Camps are forming.
Also from Cohere is this article written by Cohere CEO Aiden Gomez on AI Doomerism. This seems, at least partially, as a rejoinder to Sam Altman of OpenAI’s call for increased government regulation of AI (not that Gomez is completely anti-regulation).
Don’t Do This, Please
If you’re a lawyer, and you want to cite some cases, what do you do? You go to FindLaw, do your research, and bill your clients. Or, you could ask ChatGPT, and take a long lunch.
Except when you cite ChatGPT and discover after the fact that the cases simply do not exist.
The lawyer said, “I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic.”
This won’t be the last painful lesson people learn about hallucinations.
Do This, Please
OpenAI released documentation on how best to work with GPT, and they’re as follows:
Write Clear Instructions
Provide Reference Text
Split Complex Tasks into Smaller Subtasks
Give GPTs Time to “Think”
Use External Tools
Test Changes Systematically
Each go into specific tactics on how to do so, from first running vector searches to enhance the prompt with more information, to asking the model to explain its reasoning.
And if you want to see these kinds of best practices in action, you can see them in the paper A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT.