May 11th, 2023
Three years after setting up Talking to Computers: The Email, I’m actually doing something with it. There’s so much that is happening in conversational and natural language technologies, that keeping up is time intensive. Maybe you think the same, too, which is why I’m reviving this newsletter, with a new email every other week on interesting news and links I’ve found on the topic, with commentary.
(Subscribe years ago and forgotten that you did? No longer interested? No problem, you can unsubscribe at the bottom of this email.)
The links will generally focus on the tech and product implications of today and the near future. If you want 36 ways to use ChatGPT to build your personal brand, there are other places where you can find them.
So, on to some handpicked news…
LLMs, &c.
Anthropic Announces 100k Context Windows
This is huge news. It opens up so many new, more intelligent use cases. For example, Anthropic mentions in the linked posts analysis of an 85-page corporate filing or analyzing the entirety of a library’s docs. It’s even enough for 6 hours of a transcribed podcast.
Learn Prompt Engineering
There were a number of new prompt engineering courses to come out recently, including:
This online course, also featuring a prompt hacking competition
It’s interesting to see these, especially as others are saying that prompt engineering is only a temporary skill or prompt engineering is a skill that everyone needs to learn, as much as they did word processors thirty years ago. Any predictions on whether we’ll know more in a year?
We’ve been speaking with a lot of prompt engineering professionals recently, and one thing is clear: there’s a lot of figuring it out going on. One of my favorite anecdotes was someone who said he does so much experimentation, he’s taken to speaking his prompts, because it’s much easier than typing them.
Google: “We Have No Moat, and Neither Does OpenAI”
This leaked document comes from within Google and puts forward the argument that the rate of improvement of language models is improving so quickly that the leader(s) of today won’t be the leaders of tomorrow on raw capabilities alone. Namely: open source is doing things that closed source (e.g., OpenAI and Google) aren’t nimble enough to do.
Some examples from the document author:
Training a model that performs “90%” of ChatGPT for $100 (meanwhile OpenAI losses last year were estimated at $540 million)
The document also notes that open-source primacy is already the case in AI image generation, with Stable Diffusion in the lead there.
I’m not entirely sure I buy the argument, as there are diminishing returns on the intelligence of language models. (But, to be fair, I probably would have said that in 2020 with GPT-3…) There’s also something powerful about being the “default” in people’s minds, especially when you’re a hosted provider of a solution, like OpenAI is.
Conversely, the key takeaway for me is that there’s a mad rush to improve language models and that means that, if you want to be cutting edge, you’ve got a lot to keep up on. Now, being cutting edge isn’t everything, especially if it means spending a ton of your devs’ time switching out one model for another to eek out a 0.01% improvement. But there are still step changes to come, and you need to be flexible.
Also Google: We Have Lots of AI Built-In
Of course, Google didn’t hesitate to announce a lot around AI at Google I/O this week. Here’s a recap, and here’s more on their latest model, PaLM 2.
Vector Search
Fundraising for Weaviate and Pinecone
It’s been a busy month for vector search, with Weaviate and Pinecone announcing fundraising on the same day, with $50 million and $100 million for each Series B, respectively. Both are obviously big players in vector databases and now they each have a large war chest to put towards capturing what is still a pretty nascent market.
Oh, the Things You’ll Do (with Vector Search)
Nascent, but one that will probably grow quickly, especially if vector DB providers can make it simple for people who aren’t ML engineers to come aboard. Vector search can do so many things, like this post from Ashot Vardanian shows: Abusing Vector Search for Texts, Maps, and Chess.
And In Other News…
Other news, if you’ll allow me a bit of self-promotion this far down the newsletter, was the launch of Algolia NeuralSearch. This is what I’ve been working on for the past months, and I’m excited to get it out there. Our early beta results were nothing short of remarkable, and it really proved out that for search, the best approach is a hybrid one of both keyword and vector results for every query.