The AI summer — Benedict Evans

Good reality check from Benedict Evans: "LLMs might also be a trap: they look like products and they look magic, but they aren’t. Maybe we have to go through the slow, boring hunt for product-market fit after all."

Most technical developments take time to take hold, but not ChatGPT, which "got to 100m users in just 2 months", partly because - unlike cloud computing or mobile e-commerce - everything required for this rapid takeup was already in place.

But while "a huge number of people went off to try it ... most of them haven’t been back... why hasn’t there been much growth in the active users (as opposed to the vaguely curious)?".

By itself, an LLM "is not a product - it’s a technology that can enable a tool or a feature, and it needs to be unbundled or rebundled into new framings, UX and tools to be become useful"

In enterprise settings, "Everyone has a bunch of tests, but far fewer people are trusting something in their business to this". It does depend on the application: "LLMs are already very useful for coding and marketing", but less useful elsewhere, and while there are a lot of pilots, "the single biggest business from this in 2024 might be for consultants explaining what it is... 30% of big company CIOs don’t expect to deploy anything before 2026".

Speed trap

Maybe the speed of (light) adoption is a trap in and of itself: LLMs "look like they work, and they look generalised, and they look like a product ... But the magic might not be useful, in that form, and it might be wrong. It looks like product, but it isn’t."

And then, of course, there's the enormous pressure to catch the wave, no matter the cost: the "sense that this is the next platform shift and you have to grab it ... [LLMs] will change everything, right now, and they need all this money, and we have all this money.”

So we skipped the search for product-market fit and "went straight to ‘it’s for everything!’ before meeting an actual user".

Evans hedges his bets in his conclusion: "the AI maximalists might be right... This might be the first S-Curve in tech history that turns out to be a J-Curve. But not this year."