The law of diminishing returns applies to LLMs too

It surprises me how many people insist the law of diminishing returns won't apply to LLMs, and that the opposite ("LLMs will become a real boy!") will somehow happen.

An innovation arrives. An explosion of applications follows (maybe even a bubble 😬). It gets iteratively refined. Then you hit a plateau as the limits of that innovation are reached.

📱 Mobile phone: From "Holy moly, a computer in my pocket!" to "Look, it's 1mm thinner than last year."

🌐 Dot com: From "Anything is possible on the internet" to "Bespoke porn!"

💻 Personal computers: From "Regular folk can write documents and make spreadsheets!" to "Bespoke porn!"

🛞 The wheel: From "It's so much easier to move things!" to "Nice rims."

I regret to inform you that the curve is now flattening. We've already passed the disruption phase. You can tell model improvements are slowing because we've reached the predictable last resort: throwing more hardware at it.

So now we get refinements to the tooling around next-token prediction: RAG, orchestration, specialized agents. Useful, exciting to learn about and use, but not matching the leaps we saw in the early stages of LLM releases.

Don't get me wrong: AI will advance in mind-blowing ways, but LLMs won't magically evolve to get us there. LLMs are akin to just one segment of our brain. AGI will take brand new innovations, and there's no guaranteed schedule for those.

---------------------

And no, I don't know why the coding robot has a ladle. But I know that's the kind of goofiness that's inherent to how LLMs work.