#Chatbot Arena, a crowdsourced ranking of LLMs

In the jungle of LLMs, with all their versions and sub-families, how do you tell which models are strongest right now, and worth paying for?

There are standardized benchmarks (MMLU, MATH, HumanEval, etc.) that compare them on specific tasks. But no single model wins everywhere.

For an overall ranking that’s easier to read, have a look at Chatbot Arena (now LMArena).

It’s a “crowdsourced benchmarking” site where anyone can vote by comparing model answers side by side, a bit like audience ratings on Allociné.

The idea is simple: you ask a question, two randomly picked models answer, with their names hidden. You vote blind for the answer you like better.

What comes out is an overall ranking built on the wisdom of crowds, more robust than a few scattered examples or a reading of the official benchmarks.

Bonus: you can chat for free, and even generate images.