LMArena (formerly Chatbot Arena) is a public, web-based platform that evaluates large language models (LLMs) through anonymous, crowd-sourced pairwise comparisons. Users enter prompts for two anonymous models to respond to and vote on the model that gave the better response, in which the model's identities are revealed. Users can also choose models to test themselves.
Chatbot Arena is popular within the artificial intelligence industry, with major companies supplying their large language models, such as GPT-4o, o1, Gemini, and Claude, and using their subsequent rankings to promote them. Notably, Chinese company DeepSeek tested its prototype models in the Chatbot Arena months before its R1 model gained attention in Western media. The website has even been used for preview releases of upcoming models. However, Chatbot Arena's methodology for measuring large language model performance has been questioned as insufficient.