China's Qwen model tops 'Open LLM Leaderboard v2,' which tests AI models for Hugging Face
Hugging Face has released version 2 of the Open LLM Leaderboard, which ranks open source language models from around the world. At the time of release, the top ranking was
Open-LLM performances are plateauing, let's make the leaderboard steep again - a Hugging Face Space by open-llm-leaderboard
https://huggingface.co/spaces/open-llm-leaderboard/blog
Chinese AI models storm Hugging Face's LLM chatbot benchmark leaderboard — Alibaba runs the board as major US competitors have worsened | Tom's Hardware
https://www.tomshardware.com/tech-industry/artificial-intelligence/chinese-llms-storm-hugging-faces-chatbot-benchmark-leaderboard-alibaba-runs-the-board-as-major-us-competitors-have-worsened
To rank them, each language model was evaluated on four tasks: intelligence, short- and long-term context reasoning, complex mathematical ability, and how well it followed human instructions.
The evaluation used six benchmarks: the multiple-choice benchmark ' MMLU-Pro ,' the ' GPQA ' which measures highly specialized knowledge, the ' MuSR ' which includes questions such as solving murder mysteries, the ' MATH ' mathematical aptitude test, the ' IFEval ' which tests the ability to follow instructions, and the ' BBH ' which measures whether the answers are interesting to humans.
With over 7,500 models rated, the winner was the Qwen2-72B-Instruct. Hugging Face said, 'The Qwen2-72B-Instruct stands out from the crowd.' In fact, it was the only model to reach an average rating in the 40s.
The results can be found at the link below.
Open LLM Leaderboard 2 - a Hugging Face Space by open-llm-leaderboard
https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
This ranking is subject to change. At the time of writing, the breakdown from 1st to 10th place is as follows:
1st place: Qwen/ Qwen2-72B-Instruct
2nd place: meta-llama/ Meta-Llama-3-70B-Instruct
3rd place: Qwen/ Qwen2-72B
4th place: mistralai/ Mixtral-8x22B-Instruct-v0.1
5th place: HuggingFaceH4/ zephyr-orpo-141b-A35b-v0.1
6th place: microsoft/ Phi-3-medium-4k-instruct
7th place: 01-ai/ Yi-1.5-34B-Chat
8th place:CohereForAI/ c4ai-command-r-plus
9th place: abacusai/ Smaug-72B-v0.1
10th place: Qwen/ Qwen1.5-110B
As shown above, Qwen's models occupy three of the top 10 positions, demonstrating their overwhelming strength. Furthermore, 'Smaug-72B,' which ranked 9th this time, was the top model in Open LLM Leaderboard Version 1 as of February 2024. Smaug-72B is a model created by fine-tuning 'Qwen-72B,' which ranked 3rd this time.
Abacus AI's open-source LLM 'Smaug-72B' topped the Hugging Face Open LLM Leaderboard and outperformed GPT-3.5 in several benchmarks - GIGAZINE
Related Posts:
in AI, Posted by log1p_kr