Baidu and Zhipu AI’s large language models top Chinese generative AI rankings, but OpenAI, Anthropic remain ahead in overall performance
- Baidu’s Ernie Bot 4.0 and start-up Zhipu AI’s GLM-4 are the best-performing AI models in China, an assessment by Tsinghua University shows
- While still lagging overseas rivals in most capabilities, China’s LLMs do better in Chinese text-language tasks
Baidu’s Ernie Bot 4.0 and start-up Zhipu AI’s GLM-4 rank top among Chinese large language models (LLMs), but their foreign rivals still lead in overall capabilities, according to a new test by Tsinghua University in Beijing.
The SuperBench assessment report examined 14 representative LLMs – the technology underpinning generative artificial intelligence (AI) chatbots – and found that overseas models, such as OpenAI’s GPT-4 and Anthropic’s Claude-3, came out on top in multiple capabilities, including semantic comprehension, coding abilities and alignment with human commands.
Researchers found “obvious gaps” in the code-writing and operative abilities in the real-world environment between domestic and first-class foreign models.
The report aims to “provide objective and scientific evaluation criteria” to examine a growing number of LLMs that have emerged recently, according to a WeChat post published by Tsinghua’s Basic Model Research Centre, which conducted the assessment with the state-backed Zhongguancun Laboratory.
05:03
How does China’s AI stack up against ChatGPT?
Chinese tech giants and start-ups have been racing to improve their LLMs since OpenAI, a US start-up backed by Microsoft, launched a series of innovative tools powered by generative AI, including ChatGPT and text-to-video service Sora.
Around 200 LLMs have been introduced in China, where OpenAI’s services are officially unavailable, according to government figures.