Qwen Blog company blog

Qwen1.5-Moe: Matching 7B Model Performance with 1/3 Activated Parameters

by Qwen Team qwenlm.github.io published 2024-03-286 min read

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction Since the surge in interest sparked by Mixtral, research on mixture-of-expert (MoE) models has gained significant momentum. Both researchers and practitioners are keenly interested in understanding how to effectively train such models and assessing their…

Points

104

HN comments

10

Submissions

1

Cited by

0 commenters

AI & Machine Learning2024

Related posts from the same era

Bing: “I will not harm you unless you harm me first”

Simon Willison20233,363 points1,673 commentscited by 6 details

GPT-4o

OpenAI Blog20243,138 points2,366 commentscited by 16 details

Open source AI is the path forward

Meta Newsroom20242,360 points887 commentscited by 5 details

My AI skeptic friends are all nuts

Fly.io Blog20252,356 points2,826 commentscited by 13 details

Claude 3.7 Sonnet and Claude Code

Anthropic20252,127 points963 commentscited by 6 details

Claude 4

Anthropic20252,013 points1,170 comments details