MoE Analysis Qwen[3.5|3.6]35B-A3B

MoE Analysis Qwen[3.5|3.6]35B-A3B gslaller.github.io

MT-Bench prompts answered by Qwen3.5 and Qwen3.6 (35B-A3B), with every MoE routing decision captured: which of the 256 experts fire for each token in each of the 40 layers, and how sure the router is.

2 points•gslaller•2 days ago•1 comment•

#moe #analysis #35b #a3b #qwen #35b a3b #moe analysis #analysis qwen

1 comment

gslaller2 days ago

Did some work on visualising the firing pattern of MoE (Mixture of Experts) modules for Qwen models and other statistics. You'll see temporal consistency in the router, as one would expect.

Read the full thread on Hacker News →

1 comment

Related stories