Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models

by [unverified] · free · Last verified 2026-06-21T03:07:55.172Z

We propose Complete-muE, a framework which targets hyperparameter transfer across dense FFN and any Mixture-of-Experts (MoE) setups in transformer blocks. Existing tools such as $μ$P (requires fixed architectue) or SDE (requires fixed per-step token count) cannot directly solve the hyperparameter...