Related articles
Mixture-of-Experts
Architecture used by Mistral Large 4 in which a router selects a small group of experts per token; the model uses four of 128 experts plus one shared expert in most layers.
Related articles
Architecture used by Mistral Large 4 in which a router selects a small group of experts per token; the model uses four of 128 experts plus one shared expert in most layers.