Mixture of experts
A model architecture that routes each input through selected parameter groups rather than activating the whole model.
What Mixture of experts means
The advertised total parameter count therefore overstates the compute used per request.
Terms this definition uses
Related terms in Models and generation
How a model produces text, what its training and decoding controls change, and what none of them can prove about a source.
Autoregressive model · Transformer · Attention · Self-attention · Parameter · Inference · Pretraining · Fine-tuning · Supervised fine-tuning · Instruction tuning · RLHF · Preference optimization · Knowledge distillation · Quantization · Multimodal model · Vision-language model · Small language model · Tokenizer · Temperature · Top-p sampling · Deterministic decoding · System prompt · Developer message · Tool call · Function calling · Structured output · Constrained decoding · Chain of thought · Reasoning trace · Context caching · Context compaction · Model routing · Model drift · Model version
Questions about Mixture of experts
What is Mixture of experts?
A model architecture that routes each input through selected parameter groups rather than activating the whole model.
What does Mixture of experts not show or guarantee?
A model architecture that routes each input through selected parameter groups rather than activating the whole model.
Which area of the glossary does Mixture of experts belong to?
Models and generation: How a model produces text, what its training and decoding controls change, and what none of them can prove about a source.
← Quantization · Multimodal model →
See it in the full glossary · 731 terms across 19 areas.