Mistral 7B

Decoding Mistral AI's Large Language Models: Devendra Chaplot

Nov 21, 2024 · 18:16

Devendra Singh Chaplot of Mistral AI details the company's open-source large language models, including Mistral 7B, Mixtral 8x7B, Mixtral 8x22B, and CodeStral 22B, arguing that open models complement rather than compete with profit by serving as branding tools and driving customer acquisition for proprietary upgrades. He explains the three-stage LLM training process—pre-training on trillions of tokens, instruction tuning with prompt-response pairs, and learning from human feedback via preference optimization—emphasizing that more data does not guarantee better performance due to noise. The episode highlights Mistral's focus on optimizing the performance-to-cost ratio, with CodeStral 22B outperforming larger models like Code LLaMA 70B while being smaller and multilingual across 80+ programming languages. Practical guidance is given: prototype with high-end commercial models, then fine-tune open models for specific tasks to balance performance and cost.

No more bad outputs with structured generation: Remi Louf

Oct 14, 2024 · 15:32

Rémi Louf, CEO of .txt and co-maintainer of Outlines, argues that structured generation—guiding LLMs to produce valid regex, JSON, or context-free grammar outputs—eliminates parsing errors and hallucinations while adding negligible overhead. Outlines masks tokens that violate the target structure, enabling Mistral 7B to achieve 99.9% valid JSON (vs. 17% without). It also accelerates inference: a chatty 50-token ChatGPT answer shrinks to 8 tokens, and structured generation boosts open models beyond GPT-4—Phi-3 Medium hits 96.5% on Berkeley function calling (GPT-4 gets 93.5%). With one shot matching eight shots in accuracy, Louf contends that most text is structured and that structured generation should be the default for non-chatbot workflows.