Mistral Large 2: Complete Benchmark Performance Guide

mistral large-2 2024-07-24 Mistral Research License

In-depth analysis of Mistral Large 2 performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendormistral
Versionlarge-2
Release Date2024-07-24
Context Window128000 tokens
Input Modalitiestext
Output Modalitiestext
LicenseMistral Research License
Documentationhttps://docs.mistral.ai/

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU84%2024-07-245-shotview
HUMANEVAL92pass@12024-07-24—view
GSM8K93%2024-07-240-shot CoTview
MATH71%2024-07-240-shot CoTview
BBH81%2024-07-243-shot CoTview

Pricing

TierPriceCurrency
Input$2 / MtokUSD
Output$6 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://mistral.ai/technology/ · as of 2024-08-01

Compliance

  • Data Residency: EU
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Mistral Large 2

Model Genel Bakışı

Mistral Large 2 是 Mistral AI 旗舰模型，123B 参数，128K 上下文窗口，在代码生成、数学推理与多语言任务上表现强劲，支持欧盟数据驻留与 GDPR/ISO 27001 合规。

Temel Özellikler

SatıcıSürümYayın TarihiBağlam PenceresiGiriş ModlarıÇıkış ModlarıLisans
Mistrallarge-22024-07-24128KtexttextMistral Research License

Benchmark Performansı

BenchmarkPuanBirimNotlar
MMLU (Massive Multitask Language Understanding)84.0%5-shot
HumanEval92.0pass@1—
GSM8K (Grade School Math 8K)93.0%0-shot CoT
MATH71.0%0-shot CoT
BBH (BIG-Bench Hard)81.0%3-shot CoT

Fiyatlandırma

GirişÇıkışÖnbellek OkumaÖnbellek Yazma
————

milyon token başına

Güçlü Yönler

  • MMLU score 84.0, strong knowledge reasoning.
  • HumanEval 92.0, excellent code generation.
  • GSM8K 93.0, robust math reasoning.

Zayıf Yönler

  • 闭源专有模型，不支持自托管。

Kullanım Senaryoları

  • 代码生成与调试
  • Agent 工作流与工具调用

Referanslar