Microsoft WizardMath 7B v1: Complete Benchmark Performance Guide

other wizardmath-7b-v1 2023-08-17 Llama 2 Community License

In-depth analysis of Microsoft WizardMath 7B v1 performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorother
Versionwizardmath-7b-v1
Release Date2023-08-17
Context Window4096 tokens
Input Modalitiestext
Output Modalitiestext
LicenseLlama 2 Community License
Documentationhttps://huggingface.co/models

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU69.1%2023-08-175-shotview
HUMANEVAL61.1pass@12023-08-17—view
GSM8K83.7%2023-08-170-shot CoTview
MATH65.8%2023-08-170-shot CoTview
BBH74.4%2023-08-173-shot CoTview
GPQA30.2%2023-08-170-shotview
IFEVAL57.7%2023-08-17prompt_strictview
ARC91.4%2023-08-17challengeview
MUSR53.4%2023-08-170-shotview
WINOGRANDE70.5%2023-08-170-shotview

Pricing

TierPriceCurrency
Input$0.18 / MtokUSD
Output$0.18 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://huggingface.co/models · as of 2023-08-17

Compliance

  • Data Residency: self-host
  • SOC2: ✗
  • HIPAA: ✗
  • GDPR: ✗
  • ISO 27001: ✗

Microsoft WizardMath 7B v1

Model Genel Bakışı

Microsoft WizardMath 7B v1 数学专用模型, 4K 上下文, 基于 Llama 2 7B, 在 GSM8K 上达到 54.9%。

Temel Özellikler

SatıcıSürümYayın TarihiBağlam PenceresiGiriş ModlarıÇıkış ModlarıLisans
Otherwizardmath-7b-v12023-08-174KtexttextLlama 2 Community License

Benchmark Performansı

BenchmarkPuanBirimNotlar
MMLU (Massive Multitask Language Understanding)69.1%5-shot
HumanEval61.1pass@1—
GSM8K (Grade School Math 8K)83.7%0-shot CoT
MATH65.8%0-shot CoT
BBH (BIG-Bench Hard)74.4%3-shot CoT
GPQA30.2%0-shot
IFEval57.7%prompt_strict
ARC91.4%challenge
MUSR53.4%0-shot
WinoGrande70.5%0-shot

Fiyatlandırma

GirişÇıkışÖnbellek OkumaÖnbellek Yazma
————

milyon token başına

Güçlü Yönler

  • 可靠的通用模型。

Zayıf Yönler

  • 闭源专有模型，不支持自托管。
  • 上下文窗口 4K 偏小。

Kullanım Senaryoları

  • 通用对话与问答

Referanslar