Command R7B: Complete Benchmark Performance Guide

cohere r7b 2024-08-13 CC-BY-NC-4.0

In-depth analysis of Command R7B performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendorcohere
Versionr7b
Release Date2024-08-13
Context Window128000 tokens
Input Modalitiestext
Output Modalitiestext
LicenseCC-BY-NC-4.0
Documentationhttps://docs.cohere.com/docs

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU64.9%2024-08-135-shotview
HUMANEVAL50.1pass@12024-08-13—view
GSM8K69.7%2024-08-130-shot CoTview
MATH40.6%2024-08-130-shot CoTview
BBH66.1%2024-08-133-shot CoTview
GPQA30.9%2024-08-130-shotview
IFEVAL67.2%2024-08-13prompt_strictview
ARC90.7%2024-08-13challengeview
MUSR47%2024-08-130-shotview
WINOGRANDE74.1%2024-08-130-shotview

Pricing

TierPriceCurrency
Input$0.0375 / MtokUSD
Output$0.0375 / MtokUSD
Cache Read$0 / MtokUSD
Cache Write$0 / MtokUSD

Source: https://cohere.com/pricing · as of 2024-08-13

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✗
  • GDPR: ✓
  • ISO 27001: ✓

Command R7B

Model Genel Bakışı

Cohere Command R7B 轻量开源模型, 128K 上下文, 7B 参数, 为 RAG 与工具调用优化, 适合边缘部署。

Temel Özellikler

SatıcıSürümYayın TarihiBağlam PenceresiGiriş ModlarıÇıkış ModlarıLisans
Coherer7b2024-08-13128KtexttextCC-BY-NC-4.0

Benchmark Performansı

BenchmarkPuanBirimNotlar
MMLU (Massive Multitask Language Understanding)64.9%5-shot
HumanEval50.1pass@1—
GSM8K (Grade School Math 8K)69.7%0-shot CoT
MATH40.6%0-shot CoT
BBH (BIG-Bench Hard)66.1%3-shot CoT
GPQA30.9%0-shot
IFEval67.2%prompt_strict
ARC90.7%challenge
MUSR47.0%0-shot
WinoGrande74.1%0-shot

Fiyatlandırma

GirişÇıkışÖnbellek OkumaÖnbellek Yazma
————

milyon token başına

Güçlü Yönler

  • 可靠的通用模型。

Zayıf Yönler

  • 闭源专有模型，不支持自托管。

Kullanım Senaryoları

  • 通用对话与问答

Referanslar