Models
Mistral Small 3: Complete Benchmark Performance Guide
mistral · small-3
Released: 2025-01-29
MMLU 76.9 %
Input $0.2 / Output $0.6 · Mtok
DeepSeek V3: Complete Benchmark Performance Guide
deepseek · v3
Released: 2024-12-26
MMLU 88.5 %
Input $0.27 / Output $1.1 · Mtok
Gemini 2.0 Flash Thinking: Complete Benchmark Performance Guide
google · 2.0-flash-thinking
Released: 2024-12-19
MMLU 86.5 %
Input $0.1 / Output $0.4 · Mtok
Sonar Reasoning: Complete Benchmark Performance Guide
other · sonar-reasoning
Released: 2024-12-18
MMLU 87.7 %
Input $2 / Output $8 · Mtok
o1: Complete Benchmark Performance Guide
openai · o1
Released: 2024-12-17
MMLU 86.3 %
Input $15 / Output $60 · Mtok
Phi-4: Complete Benchmark Performance Guide
other · phi-4
Released: 2024-12-12
MMLU 69.2 %
Input $0.15 / Output $0.15 · Mtok
Gemini 2.0 Flash: Complete Benchmark Performance Guide
google · 2.0-flash
Released: 2024-12-11
MMLU 86.3 %
Input $0.1 / Output $0.4 · Mtok
Llama 3.3 70B: Complete Benchmark Performance Guide
meta · 3.3-70b
Released: 2024-12-06
MMLU 83.4 %
Input $0.9 / Output $0.9 · Mtok
OLMo 2 1124 7B: Complete Benchmark Performance Guide
other · olmo-2-1124-7b
Released: 2024-11-24
MMLU 68.3 %
Input $0.18 / Output $0.18 · Mtok
Claude 3.5 Haiku: Complete Benchmark Performance Guide
anthropic · 3.5-haiku
Released: 2024-11-04
MMLU 77.6 %
Input $0.8 / Output $4 · Mtok
Claude 3.5 Sonnet (2024-10-22): Complete Benchmark Performance Guide
anthropic · 3.5-sonnet-20241022
Released: 2024-10-22
MMLU 87.2 %
Input $3 / Output $15 · Mtok
Llama 3.1 Nemotron 70B: Complete Benchmark Performance Guide
meta · 3.1-nemotron-70b
Released: 2024-10-17
MMLU 81.5 %
Input $0.9 / Output $0.9 · Mtok
NVIDIA Llama 3.1 Nemotron 70B: Complete Benchmark Performance Guide
other · nvidia-llama-3-1-nemotron-70b
Released: 2024-10-17
MMLU 80.4 %
Input $0.9 / Output $0.9 · Mtok
Ministral 3B: Complete Benchmark Performance Guide
mistral · ministral-3b
Released: 2024-10-16
MMLU 42.4 %
Input $0.04 / Output $0.04 · Mtok
Ministral 8B: Complete Benchmark Performance Guide
mistral · ministral-8b
Released: 2024-10-16
MMLU 64.9 %
Input $0.1 / Output $0.1 · Mtok
Gemini 1.5 Flash-8B 002: Complete Benchmark Performance Guide
google · 1.5-flash-8b-002
Released: 2024-10-03
MMLU 77.5 %
Input $0.0375 / Output $0.15 · Mtok
Gemini 1.5 Flash-8B: Complete Benchmark Performance Guide
google · 1.5-flash-8b
Released: 2024-10-03
MMLU 76.6 %
Input $0.0375 / Output $0.15 · Mtok
Llama 3.2 11B Vision: Complete Benchmark Performance Guide
meta · 3.2-11b-vision
Released: 2024-09-25
MMLU 61.9 %
Input $0.55 / Output $0.55 · Mtok
Llama 3.2 1B: Complete Benchmark Performance Guide
meta · 3.2-1b
Released: 2024-09-25
MMLU 48.6 %
Input $0.1 / Output $0.1 · Mtok
Llama 3.2 3B: Complete Benchmark Performance Guide
meta · 3.2-3b
Released: 2024-09-25
MMLU 52.5 %
Input $0.15 / Output $0.15 · Mtok
Llama 3.2 90B Vision: Complete Benchmark Performance Guide
meta · 3.2-90b-vision
Released: 2024-09-25
MMLU 76.5 %
Input $1.2 / Output $1.2 · Mtok
Gemini 1.5 Flash 002: Complete Benchmark Performance Guide
google · 1.5-flash-002
Released: 2024-09-24
MMLU 81.1 %
Input $0.075 / Output $0.3 · Mtok
Gemini 1.5 Pro 002: Complete Benchmark Performance Guide
google · 1.5-pro-002
Released: 2024-09-24
MMLU 87 %
Input $1.25 / Output $5 · Mtok
Qwen2.5 0.5B: Complete Benchmark Performance Guide
alibaba · 2.5-0-5b
Released: 2024-09-19
MMLU 41.5 %
Input $0.02 / Output $0.02 · Mtok
Qwen2.5 1.5B: Complete Benchmark Performance Guide
alibaba · 2.5-1-5b
Released: 2024-09-19
MMLU 54.5 %
Input $0.05 / Output $0.05 · Mtok
Qwen2.5 14B: Complete Benchmark Performance Guide
alibaba · 2.5-14b
Released: 2024-09-19
MMLU 75.4 %
Input $0.25 / Output $0.35 · Mtok
Qwen2.5 32B: Complete Benchmark Performance Guide
alibaba · 2.5-32b
Released: 2024-09-19
MMLU 78.5 %
Input $0.35 / Output $0.45 · Mtok
Qwen2.5 3B: Complete Benchmark Performance Guide
alibaba · 2.5-3b
Released: 2024-09-19
MMLU 56.8 %
Input $0.1 / Output $0.1 · Mtok
Qwen2.5 72B: Complete Benchmark Performance Guide
alibaba · 2.5-72b
Released: 2024-09-19
MMLU 86.1 %
Input $0.5 / Output $0.8 · Mtok
Qwen2.5 7B: Complete Benchmark Performance Guide
alibaba · 2.5-7b
Released: 2024-09-19
MMLU 73.6 %
Input $0.15 / Output $0.15 · Mtok
o1 mini: Complete Benchmark Performance Guide
openai · o1-mini
Released: 2024-09-12
MMLU 78.2 %
Input $3 / Output $12 · Mtok
o1 Preview: Complete Benchmark Performance Guide
openai · o1-preview
Released: 2024-09-12
MMLU 85.3 %
Input $15 / Output $60 · Mtok
OLMo 1.7 7B: Complete Benchmark Performance Guide
other · olmo-1-7-7b
Released: 2024-08-23
MMLU 65.5 %
Input $0.18 / Output $0.18 · Mtok
Phi-3.5 Mini: Complete Benchmark Performance Guide
other · phi-3-5-mini
Released: 2024-08-16
MMLU 55.7 %
Input $0.1 / Output $0.1 · Mtok
Phi-3.5 MoE: Complete Benchmark Performance Guide
other · phi-3-5-moe
Released: 2024-08-16
MMLU 75.7 %
Input $0.2 / Output $0.2 · Mtok
Command R (08-2024): Complete Benchmark Performance Guide
cohere · r08-2024
Released: 2024-08-13
MMLU 77.8 %
Input $0.15 / Output $0.6 · Mtok
Command R+ (08-2024): Complete Benchmark Performance Guide
cohere · r-plus-08-2024
Released: 2024-08-13
MMLU 82.3 %
Input $2.5 / Output $10 · Mtok
Command R7B: Complete Benchmark Performance Guide
cohere · r7b
Released: 2024-08-13
MMLU 64.9 %
Input $0.0375 / Output $0.0375 · Mtok
Grok-2 Mini: Complete Benchmark Performance Guide
xai · 2-mini
Released: 2024-08-13
MMLU 80.6 %
Input $0.1 / Output $0.4 · Mtok
Grok-2 Vision: Complete Benchmark Performance Guide
xai · 2-vision
Released: 2024-08-13
MMLU 83.4 %
Input $2 / Output $10 · Mtok
Grok-2: Complete Benchmark Performance Guide
xai · 2
Released: 2024-08-13
MMLU 87.5 %
Input $2 / Output $10 · Mtok
Sonar Huge: Complete Benchmark Performance Guide
other · sonar-huge
Released: 2024-08-13
MMLU 81 %
Input $2 / Output $2 · Mtok
GLM-4 Air: Complete Benchmark Performance Guide
other · glm-4-air
Released: 2024-08-12
MMLU 78.9 %
Input $0.1 / Output $0.1 · Mtok
GLM-4 Flash: Complete Benchmark Performance Guide
other · glm-4-flash
Released: 2024-08-12
MMLU 80 %
Input $0 / Output $0 · Mtok
GLM-4 Plus: Complete Benchmark Performance Guide
other · glm-4-plus
Released: 2024-08-12
MMLU 84.3 %
Input $0.5 / Output $0.5 · Mtok
Hermes 3 Llama 3.1 405B: Complete Benchmark Performance Guide
other · hermes-3-llama-3-1-405b
Released: 2024-08-12
MMLU 83.9 %
Input $3 / Output $3 · Mtok
Hermes 3 Llama 3.1 70B: Complete Benchmark Performance Guide
other · hermes-3-llama-3-1-70b
Released: 2024-08-12
MMLU 76.7 %
Input $0.8 / Output $0.8 · Mtok
Hermes 3 Llama 3.1 8B: Complete Benchmark Performance Guide
other · hermes-3-llama-3-1-8b
Released: 2024-08-12
MMLU 73.1 %
Input $0.18 / Output $0.18 · Mtok
Jamba 1.5 Large: Complete Benchmark Performance Guide
other · jamba-1-5-large
Released: 2024-08-07
MMLU 84.3 %
Input $2 / Output $8 · Mtok
Jamba 1.5 Mini: Complete Benchmark Performance Guide
other · jamba-1-5-mini
Released: 2024-08-07
MMLU 77.9 %
Input $0.3 / Output $0.5 · Mtok
Jamba 1.5: Complete Benchmark Performance Guide
other · jamba-1-5
Released: 2024-08-07
MMLU 78.5 %
Input $0.5 / Output $0.7 · Mtok
GPT-4o (2024-08-06): Complete Benchmark Performance Guide
openai · 4o-2024-08-06
Released: 2024-08-06
MMLU 87.6 %
Input $2.5 / Output $10 · Mtok
Mistral Large 2: Complete Benchmark Performance Guide
mistral · large-2
Released: 2024-07-24
MMLU 84 %
Input $2 / Output $6 · Mtok
Yi Large Turbo: Complete Benchmark Performance Guide
other · yi-large-turbo
Released: 2024-07-24
MMLU 84.4 %
Input $2 / Output $2 · Mtok
Llama 3.1 405B: Complete Benchmark Performance Guide
meta · 3.1-405b
Released: 2024-07-23
MMLU 88.6 %
Input $5 / Output $15 · Mtok
Llama 3.1 70B: Complete Benchmark Performance Guide
meta · 3.1-70b
Released: 2024-07-23
MMLU 75.6 %
Input $0.9 / Output $0.9 · Mtok
Llama 3.1 8B: Complete Benchmark Performance Guide
meta · 3.1-8b
Released: 2024-07-23
MMLU 73 %
Input $0.18 / Output $0.18 · Mtok
Llama Guard 3 8B: Complete Benchmark Performance Guide
meta · guard-3-8b
Released: 2024-07-23
MMLU 40.1 %
Input $0.18 / Output $0.18 · Mtok
GPT-4o mini: Complete Benchmark Performance Guide
openai · 4o-mini
Released: 2024-07-18
MMLU 79.7 %
Input $0.15 / Output $0.6 · Mtok
Mistral Nemo: Complete Benchmark Performance Guide
mistral · nemo
Released: 2024-07-18
MMLU 69.2 %
Input $0.18 / Output $0.18 · Mtok
Codestral Mamba: Complete Benchmark Performance Guide
mistral · codestral-mamba
Released: 2024-07-16
MMLU 59.3 %
Input $0.25 / Output $0.25 · Mtok
Mathstral 7B: Complete Benchmark Performance Guide
mistral · mathstral-7b
Released: 2024-07-16
MMLU 69.1 %
Input $0.18 / Output $0.18 · Mtok
StarChat2 15B v0.1: Complete Benchmark Performance Guide
other · starchat2-15b-v0.1
Released: 2024-07-05
MMLU 74.9 %
Input $0.3 / Output $0.3 · Mtok
Gemma 2 27B: Complete Benchmark Performance Guide
google · gemma-2-27b
Released: 2024-06-27
MMLU 77.4 %
Input $0.7 / Output $0.7 · Mtok
Gemma 2 9B: Complete Benchmark Performance Guide
google · gemma-2-9b
Released: 2024-06-27
MMLU 64.3 %
Input $0.3 / Output $0.3 · Mtok
StableLM 3 4B: Complete Benchmark Performance Guide
other · stablelm-3-4b
Released: 2024-06-26
MMLU 50.6 %
Input $0.15 / Output $0.15 · Mtok
DeepSeek Coder V2: Complete Benchmark Performance Guide
deepseek · coder-v2
Released: 2024-06-21
MMLU 72.3 %
Input $0.14 / Output $0.28 · Mtok
Claude 3.5 Sonnet: Complete Benchmark Performance Guide
anthropic · 3.5-sonnet
Released: 2024-06-20
MMLU 88.7 %
Input $3 / Output $15 · Mtok
DeepSeek V2 Chat: Complete Benchmark Performance Guide
deepseek · v2-chat
Released: 2024-06-17
MMLU 76.6 %
Input $0.14 / Output $0.28 · Mtok
Qwen2 1.5B: Complete Benchmark Performance Guide
alibaba · 2-1-5b
Released: 2024-06-07
MMLU 40.5 %
Input $0.05 / Output $0.05 · Mtok
Qwen2 57B: Complete Benchmark Performance Guide
alibaba · 2-57b
Released: 2024-06-07
MMLU 77.7 %
Input $0.4 / Output $0.7 · Mtok
Qwen2 72B: Complete Benchmark Performance Guide
alibaba · 2-72b
Released: 2024-06-07
MMLU 80.2 %
Input $0.5 / Output $0.8 · Mtok
Qwen2 7B: Complete Benchmark Performance Guide
alibaba · 2-7b
Released: 2024-06-07
MMLU 63.4 %
Input $0.15 / Output $0.15 · Mtok
GLM-4 9B Chat: Complete Benchmark Performance Guide
other · glm-4-9b-chat
Released: 2024-06-05
MMLU 70.7 %
Input $0.18 / Output $0.18 · Mtok
GLM-4V 9B: Complete Benchmark Performance Guide
other · glm-4v-9b
Released: 2024-06-05
MMLU 73.4 %
Input $0.18 / Output $0.18 · Mtok
Codestral: Complete Benchmark Performance Guide
mistral · codestral
Released: 2024-05-29
MMLU 75.1 %
Input $0.3 / Output $0.9 · Mtok
Mistral 7B v0.3: Complete Benchmark Performance Guide
mistral · 7b-v0.3
Released: 2024-05-22
MMLU 72.6 %
Input $0.18 / Output $0.18 · Mtok
OpenChat 3.6 8B: Complete Benchmark Performance Guide
other · openchat-3-6-8b
Released: 2024-05-21
MMLU 69.2 %
Input $0.18 / Output $0.18 · Mtok
Phi-3 Medium: Complete Benchmark Performance Guide
other · phi-3-medium
Released: 2024-05-21
MMLU 62.1 %
Input $0.2 / Output $0.2 · Mtok
Phi-3 Small: Complete Benchmark Performance Guide
other · phi-3-small
Released: 2024-05-21
MMLU 71.8 %
Input $0.15 / Output $0.15 · Mtok
Phi-3 Vision: Complete Benchmark Performance Guide
other · phi-3-vision
Released: 2024-05-21
MMLU 56.4 %
Input $0.2 / Output $0.2 · Mtok
Gemini 1.5 Flash: Complete Benchmark Performance Guide
google · 1.5-flash
Released: 2024-05-14
MMLU 76 %
Input $0.075 / Output $0.3 · Mtok
GPT-4o (2024-05-13): Complete Benchmark Performance Guide
openai · 4o-2024-05-13
Released: 2024-05-13
MMLU 89.8 %
Input $5 / Output $15 · Mtok
GPT-4o: Complete Benchmark Performance Guide
openai · 4o
Released: 2024-05-13
MMLU 88.7 %
Input $2.5 / Output $10 · Mtok
Sonar Large: Complete Benchmark Performance Guide
other · sonar-large
Released: 2024-05-13
MMLU 82.5 %
Input $1 / Output $1 · Mtok
Sonar Small: Complete Benchmark Performance Guide
other · sonar-small
Released: 2024-05-13
MMLU 80 %
Input $0.2 / Output $0.2 · Mtok
Yi 1.5 34B: Complete Benchmark Performance Guide
other · yi-1-5-34b
Released: 2024-05-13
MMLU 76.1 %
Input $0.4 / Output $0.4 · Mtok
Yi 1.5 6B: Complete Benchmark Performance Guide
other · yi-1-5-6b
Released: 2024-05-13
MMLU 61.1 %
Input $0.15 / Output $0.15 · Mtok
Yi 1.5 9B: Complete Benchmark Performance Guide
other · yi-1-5-9b
Released: 2024-05-13
MMLU 67.8 %
Input $0.18 / Output $0.18 · Mtok
Yi Large: Complete Benchmark Performance Guide
other · yi-large
Released: 2024-05-13
MMLU 82.8 %
Input $3 / Output $3 · Mtok
Yi Vision: Complete Benchmark Performance Guide
other · yi-vision
Released: 2024-05-13
MMLU 85.5 %
Input $3 / Output $3 · Mtok
DeepSeek V2: Complete Benchmark Performance Guide
deepseek · v2
Released: 2024-05-07
MMLU 78.2 %
Input $0.14 / Output $0.28 · Mtok
Phi-3 Mini: Complete Benchmark Performance Guide
other · phi-3-mini
Released: 2024-04-22
MMLU 43.6 %
Input $0.1 / Output $0.1 · Mtok
Llama 3 70B: Complete Benchmark Performance Guide
meta · 3-70b
Released: 2024-04-18
MMLU 79.5 %
Input $0.9 / Output $0.9 · Mtok
Llama 3 8B: Complete Benchmark Performance Guide
meta · 3-8b
Released: 2024-04-18
MMLU 67.8 %
Input $0.18 / Output $0.18 · Mtok
Llama Guard 2 8B: Complete Benchmark Performance Guide
meta · guard-2-8b
Released: 2024-04-18
MMLU 51 %
Input $0.18 / Output $0.18 · Mtok
Microsoft WizardLM 2 7B: Complete Benchmark Performance Guide
other · wizardlm-2-7b
Released: 2024-04-15
MMLU 65.7 %
Input $0.18 / Output $0.18 · Mtok
Microsoft WizardLM 2 8x22B: Complete Benchmark Performance Guide
other · wizardlm-2-8x22b
Released: 2024-04-15
MMLU 78.9 %
Input $1.2 / Output $1.2 · Mtok
WizardLM Team WizardLM 2 8x22B: Complete Benchmark Performance Guide
other · wizardlmteam-wizardlm-2-8x22b
Released: 2024-04-15
MMLU 79.2 %
Input $1.2 / Output $1.2 · Mtok
Mixtral 8x22B: Complete Benchmark Performance Guide
mistral · 8x22b
Released: 2024-04-10
MMLU 77.8 %
Input $1.2 / Output $1.2 · Mtok
Command R+: Complete Benchmark Performance Guide
cohere · r-plus
Released: 2024-04-04
MMLU 75 %
Input $2.5 / Output $10 · Mtok
Jamba Instruct: Complete Benchmark Performance Guide
other · jamba-instruct
Released: 2024-03-28
MMLU 76.2 %
Input $0.5 / Output $0.7 · Mtok
DBRX Base: Complete Benchmark Performance Guide
other · dbrx-base
Released: 2024-03-27
MMLU 79.9 %
Input $0.75 / Output $0.75 · Mtok
DBRX Instruct: Complete Benchmark Performance Guide
other · dbrx-instruct
Released: 2024-03-27
MMLU 81.3 %
Input $0.75 / Output $0.75 · Mtok
Mistral 7B v0.2: Complete Benchmark Performance Guide
mistral · 7b-v0.2
Released: 2024-03-23
MMLU 64.4 %
Input $0.18 / Output $0.18 · Mtok
Zephyr ORPO 141B Alpha: Complete Benchmark Performance Guide
other · zephyr-orpo-141b-alpha
Released: 2024-03-22
MMLU 79.5 %
Input $1.2 / Output $1.2 · Mtok
Command R: Complete Benchmark Performance Guide
cohere · r
Released: 2024-03-11
MMLU 79.3 %
Input $0.15 / Output $0.6 · Mtok
Claude 3 Haiku (2024-03-07): Complete Benchmark Performance Guide
anthropic · 3-haiku-20240307
Released: 2024-03-07
MMLU 77.1 %
Input $0.25 / Output $1.25 · Mtok
Claude 3 Haiku: Complete Benchmark Performance Guide
anthropic · 3-haiku
Released: 2024-03-07
MMLU 77.9 %
Input $0.25 / Output $1.25 · Mtok
Claude 3 Opus: Complete Benchmark Performance Guide
anthropic · 3-opus
Released: 2024-03-04
MMLU 81.2 %
Input $15 / Output $75 · Mtok
Claude 3 Sonnet: Complete Benchmark Performance Guide
anthropic · 3-sonnet
Released: 2024-03-04
MMLU 81.5 %
Input $3 / Output $15 · Mtok
Claude 3 Opus (2024-02-29): Complete Benchmark Performance Guide
anthropic · 3-opus-20240229
Released: 2024-02-29
MMLU 85.7 %
Input $15 / Output $75 · Mtok
Claude 3 Sonnet (2024-02-29): Complete Benchmark Performance Guide
anthropic · 3-sonnet-20240229
Released: 2024-02-29
MMLU 84 %
Input $3 / Output $15 · Mtok
StarCoder2 15B: Complete Benchmark Performance Guide
other · starcoder2-15b
Released: 2024-02-28
MMLU 60.6 %
Input $0.3 / Output $0.3 · Mtok
StarCoder2 3B: Complete Benchmark Performance Guide
other · starcoder2-3b
Released: 2024-02-28
MMLU 60.6 %
Input $0.12 / Output $0.12 · Mtok
StarCoder2 7B: Complete Benchmark Performance Guide
other · starcoder2-7b
Released: 2024-02-28
MMLU 67.4 %
Input $0.2 / Output $0.2 · Mtok
Mistral Large: Complete Benchmark Performance Guide
mistral · large
Released: 2024-02-26
MMLU 82 %
Input $4 / Output $12 · Mtok
Mistral Small: Complete Benchmark Performance Guide
mistral · small
Released: 2024-02-26
MMLU 81.2 %
Input $0.2 / Output $0.6 · Mtok
Mistral Tiny: Complete Benchmark Performance Guide
mistral · tiny
Released: 2024-02-26
MMLU 54.2 %
Input $0.15 / Output $0.15 · Mtok
Qwen1.5 110B: Complete Benchmark Performance Guide
alibaba · 1.5-110b
Released: 2024-02-25
MMLU 86.1 %
Input $0.7 / Output $1.2 · Mtok
Qwen1.5 14B: Complete Benchmark Performance Guide
alibaba · 1.5-14b
Released: 2024-02-25
MMLU 78.4 %
Input $0.25 / Output $0.35 · Mtok
Qwen1.5 32B: Complete Benchmark Performance Guide
alibaba · 1.5-32b
Released: 2024-02-25
MMLU 81.9 %
Input $0.35 / Output $0.45 · Mtok
Qwen1.5 72B: Complete Benchmark Performance Guide
alibaba · 1.5-72b
Released: 2024-02-25
MMLU 81.3 %
Input $0.5 / Output $0.8 · Mtok
Gemma 2B: Complete Benchmark Performance Guide
google · gemma-2b
Released: 2024-02-21
MMLU 48 %
Input $0.1 / Output $0.1 · Mtok
Gemma 7B: Complete Benchmark Performance Guide
google · gemma-7b
Released: 2024-02-21
MMLU 69.6 %
Input $0.22 / Output $0.22 · Mtok
Nous Hermes 2 Mixtral 8x7B: Complete Benchmark Performance Guide
other · nous-hermes-2-mixtral-8x7b
Released: 2024-02-19
MMLU 81.4 %
Input $0.5 / Output $0.5 · Mtok
Nous Hermes 2 Solar 10.7B: Complete Benchmark Performance Guide
other · nous-hermes-2-solar-10-7b
Released: 2024-02-19
MMLU 63.4 %
Input $0.2 / Output $0.2 · Mtok
Gemini 1.0 Flash: Complete Benchmark Performance Guide
google · 1.0-flash
Released: 2024-02-15
MMLU 78.7 %
Input $0.075 / Output $0.3 · Mtok
Gemini 1.5 Pro: Complete Benchmark Performance Guide
google · 1.5-pro
Released: 2024-02-15
MMLU 85.9 %
Input $1.25 / Output $5 · Mtok
DeepSeek Math 7B: Complete Benchmark Performance Guide
deepseek · math-7b
Released: 2024-02-06
MMLU 69.3 %
Input $0.14 / Output $0.14 · Mtok
OLMo 7B Instruct: Complete Benchmark Performance Guide
other · olmo-7b-instruct
Released: 2024-02-01
MMLU 65.6 %
Input $0.18 / Output $0.18 · Mtok
OLMo 7B SFT: Complete Benchmark Performance Guide
other · olmo-7b-sft
Released: 2024-02-01
MMLU 73.4 %
Input $0.18 / Output $0.18 · Mtok
OLMo 7B: Complete Benchmark Performance Guide
other · olmo-7b
Released: 2024-02-01
MMLU 62.6 %
Input $0.18 / Output $0.18 · Mtok
Code Llama 70B: Complete Benchmark Performance Guide
meta · code-llama-70b
Released: 2024-01-29
MMLU 62.1 %
Input $0.9 / Output $0.9 · Mtok
DeepSeek Coder 33B: Complete Benchmark Performance Guide
deepseek · coder-33b
Released: 2024-01-25
MMLU 65.9 %
Input $0.28 / Output $0.28 · Mtok
DeepSeek Coder 7B: Complete Benchmark Performance Guide
deepseek · coder-7b
Released: 2024-01-25
MMLU 63.6 %
Input $0.14 / Output $0.14 · Mtok
GPT-4 0125 Preview: Complete Benchmark Performance Guide
openai · 4-0125-preview
Released: 2024-01-25
MMLU 82.6 %
Input $10 / Output $30 · Mtok
GPT-4 Vision: Complete Benchmark Performance Guide
openai · 4-vision
Released: 2024-01-25
MMLU 83.9 %
Input $10 / Output $30 · Mtok
Nous Hermes 2 Yi 34B: Complete Benchmark Performance Guide
other · nous-hermes-2-yi-34b
Released: 2024-01-22
MMLU 76.9 %
Input $0.4 / Output $0.4 · Mtok
StableCode 3B: Complete Benchmark Performance Guide
other · stablecode-3b
Released: 2024-01-19
MMLU 77.4 %
Input $0.12 / Output $0.12 · Mtok
StableLM 2 1.6B: Complete Benchmark Performance Guide
other · stablelm-2-1-6b
Released: 2024-01-19
MMLU 52.7 %
Input $0.1 / Output $0.1 · Mtok
StableLM 2 12B: Complete Benchmark Performance Guide
other · stablelm-2-12b
Released: 2024-01-19
MMLU 75.7 %
Input $0.3 / Output $0.3 · Mtok
StableLM Zephyr 3B: Complete Benchmark Performance Guide
other · stablelm-zephyr-3b
Released: 2024-01-19
MMLU 44.3 %
Input $0.12 / Output $0.12 · Mtok
DeepSeek LLM 67B: Complete Benchmark Performance Guide
deepseek · llm-67b
Released: 2024-01-05
MMLU 75.2 %
Input $0.4 / Output $0.4 · Mtok
Command Nightly: Complete Benchmark Performance Guide
cohere · nightly
Released: 2024-01-01
MMLU 85.2 %
Input $2.5 / Output $10 · Mtok
Argilla Notus 7B v1: Complete Benchmark Performance Guide
other · notus-7b-v1
Released: 2023-12-15
MMLU 71.4 %
Input $0.18 / Output $0.18 · Mtok
Mistral Medium: Complete Benchmark Performance Guide
mistral · medium
Released: 2023-12-11
MMLU 82.9 %
Input $2.7 / Output $8.1 · Mtok
Mixtral 8x7B: Complete Benchmark Performance Guide
mistral · 8x7b
Released: 2023-12-11
MMLU 77.2 %
Input $0.7 / Output $0.7 · Mtok
Phi-2: Complete Benchmark Performance Guide
other · phi-2
Released: 2023-12-11
MMLU 57.5 %
Input $0.1 / Output $0.1 · Mtok
OpenChat 3.5 1210: Complete Benchmark Performance Guide
other · openchat-3-5-1210
Released: 2023-12-10
MMLU 70.5 %
Input $0.18 / Output $0.18 · Mtok
Gemini 1.0 Pro: Complete Benchmark Performance Guide
google · 1.0-pro
Released: 2023-12-06
MMLU 84.5 %
Input $0.5 / Output $1.5 · Mtok
Gemini 1.0 Ultra: Complete Benchmark Performance Guide
google · 1.0-ultra
Released: 2023-12-06
MMLU 80.3 %
Input $7 / Output $21 · Mtok
Claude 2.1: Complete Benchmark Performance Guide
anthropic · 2.1
Released: 2023-11-21
MMLU 60.9 %
Input $11 / Output $32 · Mtok
Orca 2 13B: Complete Benchmark Performance Guide
other · orca-2-13b
Released: 2023-11-20
MMLU 78.4 %
Input $0.3 / Output $0.3 · Mtok
Capybara 1.5B: Complete Benchmark Performance Guide
other · capybara-1-5b
Released: 2023-11-15
MMLU 51.7 %
Input $0.08 / Output $0.08 · Mtok
Nous Capybara 34B: Complete Benchmark Performance Guide
other · nous-capybara-34b
Released: 2023-11-15
MMLU 53.2 %
Input $0.4 / Output $0.4 · Mtok
GPT-4 1106 Preview: Complete Benchmark Performance Guide
openai · 4-1106-preview
Released: 2023-11-06
MMLU 81.8 %
Input $10 / Output $30 · Mtok
GPT-4 Turbo: Complete Benchmark Performance Guide
openai · turbo
Released: 2023-11-06
MMLU 84.4 %
Input $10 / Output $30 · Mtok
Yi 34B: Complete Benchmark Performance Guide
other · yi-34b
Released: 2023-11-05
MMLU 55.5 %
Input $0.4 / Output $0.4 · Mtok
Yi 6B: Complete Benchmark Performance Guide
other · yi-6b
Released: 2023-11-05
MMLU 68 %
Input $0.18 / Output $0.18 · Mtok
Grok Beta: Complete Benchmark Performance Guide
xai · beta
Released: 2023-11-04
MMLU 51 %
Input $1 / Output $5 · Mtok
Grok Vision Beta: Complete Benchmark Performance Guide
xai · vision-beta
Released: 2023-11-04
MMLU 63.3 %
Input $1 / Output $5 · Mtok
Embed English v3: Complete Benchmark Performance Guide
cohere · embed-english-v3
Released: 2023-11-01
MMLU 0 %
Input $0.1 / Output $0.1 · Mtok
ChatGLM3 6B: Complete Benchmark Performance Guide
other · chatglm3-6b
Released: 2023-10-27
MMLU 58 %
Input $0.18 / Output $0.18 · Mtok
Zephyr 7B Beta: Complete Benchmark Performance Guide
other · zephyr-7b-beta
Released: 2023-10-25
MMLU 65.1 %
Input $0.18 / Output $0.18 · Mtok
Llemma 7B: Complete Benchmark Performance Guide
other · llemma-7b
Released: 2023-10-17
MMLU 66 %
Input $0.18 / Output $0.18 · Mtok
Zephyr 7B Alpha: Complete Benchmark Performance Guide
other · zephyr-7b-alpha
Released: 2023-10-04
MMLU 52.2 %
Input $0.18 / Output $0.18 · Mtok
Mistral 7B v0.1: Complete Benchmark Performance Guide
mistral · 7b-v0.1
Released: 2023-09-27
MMLU 68.4 %
Input $0.18 / Output $0.18 · Mtok
Nous Capybara 7B: Complete Benchmark Performance Guide
other · nous-capybara-7b
Released: 2023-09-26
MMLU 64.4 %
Input $0.18 / Output $0.18 · Mtok
GPT-4 Vision Preview: Complete Benchmark Performance Guide
openai · 4-vision-preview
Released: 2023-09-25
MMLU 85 %
Input $10 / Output $30 · Mtok
Open-Platypus: Complete Benchmark Performance Guide
other · open-platypus
Released: 2023-09-15
MMLU 68.7 %
Input $0.18 / Output $0.18 · Mtok
Command Light: Complete Benchmark Performance Guide
cohere · light
Released: 2023-09-14
MMLU 52.8 %
Input $0.5 / Output $0.5 · Mtok
Phi-1: Complete Benchmark Performance Guide
other · phi-1
Released: 2023-09-14
MMLU 68.1 %
Input $0.1 / Output $0.1 · Mtok
Phi-1.5: Complete Benchmark Performance Guide
other · phi-1-5
Released: 2023-09-14
MMLU 58.4 %
Input $0.1 / Output $0.1 · Mtok
Baichuan2 13B Chat: Complete Benchmark Performance Guide
other · baichuan2-13b-chat
Released: 2023-09-06
MMLU 58.6 %
Input $0.3 / Output $0.3 · Mtok
Baichuan2 7B Chat: Complete Benchmark Performance Guide
other · baichuan2-7b-chat
Released: 2023-09-06
MMLU 57.9 %
Input $0.18 / Output $0.18 · Mtok
Falcon 180B: Complete Benchmark Performance Guide
other · falcon-180b
Released: 2023-09-06
MMLU 76.7 %
Input $1 / Output $1 · Mtok
Code Llama 13B: Complete Benchmark Performance Guide
meta · code-llama-13b
Released: 2023-08-24
MMLU 65.9 %
Input $0.3 / Output $0.3 · Mtok
Code Llama 34B: Complete Benchmark Performance Guide
meta · code-llama-34b
Released: 2023-08-24
MMLU 74.5 %
Input $0.5 / Output $0.5 · Mtok
Code Llama 7B: Complete Benchmark Performance Guide
meta · code-llama-7b
Released: 2023-08-24
MMLU 58.5 %
Input $0.2 / Output $0.2 · Mtok
TigerBot 70B Chat: Complete Benchmark Performance Guide
other · tigerbot-70b-chat
Released: 2023-08-22
MMLU 71.7 %
Input $0.9 / Output $0.9 · Mtok
Microsoft WizardCoder Python 34B: Complete Benchmark Performance Guide
other · wizardcoder-python-34b
Released: 2023-08-17
MMLU 60 %
Input $0.5 / Output $0.5 · Mtok
Microsoft WizardMath 7B v1: Complete Benchmark Performance Guide
other · wizardmath-7b-v1
Released: 2023-08-17
MMLU 69.1 %
Input $0.18 / Output $0.18 · Mtok
Llama 2 13B: Complete Benchmark Performance Guide
meta · 2-13b
Released: 2023-07-18
MMLU 56.4 %
Input $0.3 / Output $0.3 · Mtok
Llama 2 70B: Complete Benchmark Performance Guide
meta · 2-70b
Released: 2023-07-18
MMLU 53 %
Input $0.9 / Output $0.9 · Mtok
Llama 2 7B: Complete Benchmark Performance Guide
meta · 2-7b
Released: 2023-07-18
MMLU 70.8 %
Input $0.2 / Output $0.2 · Mtok
Claude 2: Complete Benchmark Performance Guide
anthropic · 2
Released: 2023-07-11
MMLU 66.3 %
Input $11 / Output $32 · Mtok
MPT 30B: Complete Benchmark Performance Guide
other · mpt-30b
Released: 2023-06-22
MMLU 55.2 %
Input $0.4 / Output $0.4 · Mtok
GPT-3.5 Turbo 16K: Complete Benchmark Performance Guide
openai · 3.5-turbo-16k
Released: 2023-06-13
MMLU 64.2 %
Input $3 / Output $4 · Mtok
Falcon 40B: Complete Benchmark Performance Guide
other · falcon-40b
Released: 2023-05-25
MMLU 56.7 %
Input $0.4 / Output $0.4 · Mtok
Chat Bison: Complete Benchmark Performance Guide
google · chat-bison
Released: 2023-05-10
MMLU 63.1 %
Input $0.5 / Output $1 · Mtok
Code Bison: Complete Benchmark Performance Guide
google · code-bison
Released: 2023-05-10
MMLU 65 %
Input $0.5 / Output $1 · Mtok
PaLM 2: Complete Benchmark Performance Guide
google · palm-2
Released: 2023-05-10
MMLU 68.4 %
Input $1 / Output $2 · Mtok
Text Bison: Complete Benchmark Performance Guide
google · text-bison
Released: 2023-05-10
MMLU 53.9 %
Input $0.5 / Output $1 · Mtok
MPT 7B: Complete Benchmark Performance Guide
other · mpt-7b
Released: 2023-05-04
MMLU 51.7 %
Input $0.18 / Output $0.18 · Mtok
Claude Instant 1: Complete Benchmark Performance Guide
anthropic · instant-1
Released: 2023-03-14
MMLU 57.1 %
Input $0.8 / Output $2.4 · Mtok
GPT-4 32K: Complete Benchmark Performance Guide
openai · 4-32k
Released: 2023-03-14
MMLU 80.8 %
Input $60 / Output $120 · Mtok
GPT-4: Complete Benchmark Performance Guide
openai · 4
Released: 2023-03-14
MMLU 85.8 %
Input $30 / Output $60 · Mtok
Flan-UL2: Complete Benchmark Performance Guide
other · flan-ul2
Released: 2023-03-03
MMLU 58.7 %
Input $0.7 / Output $0.7 · Mtok
GPT-3.5 Turbo: Complete Benchmark Performance Guide
openai · 3.5-turbo
Released: 2023-03-01
MMLU 61.1 %
Input $0.5 / Output $1.5 · Mtok
Jurassic-2 Mid: Complete Benchmark Performance Guide
other · j2-mid
Released: 2023-01-12
MMLU 55.9 %
Input $1 / Output $1 · Mtok
Jurassic-2 Ultra: Complete Benchmark Performance Guide
other · j2-ultra
Released: 2023-01-12
MMLU 62.5 %
Input $2 / Output $2 · Mtok
Flan-T5 XL: Complete Benchmark Performance Guide
other · flan-t5-xl
Released: 2022-12-07
MMLU 60.4 %
Input $0.3 / Output $0.3 · Mtok
Flan-T5 XXL: Complete Benchmark Performance Guide
other · flan-t5-xxl
Released: 2022-12-07
MMLU 52 %
Input $0.5 / Output $0.5 · Mtok
GPT-3.5: Complete Benchmark Performance Guide
openai · 3.5
Released: 2022-11-30
MMLU 50.3 %
Input $2 / Output $2 · Mtok