Tarea: back · Meta
Llama 3.1 8B Instruct
chatLlamaLlama 3.1 Community License
parámetros
8,030,000,000
Context
131,072
Muestras
4
Benchmarks
RealMediana (tok/s)
75.3
Promedio (tok/s)
78.8
P25
28
P75
126.1
Explorador de benchmarks
| Cuantización | Runtime | GPU | Mediana (tok/s) | Muestras |
|---|---|---|---|---|
| Q4_K_M | llama.cpp | RTX 4090 | 158 | 1 |
| Q4_K_M | Ollama | RX 7800 XT | 38 | 1 |
| Q4_K_M | Ollama | M1 Max (32 GPU cores) | 63 | 1 |
| Q4_K_M | Ollama | RTX 4080 | 118 | 1 |
| Q4_K_M | Ollama | RTX 4070 | 82 | 1 |
| Q4_K_M | llama.cpp | RTX 3060 | 35.2 | 1 |
| Q4_K_M | llama.cpp | RTX 3090 | 115.4 | 1 |
| Q4_K_M | llama.cpp | Ninguno | 6.4 | 1 |