Dumb 1.1 Rc 2
It is an AI model that balances Dumb-1.1-Rc-1.
Modellquelle
- Art
- Text model
- Dateigröße
- 72,2 MB
- Version
- vmodel
- Aktualisiert
- 3. Okt. 2026
- Verleger
- @56m
- Downloads
- 19
- Parameter
- 56M
- Aufgaben
- Textgenerierung
- Quelle
- Hugging Face
Quellenbeschreibung
Dumb-1.1
It is an AI model that balances Dumb-1.1-Rc-1.
You can tell more detailed lies, and the details of the benchmark are written.
benchmark result
| Benchmark | Metis-1.5 (898M/A340M) | Dumb-1.1 (18.9M) | Ant (10M) |
|---|---|---|---|
| MMLU (acc) | 23.60% | 23.22% | 25.43% |
| HellaSwag (acc_norm) | 30.40% | 25.74% | 26.72% |
| ARC-Easy (acc_norm) | 41.30% | 30.35% | 25.42% |
| PIQA (acc_norm) | 54.70% | 51.74% | 50.72% |
| BoolQ (acc) | 47.70% | 62.17% | 37.82% |
| SciQ (acc_norm) | - | 32.10% | 21.50% |
| ARC-Challenge (acc_norm) | 25.90% | 20.73% | - |
| WinoGrande (acc) | 51.50% | - | 49.64% |
| OpenBookQA (acc_norm) | 29.60% | 25.20% | - |
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 3. Okt.
Modellartefakte
1 Artefakt
Modellartefakte
1 ArtefaktQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- language: - en license: apache-2.0 pipeline_tag: text-generation tags: - text-generation model-index: - name: Dumb-1.1-Rc-2 results: - task: type: text-generation name: Text Generation dataset: name: MMLU type: cais/mmlu config: all split: test metrics: # --- Overall & Categories --- - name: MMLU (Overall) type: accuracy value: 23.22 - name: Humanities (Category) type: accuracy value: 24.31 - name: Social Sciences (Category) type: accuracy value: 21.74 - name: STEM (Category) type: accuracy value: 22.33 - name: Other (Category) type: accuracy value: 23.91 # --- Humanities Subtasks --- - name: Subtask - Formal Logic type: accuracy value: 34.13 - name: Subtask - High School European History type: accuracy value: 21.82 - name: Subtask - High School US History type: accuracy value: 25.00 - name: Subtask - High School World History type: accuracy value: 26.16 - name: Subtask - International Law type: accuracy value: 23.97 - name: Subtask - Jurisprudence type: accuracy value: 25.93 - name: Subtask - Logical Fallacies type: accuracy value: 22.09 - name: Subtask - Moral Disputes type: accuracy value: 24.86 - name: Subtask - Moral Scenarios type: accuracy value: 23.80 - name: Subtask - Philosophy type: accuracy value: 18.65 - name: Subtask - Prehistory type: accuracy value: 21.30 - name: Subtask - Professional Law type: accuracy value: 24.64 - name: Subtask - World Religions type: accuracy value: 32.16 # --- Other Subtasks --- - name: Subtask - Business Ethics type: accuracy value: 30.00 - name: Subtask - Clinical Knowledge type: accuracy value: 20.38 - name: Subtask - College Medicine type: accuracy value: 21.39 - name: Subtask - Global Facts type: accuracy value: 18.00 - name: Subtask - Human Aging type: accuracy value: 31.39 - name: Subtask - Management type: accuracy value: 17.48 - name: Subtask - Marketing type: accuracy value: 29.06 - name: Subtask - Medical Genetics type: accuracy value: 29.00 - name: Subtask - Miscellaneous type: accuracy value: 23.75 - name: Subtask - Nutrition type: accuracy value: 21.57 - name: Subtask - Professional Accounting type: accuracy value: 23.40 - name: Subtask - Professional Medicine type: accuracy value: 19.85 - name: Subtask - Virology type: accuracy value: 28.31 # --- Social Sciences Subtasks --- - name: Subt...
Source context: 19 downloads · 0 likes · Pipeline text-generation · Repo 56m/Dumb-1.1
Source context: 54 downloads · 0 likes · Pipeline text-generation · Repo 56m/Dumb-1.1
