Update README.md
Browse files
README.md
CHANGED
|
@@ -1,55 +1,45 @@
|
|
| 1 |
-
---
|
| 2 |
-
library_name: transformers
|
| 3 |
-
tags: []
|
| 4 |
-
---
|
| 5 |
-
|
| 6 |
-
|
| 7 |
<p align="center">
|
| 8 |
<img src="https://github.com/MLP-Lab/KORMo-tutorial/blob/main/tutorial/attachment/kormo_logo.png?raw=true" style="width: 100%; max-width: 1100px;">
|
| 9 |
</p>
|
| 10 |
|
| 11 |
-
|
| 12 |
-
---
|
| 13 |
# π¦Ύ KORMo-10B
|
| 14 |
|
| 15 |
-
**KORMo-10B
|
| 16 |
-
|
| 17 |
|
| 18 |
-
- π§
|
| 19 |
-
- π£οΈ
|
| 20 |
-
- πͺ
|
| 21 |
-
- π§ͺ
|
| 22 |
|
| 23 |
---
|
| 24 |
|
| 25 |
## π Links
|
| 26 |
|
| 27 |
-
- π€ **Hugging Face**:
|
| 28 |
-
- π» **GitHub Repository**: [π
|
| 29 |
|
| 30 |
---
|
| 31 |
|
| 32 |
-
## π
|
| 33 |
-
- π **2025
|
| 34 |
|
| 35 |
---
|
| 36 |
|
| 37 |
-
##
|
| 38 |
-
|
|
| 39 |
-
|
| 40 |
| Architecture | Transformer Decoder |
|
| 41 |
| Parameters | 10.8B |
|
| 42 |
| Context Length | 128K |
|
| 43 |
| Languages | Korean, English |
|
| 44 |
| License | Apache 2.0 |
|
| 45 |
|
| 46 |
-
|
| 47 |
---
|
| 48 |
|
| 49 |
-
## π
|
| 50 |
-
|
| 51 |
-
### π μ λ νκ° (Quantitative Evaluation)
|
| 52 |
|
|
|
|
| 53 |
|
| 54 |
| Benchmark | **KORMo-10B** | smolLM3-3B | olmo2-7B | olmo2-13B | kanana1.5-8B | qwen3-8B | llama3.1-8B | gemma3-4B | gemma3-12B |
|
| 55 |
|:-----------|---------------:|-----------:|---------:|---------:|------------:|--------:|-----------:|---------:|----------:|
|
|
@@ -81,7 +71,9 @@ tags: []
|
|
| 81 |
| kr_clinical_qa | 77.32 | 53.97 | 48.33 | 46.22 | 65.84 | 80.00 | 63.54 | 60.00 | 77.22 |
|
| 82 |
| **Korean Avg.** | **58.15** | 47.37 | 35.82 | 39.34 | 60.94 | 63.35 | 49.60 | 49.60 | 60.37 |
|
| 83 |
|
| 84 |
-
|
|
|
|
|
|
|
| 85 |
|
| 86 |
| Benchmark | KORMo-10B | smolLM3-3B | olmo2-7B | olmo2-13B | kanana1.5-8B | qwen3-8B | llama3.1-8B | exaone3.5-8B* | gemma3-12B |
|
| 87 |
|:----------|---------:|----------:|---------:|---------:|------------:|--------:|------------:|-------------:|-----------:|
|
|
@@ -93,6 +85,6 @@ tags: []
|
|
| 93 |
---
|
| 94 |
|
| 95 |
## Contact
|
| 96 |
-
|
| 97 |
|
| 98 |
-
## Contributor
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
<p align="center">
|
| 2 |
<img src="https://github.com/MLP-Lab/KORMo-tutorial/blob/main/tutorial/attachment/kormo_logo.png?raw=true" style="width: 100%; max-width: 1100px;">
|
| 3 |
</p>
|
| 4 |
|
|
|
|
|
|
|
| 5 |
# π¦Ύ KORMo-10B
|
| 6 |
|
| 7 |
+
**KORMo-10B** is a **10.8B parameter fully open LLM** capable of handling both **Korean and English**.
|
| 8 |
+
The model, training code, and training data are all **fully open**, allowing anyone to reproduce and extend it.
|
| 9 |
|
| 10 |
+
- π§ **Model Size**: 10.8B parameters
|
| 11 |
+
- π£οΈ **Languages**: Korean / English
|
| 12 |
+
- πͺ **Training Data**: Synthetic data + public datasets
|
| 13 |
+
- π§ͺ **License**: Apache 2.0 (commercial use permitted)
|
| 14 |
|
| 15 |
---
|
| 16 |
|
| 17 |
## π Links
|
| 18 |
|
| 19 |
+
- π€ **Hugging Face**: [π Model Download](https://huggingface.co/KORMo-Team)
|
| 20 |
+
- π» **GitHub Repository**: [π Training and Inference Code](https://github.com/MLP-Lab/KORMo-tutorial)
|
| 21 |
|
| 22 |
---
|
| 23 |
|
| 24 |
+
## π Update News
|
| 25 |
+
- π **Oct 2025**: Official release of KORMo v1.0!
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
+
## Model Architecture
|
| 30 |
+
| Item | Description |
|
| 31 |
+
|:----|:------------|
|
| 32 |
| Architecture | Transformer Decoder |
|
| 33 |
| Parameters | 10.8B |
|
| 34 |
| Context Length | 128K |
|
| 35 |
| Languages | Korean, English |
|
| 36 |
| License | Apache 2.0 |
|
| 37 |
|
|
|
|
| 38 |
---
|
| 39 |
|
| 40 |
+
## π Benchmark Performance
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
### π Quantitative Evaluation
|
| 43 |
|
| 44 |
| Benchmark | **KORMo-10B** | smolLM3-3B | olmo2-7B | olmo2-13B | kanana1.5-8B | qwen3-8B | llama3.1-8B | gemma3-4B | gemma3-12B |
|
| 45 |
|:-----------|---------------:|-----------:|---------:|---------:|------------:|--------:|-----------:|---------:|----------:|
|
|
|
|
| 71 |
| kr_clinical_qa | 77.32 | 53.97 | 48.33 | 46.22 | 65.84 | 80.00 | 63.54 | 60.00 | 77.22 |
|
| 72 |
| **Korean Avg.** | **58.15** | 47.37 | 35.82 | 39.34 | 60.94 | 63.35 | 49.60 | 49.60 | 60.37 |
|
| 73 |
|
| 74 |
+
---
|
| 75 |
+
|
| 76 |
+
## π Qualitative Evaluation (LLM-as-a-Judge)
|
| 77 |
|
| 78 |
| Benchmark | KORMo-10B | smolLM3-3B | olmo2-7B | olmo2-13B | kanana1.5-8B | qwen3-8B | llama3.1-8B | exaone3.5-8B* | gemma3-12B |
|
| 79 |
|:----------|---------:|----------:|---------:|---------:|------------:|--------:|------------:|-------------:|-----------:|
|
|
|
|
| 85 |
---
|
| 86 |
|
| 87 |
## Contact
|
| 88 |
+
- KyungTae Lim, Professor at Seoultech. `[email protected]`
|
| 89 |
|
| 90 |
+
## Contributor
|