Update README.md
Browse files
README.md
CHANGED
|
@@ -16,17 +16,17 @@ datasets:
|
|
| 16 |
---
|
| 17 |
|
| 18 |
|
| 19 |
-
# Sumo-
|
| 20 |
|
| 21 |

|
| 22 |
|
| 23 |
-
### Tensorplex Labs Unveils Sumo-
|
| 24 |
|
| 25 |
-
[Tensorplex Labs]((https://tensorplex.ai)) is proud to announce that its latest top-performing model on Bittensor Subnet 9, Sumo-
|
| 26 |
has outperformed notable models such as TII Falcon 7B and Meta's Llama-2-7b-hf. This achievement highlights the potential of decentralized networks
|
| 27 |
like Bittensor and underscores Tensorplex Labs' commitment to advancing open-source AI technologies.
|
| 28 |
|
| 29 |
-
"Sumo" represents the family of models developed by Tensorplex, and "
|
| 30 |
|
| 31 |
Bittensor Subnet 9 serves a unique role within the Bittensor ecosystem by rewarding miners who produce pretrained foundational models on the Falcon Refined Web dataset. This subnet functions as a continuous benchmark, where miners are incentivized to achieve the best performance metrics using a model under the parameter limit. The competitive nature of Subnet 9 drives rapid advancements and refinements in large language model training.
|
| 32 |
|
|
@@ -45,7 +45,7 @@ Since the parameter limit was upgraded to 7 billion on April 19, 2024, Tensorple
|
|
| 45 |
- **Training Objective**: Causal Language Modeling (next token prediction)
|
| 46 |
- **Original Model Repo**: [tensorplex-labs/pretraining-sn9-7B-1](https://huggingface.co/tensorplex-labs/pretraining-sn9-7B-1)
|
| 47 |
|
| 48 |
-
Sumo-
|
| 49 |
|
| 50 |
⛔ **This is a pretrained base model, which hasn't been aligned yet. Use with caution or finetune further on downstream tasks before deployment.**
|
| 51 |
|
|
@@ -63,7 +63,7 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
| 63 |
import transformers
|
| 64 |
import torch
|
| 65 |
|
| 66 |
-
model = "tensorplex-labs/Sumo-
|
| 67 |
|
| 68 |
tokenizer = AutoTokenizer.from_pretrained(model)
|
| 69 |
pipeline = transformers.pipeline(
|
|
@@ -95,7 +95,7 @@ This model has been trained with [tiiuae/falcon-refinedweb](https://huggingface.
|
|
| 95 |
|
| 96 |
## Evaluation
|
| 97 |
|
| 98 |
-
Sumo-
|
| 99 |
establishing itself as the leading model in aggregate across various evaluation tasks.
|
| 100 |
Such benchmarks include ARC Challenge, GSM8K, HellaSwag, MMLU, TruthfulQA, and Winogrande.
|
| 101 |
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
|
| 19 |
+
# Sumo-T9-7B-v0.1
|
| 20 |
|
| 21 |

|
| 22 |
|
| 23 |
+
### Tensorplex Labs Unveils Sumo-T9-7B: Beating Notable 7b Pretrained Models
|
| 24 |
|
| 25 |
+
[Tensorplex Labs]((https://tensorplex.ai)) is proud to announce that its latest top-performing model on Bittensor Subnet 9, Sumo-T9-7B,
|
| 26 |
has outperformed notable models such as TII Falcon 7B and Meta's Llama-2-7b-hf. This achievement highlights the potential of decentralized networks
|
| 27 |
like Bittensor and underscores Tensorplex Labs' commitment to advancing open-source AI technologies.
|
| 28 |
|
| 29 |
+
"Sumo" represents the family of models developed by Tensorplex, and "T9" designates the top-performing model specifically trained for Bittensor Subnet 9.
|
| 30 |
|
| 31 |
Bittensor Subnet 9 serves a unique role within the Bittensor ecosystem by rewarding miners who produce pretrained foundational models on the Falcon Refined Web dataset. This subnet functions as a continuous benchmark, where miners are incentivized to achieve the best performance metrics using a model under the parameter limit. The competitive nature of Subnet 9 drives rapid advancements and refinements in large language model training.
|
| 32 |
|
|
|
|
| 45 |
- **Training Objective**: Causal Language Modeling (next token prediction)
|
| 46 |
- **Original Model Repo**: [tensorplex-labs/pretraining-sn9-7B-1](https://huggingface.co/tensorplex-labs/pretraining-sn9-7B-1)
|
| 47 |
|
| 48 |
+
Sumo-T9-7B-v0.1 features a larger vocabulary size (100k), compatible with the GPT-4 tokenizer, ensuring its versatility across various natural language processing tasks.
|
| 49 |
|
| 50 |
⛔ **This is a pretrained base model, which hasn't been aligned yet. Use with caution or finetune further on downstream tasks before deployment.**
|
| 51 |
|
|
|
|
| 63 |
import transformers
|
| 64 |
import torch
|
| 65 |
|
| 66 |
+
model = "tensorplex-labs/Sumo-T9-7B-v0.1"
|
| 67 |
|
| 68 |
tokenizer = AutoTokenizer.from_pretrained(model)
|
| 69 |
pipeline = transformers.pipeline(
|
|
|
|
| 95 |
|
| 96 |
## Evaluation
|
| 97 |
|
| 98 |
+
Sumo-T9-7B-v0.1 has outperformed notable models such as TII Falcon 7B, Meta's Llama-2-7b and Llama-1-7b in zero-shot performance,
|
| 99 |
establishing itself as the leading model in aggregate across various evaluation tasks.
|
| 100 |
Such benchmarks include ARC Challenge, GSM8K, HellaSwag, MMLU, TruthfulQA, and Winogrande.
|
| 101 |
|