Instructions to use oneonlee/LDCC-SOLAR-gugutypus-10.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use oneonlee/LDCC-SOLAR-gugutypus-10.7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="oneonlee/LDCC-SOLAR-gugutypus-10.7B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("oneonlee/LDCC-SOLAR-gugutypus-10.7B")
model = AutoModelForCausalLM.from_pretrained("oneonlee/LDCC-SOLAR-gugutypus-10.7B")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use oneonlee/LDCC-SOLAR-gugutypus-10.7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "oneonlee/LDCC-SOLAR-gugutypus-10.7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oneonlee/LDCC-SOLAR-gugutypus-10.7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/oneonlee/LDCC-SOLAR-gugutypus-10.7B

SGLang

How to use oneonlee/LDCC-SOLAR-gugutypus-10.7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "oneonlee/LDCC-SOLAR-gugutypus-10.7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oneonlee/LDCC-SOLAR-gugutypus-10.7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "oneonlee/LDCC-SOLAR-gugutypus-10.7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "oneonlee/LDCC-SOLAR-gugutypus-10.7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use oneonlee/LDCC-SOLAR-gugutypus-10.7B with Docker Model Runner:
```
docker model run hf.co/oneonlee/LDCC-SOLAR-gugutypus-10.7B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

LDCC-SOLAR-gugutypus-10.7B

Model Details

Model Developers

DongGeon Lee (oneonlee)

Model Architecture

LDCC-SOLAR-gugutypus-10.7B is a instruction fine-tuned auto-regressive language model, based on the SOLAR transformer architecture.

Base Model

LDCC/LDCC-SOLAR-10.7B

Training Dataset

kyujinpy/KOR-gugugu-platypus-set

Model comparisons

Ko-LLM leaderboard (2024/03/01) [link]

Model	Average	Ko-ARC	Ko-HellaSwag	Ko-MMLU	Ko-TruthfulQA	Ko-CommonGen V2
oneonlee/KoSOLAR-v0.2-gugutypus-10.7B	51.17	47.78	58.29	47.27	48.31	54.19
oneonlee/LDCC-SOLAR-gugutypus-10.7B	49.45	45.9	55.46	47.96	48.93	49

(KOR) AI-Harness evaluation [link]

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
KMMLU	N/A	none	0	acc	0.3329	±	0.0794
KMMLU	N/A	none	5	acc	0.3969	±	0.0816
KoBEST-HellaSwag	0	none	0	acc	0.4260	±	0.0221
KoBEST-HellaSwag	0	none	5	acc	0.4260	±	0.0221
KoBEST-BoolQ	0	none	0	acc	0.7792	±	0.0111
KoBEST-BoolQ	0	none	5	acc	0.8925	±	0.0083
KoBEST-COPA	0	none	0	acc	0.6670	±	0.0149
KoBEST-COPA	0	none	5	acc	0.7070	±	0.0144
KoBEST-SentiNeg	0	none	0	acc	0.7582	±	0.0215
KoBEST-SentiNeg	0	none	5	acc	0.9219	±	0.0135

(ENG) AI-Harness evaluation [link]

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
MMLU	N/A	none	0	acc	0.5826	±	0.1432
MMLU	N/A	none	5	acc	0.6124	±	0.1275
HellaSwag	1	none	0	acc	0.6075	±	0.0049
HellaSwag	1	none	5	acc	0.6534	±	0.0047
BoolQ	2	none	0	acc	0.8737	±	0.0058
BoolQ	2	none	5	acc	0.8878	±	0.0055
COPA	1	none	0	acc	0.8300	±	0.0378
COPA	1	none	5	acc	0.9300	±	0.0256
truthfulqa	N/A	none	0	acc	0.4249	±	0.0023
truthfulqa	N/A	none	5	acc	-	±	-

Implementation Code

### LDCC-SOLAR-gugutypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "oneonlee/LDCC-SOLAR-gugutypus-10.7B"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained(repo)

Downloads last month: 9

Safetensors

Model size

11B params

Tensor type

F32

Model tree for oneonlee/LDCC-SOLAR-gugutypus-10.7B

Base model

LDCC/LDCC-SOLAR-10.7B

Finetuned

(9)

this model

Quantizations

3 models

oneonlee
/

LDCC-SOLAR-gugutypus-10.7B

LDCC-SOLAR-gugutypus-10.7B

Model Details

Model comparisons

Implementation Code

Model tree for oneonlee/LDCC-SOLAR-gugutypus-10.7B

Dataset used to train oneonlee/LDCC-SOLAR-gugutypus-10.7B