Instructions to use Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0")
model = AutoModelForCausalLM.from_pretrained("Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0

SGLang

How to use Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0 with Docker Model Runner:
```
docker model run hf.co/Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Model Card for LION-LLaMA-3-8b-dpo-v1.0

The LION-series are trained using an empirically optimized pipeline that consists of three stages: SFT, DPO, and online preference learning (online DPO). We find simple techniques such as sequence packing, loss masking in SFT, increasing the preference dataset size in DPO, and online DPO training can significantly improve the performance of language models. Our best models (the LION-series) exceed the performance of the official instruct models tuned with closed-source data and algorithms.

For training datasets, code, and evaluation scripts, please refer to our paper and codebase.

Model description

This model is finetuned from Columbia-NLP/LION-LLaMA-3-8b-sft-v1.0 using DPO from the LION pipeline.

Model type: meta-llama/Meta-Llama-3-8B
Language(s) (NLP): Primarily English
License: LLaMa-3 Terms of Use
Finetuned from model: Columbia-NLP/LION-LLaMA-3-8b-sft-v1.0

Performance

Model	Method	Size	Arena-Hard	AlpacaEval-2	MT-Bench	OpenLLM
LLaMA-3-8b	-	8B	-	-	-	63.05
LLaMA-3-8b-it	SFT+RS+DPO+PPO	8B	20.6	22.9	8.00	68.28
LION-LLaMA-3-8b-sft-v1.0 (ours)	SFT	8B	11.3	17.9	7.58	68.71
⮕ LION-LLaMA-3-8b-dpo-v1.0 (ours)	SFT+DPO	8B	19.1	21.8	8.12	71.28
LION-LLaMA-3-8b-odpo-v1.0 (ours)	SFT+DPO+ODPO	8B	22.0	26.8	8.19	71.41

Intended uses

To ensure reproducibility, please use the following chat templates:

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0",
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
messages = [
    # no system message for LLaMa
    {
        "role": "user", 
        "content": "Write a short paragraph where every sentence starts with the letter A."
    },
]
outputs = pipe(
    messages,
    max_new_tokens=128,
    do_sample=False,
    stop_sequence="<|im_end|>",
)
print(outputs[0]["generated_text"][-1]["content"])
# Astonishingly, all animals adore appealing, aromatic apples.
# An array of apples always attracts adventurous ants.
# After analyzing, ants ascertain the apple's accessibility.
# Anticipation amplifies as they approach, anticipating an appetizing treat.

to inspect the chat template/manually do generation:

tokenizer = AutoTokenizer.from_pretrained("Columbia-NLP/LION-LLaMA-3-8b-dpo-v1.0")
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(prompt)
# tokenize prompt and use model.generate

Training details

Please refer to our paper and codebase.

Citation Information

If you find this model useful in your work, please consider citing our paper:

@misc{yu2024lionsempiricallyoptimizedapproach,
      title={LIONs: An Empirically Optimized Approach to Align Language Models}, 
      author={Xiao Yu and Qingyang Wu and Yu Li and Zhou Yu},
      year={2024},
      eprint={2407.06542},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.06542}, 
}