Hugging Face's logo

RichardErkhov
/
Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf

GGUF
conversational
Model card Files
xet
 Community
How to use from
llama.cpp
Install from brew 
brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
# Run inference directly in the terminal:
llama-cli -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
# Run inference directly in the terminal:
llama-cli -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
# Run inference directly in the terminal:
./llama-cli -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
# Run inference directly in the terminal:
./build/bin/llama-cli -hf RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
Use Docker 
docker model run hf.co/RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf:
Quick Links

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Gemma-2-9B-IT-With-Deeper-Safety-Alignment - GGUF

Name Quant method Size
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q2_K.gguf Q2_K 3.54GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.IQ3_XS.gguf IQ3_XS 3.86GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.IQ3_S.gguf IQ3_S 4.04GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q3_K_S.gguf Q3_K_S 4.04GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.IQ3_M.gguf IQ3_M 4.19GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q3_K.gguf Q3_K 4.43GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q3_K_M.gguf Q3_K_M 4.43GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q3_K_L.gguf Q3_K_L 4.78GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.IQ4_XS.gguf IQ4_XS 4.86GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q4_0.gguf Q4_0 5.07GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.IQ4_NL.gguf IQ4_NL 5.1GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q4_K_S.gguf Q4_K_S 5.1GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q4_K.gguf Q4_K 5.37GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q4_K_M.gguf Q4_K_M 5.37GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q4_1.gguf Q4_1 5.55GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q5_0.gguf Q5_0 6.04GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q5_K_S.gguf Q5_K_S 6.04GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q5_K.gguf Q5_K 6.19GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q5_K_M.gguf Q5_K_M 6.19GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q5_1.gguf Q5_1 6.52GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q6_K.gguf Q6_K 7.07GB
Gemma-2-9B-IT-With-Deeper-Safety-Alignment.Q8_0.gguf Q8_0 9.15GB

Original model description:

license: gemma

A Gemma-2-9B-IT checkpoint implemented with deeper safety alignment, using the data augmentation approach proposed in the paper Safety Alignment Should Be Made More Than Just a Few Tokens Deep

Citation:

@article{qi2024safety,
  title={Safety Alignment Should Be Made More Than Just a Few Tokens Deep},
  author={Qi, Xiangyu and Panda, Ashwinee and Lyu, Kaifeng and Ma, Xiao and Roy, Subhrajit and Beirami, Ahmad and Mittal, Prateek and Henderson, Peter},
  journal={arXiv preprint arXiv:2406.05946},
  year={2024}
}
Downloads last month
791
GGUF
Model size
9B params
Architecture
gemma2
Hardware compatibility
Log In to add your hardware

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for RichardErkhov/Unispac_-_Gemma-2-9B-IT-With-Deeper-Safety-Alignment-gguf