Instructions to use lianghsun/gemma-3-tw-270m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/gemma-3-tw-270m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/gemma-3-tw-270m")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/gemma-3-tw-270m")
model = AutoModelForCausalLM.from_pretrained("lianghsun/gemma-3-tw-270m")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lianghsun/gemma-3-tw-270m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/gemma-3-tw-270m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lianghsun/gemma-3-tw-270m

SGLang

How to use lianghsun/gemma-3-tw-270m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/gemma-3-tw-270m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/gemma-3-tw-270m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/gemma-3-tw-270m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lianghsun/gemma-3-tw-270m with Docker Model Runner:
```
docker model run hf.co/lianghsun/gemma-3-tw-270m
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for gemma-3-tw-270m

gemma-3-tw-270m 是以 google/gemma-3-270m 為基底，針對繁體中文（zh-tw）與台灣語境進行持續預訓練（continued pretraining, CPT）之輕量級基底模型，作為 gemma-3-tw-270m-it、keyboard-warrior 等下游模型的繁中底座。

⚠️ 規格重點：本模型為 270M 參數小型語言模型（SLM）、純文本單模態、僅做 CPT、未做指令微調，需自行 SFT 後才有對話能力。

Model Details

Google 在 2025 年釋出 Gemma-3 270M，提供端側可部署的小規模權重，但其原生繁中能力有限、且對台灣語境（用詞、文化、本地常識）覆蓋不足。gemma-3-tw-270m 透過持續預訓練，把繁中網頁、教育、生活、政府公開文本等語料注入模型參數，讓它在保留 Gemma-3 270M 基底架構的前提下，先一步具備可用的繁中底層能力，作為下游 SFT／DPO 的起點。

核心特點 (Key Features)

小型化、可端側部署：270M 參數，可在筆電 CPU 與邊緣裝置運行；亦能透過 ONNX／transformers.js 部署到瀏覽器。
繁中底座：以繁體中文與台灣語境語料做 CPT，大幅提升原版 Gemma-3 270M 的繁中流暢度。
下游可微調：作為 instruct、reasoning、persona、領域應用等微調工程的起點。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: google/gemma-3-270m
Model type: Gemma3ForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: gemma (Google usage license)

Model Sources

Repository: lianghsun/gemma-3-tw-270m

Citation

@misc{gemma_3_tw_270m,
  title        = {gemma-3-tw-270m: A Traditional Chinese Continued-Pretrained Gemma-3 270M Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/lianghsun/gemma-3-tw-270m}}
}