Hugging Face's logo

avtc
/
MiniMax-M2-GPTQMODEL-W4A16

Text Generation
Safetensors
vllm
minimax
gptq
4-bit precision
quantization
Mixture of Experts
w4a16
conversational
custom_code
Model card Files
xet
 Community
avtc commited on
Commit
76fd683
·
verified ·
1 Parent(s): 6256384

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -6
README.md CHANGED
@@ -32,12 +32,7 @@ This repository contains a **4-bit quantized version** of the MiniMax-M2 model.
32
  The quantization was performed using **[GPTQModel](https://github.com/ModelCloud/GPTQModel)** with an experimental modification that **feeds the whole dataset to each expert** to achieve improved quality.
33
 
34
  **Calibration Dataset:**
35
- The dataset used during quantization consists of 1536 samples:
36
- * c4/en (1024)
37
- * arc (168)
38
- * gsm8k (168)
39
- * humaneval (168)
40
- * alpaca (20)
41
 
42
  **Hardware & Performance:**
43
  This model is verified to run with Tensor Parallel (TP) on **8x NVIDIA RTX 3090** GPUs with a context window of **192,500 tokens**.
 
32
  The quantization was performed using **[GPTQModel](https://github.com/ModelCloud/GPTQModel)** with an experimental modification that **feeds the whole dataset to each expert** to achieve improved quality.
33
 
34
  **Calibration Dataset:**
35
+ The dataset used during quantization consists of 1536 samples: c4/en (1024), arc (164), gsm8k (164), humaneval (164), alpaca (20)
 
 
 
 
 
36
 
37
  **Hardware & Performance:**
38
  This model is verified to run with Tensor Parallel (TP) on **8x NVIDIA RTX 3090** GPUs with a context window of **192,500 tokens**.