Update README.md
Browse files
README.md
CHANGED
|
@@ -177,11 +177,6 @@ print(f"Peak Memory Usage: {mem:.02f} GB")
|
|
| 177 |
|
| 178 |
# Model Performance
|
| 179 |
|
| 180 |
-
Need to install vllm nightly to get some recent changes
|
| 181 |
-
```
|
| 182 |
-
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
| 183 |
-
```
|
| 184 |
-
|
| 185 |
## Results (H100 machine)
|
| 186 |
| Benchmark | | |
|
| 187 |
|----------------------------------|----------------|--------------------------|
|
|
@@ -199,6 +194,11 @@ Download sharegpt dataset: `wget https://huggingface.co/datasets/anon8231489123/
|
|
| 199 |
Other datasets can be found in: https://github.com/vllm-project/vllm/tree/main/benchmarks
|
| 200 |
## benchmark_latency
|
| 201 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
Run the following under `vllm` source code root folder:
|
| 203 |
|
| 204 |
### baseline
|
|
|
|
| 177 |
|
| 178 |
# Model Performance
|
| 179 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 180 |
## Results (H100 machine)
|
| 181 |
| Benchmark | | |
|
| 182 |
|----------------------------------|----------------|--------------------------|
|
|
|
|
| 194 |
Other datasets can be found in: https://github.com/vllm-project/vllm/tree/main/benchmarks
|
| 195 |
## benchmark_latency
|
| 196 |
|
| 197 |
+
Need to install vllm nightly to get some recent changes
|
| 198 |
+
```
|
| 199 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
| 200 |
+
```
|
| 201 |
+
|
| 202 |
Run the following under `vllm` source code root folder:
|
| 203 |
|
| 204 |
### baseline
|