Mimi89757 commited on
Commit
ae9acd3
·
unverified ·
1 Parent(s): a048ef3

cann : add Ascend NPU instructions (#2410)

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -21,6 +21,7 @@ High-performance inference of [OpenAI's Whisper](https://github.com/openai/whisp
21
  - Support for CPU-only inference
22
  - [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
23
  - [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
 
24
  - [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)
25
 
26
  Supported platforms:
@@ -448,6 +449,39 @@ cmake -DWHISPER_MKL=ON ..
448
  WHISPER_MKL=1 make -j
449
  ```
450
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
451
  ## Docker
452
 
453
  ### Prerequisites
 
21
  - Support for CPU-only inference
22
  - [Efficient GPU support for NVIDIA](https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas)
23
  - [OpenVINO Support](https://github.com/ggerganov/whisper.cpp#openvino-support)
24
+ - [Ascend NPU Support](https://github.com/ggerganov/whisper.cpp#ascend-npu-support)
25
  - [C-style API](https://github.com/ggerganov/whisper.cpp/blob/master/include/whisper.h)
26
 
27
  Supported platforms:
 
449
  WHISPER_MKL=1 make -j
450
  ```
451
 
452
+ ## Ascend NPU support
453
+
454
+ Ascend NPU provides inference acceleration via [`CANN`](https://www.hiascend.com/en/software/cann) and AI cores.
455
+
456
+ First, check if your Ascend NPU device is supported:
457
+
458
+ **Verified devices**
459
+ | Ascend NPU | Status |
460
+ |:-----------------------------:|:-------:|
461
+ | Atlas 300T A2 | Support |
462
+
463
+ Then, make sure you have installed [`CANN toolkit`](https://www.hiascend.com/en/software/cann/community) . The lasted version of CANN is recommanded.
464
+
465
+ Now build `whisper.cpp` with CANN support:
466
+
467
+ ```
468
+ mkdir build
469
+ cd build
470
+ cmake .. -D GGML_CANN=on
471
+ make -j
472
+ ```
473
+
474
+ Run the inference examples as usual, for example:
475
+
476
+ ```
477
+ ./build/bin/main -f samples/jfk.wav -m models/ggml-base.en.bin -t 8
478
+ ```
479
+
480
+ *Notes:*
481
+
482
+ - If you have trouble with Ascend NPU device, please create a issue with **[CANN]** prefix/tag.
483
+ - If you run successfully with your Ascend NPU device, please help update the table `Verified devices`.
484
+
485
  ## Docker
486
 
487
  ### Prerequisites