Installation Video and Testing - Step by Step
#2
by
fahdmirzac
- opened
Hi,
Kudos on producing such a sublime model. I did a local installation and testing video :
https://youtu.be/RTDjb0V69bM?si=VFDtua6caEkMnnJp
Thanks and regards,
Fahd
Thank you for this great video @fahdmirzac ! Just one comment that might explain the low speed at the end: the model loaded in FP32 in Transformers massively degrades speed (for every model). You should get a much higher throughput with llama.cpp. Cheers!