Buradasın
Llama-cpp-python GPU Support Solutions
stackoverflow.com/questions/76963311/llama-cpp-python-not-using-nvidia-gpu-cudaYapay zekadan makale özeti
- Initial Problem
- User couldn't get GPU support for llama-cpp-python on Ubuntu 20.04
- Model worked only on CPU with NVIDIA GTX 1060 6GB
- Issue persisted despite oobabooga text-generation-webui working properly
- Solution
- Exporting libllama.so shared library before running Python interpreter resolved the problem
- CUDA toolkit installation was necessary for GPU support
- CUDA_HOME access was required after toolkit installation
- Additional Steps
- CUDA toolkit must be downloaded from NVIDIA developer website
- Compile llama-cpp-python with appropriate environment variables
- CUDA architecture can be specified during compilation
- BLAS=1 indicates GPU usage after model initialization