Tag See all →

#nvidia

1 stories

Banner: Ollama container running on CPU because the GPU is not passed into Docker

DevOps & CI/CD Bug Report

Ollama in Docker runs on CPU only: GPU not passed to the container

Ryan Pham · Feb 19, 2025 · 5 min read

Our self-hosted LLM box produced three tokens per second on a server with an RTX 4090. ollama ps reported 100% CPU: the container had no access to the GPU. Installing the NVIDIA Container Toolkit and requesting the GPU in Compose brought it to 120 tokens per second.