1# Setting Up CUDA on Fedora
2
3In this guide we setup [Nvidia CUDA](https://docs.nvidia.com/cuda/) in a toolbox container. This guide is applicable for:
4
5- [Fedora Workstation](https://fedoraproject.org/workstation/)
6- [Atomic Desktops for Fedora](https://fedoraproject.org/atomic-desktops/)
7- [Fedora Spins](https://fedoraproject.org/spins)
8- [Other Distributions](https://containertoolbx.org/distros/), including `Red Hat Enterprise Linux >= 8.5`, `Arch Linux`, and `Ubuntu`.
9
10## Table of Contents
11
12- [Prerequisites](#prerequisites)
13- [Using the Fedora 41 CUDA Repository](#using-the-fedora-41-cuda-repository)
14- [Creating a Fedora Toolbox Environment](#creating-a-fedora-toolbox-environment)
15- [Installing Essential Development Tools](#installing-essential-development-tools)
16- [Adding the CUDA Repository](#adding-the-cuda-repository)
17- [Installing Nvidia Driver Libraries](#installing-nvidia-driver-libraries)
18- [Installing the CUDA Meta-Package](#installing-the-cuda-meta-package)
19- [Configuring the Environment](#configuring-the-environment)
20- [Verifying the Installation](#verifying-the-installation)
21- [Conclusion](#conclusion)
22- [Troubleshooting](#troubleshooting)
23- [Additional Notes](#additional-notes)
24- [References](#references)
25
26## Prerequisites
27
28- **Toolbox Installed on the Host System** `Fedora Silverblue` and `Fedora Workstation` both have toolbox by default, other distributions may need to install the [toolbox package](https://containertoolbx.org/install/).
29- **NVIDIA Drivers and Graphics Card installed on Host System (recommended)** To run CUDA program, such as `llama.cpp`, the host should be setup to access your NVIDIA hardware. Fedora Hosts can use the [RPM Fusion Repository](https://rpmfusion.org/Howto/NVIDIA).
30- **Internet connectivity** to download packages.
31
32### Using the Fedora 41 CUDA Repository
33
34The latest release is 41.
35
36- [Fedora 41 CUDA Repository](https://developer.download.nvidia.com/compute/cuda/repos/fedora41/x86_64/)
37
38**Note:** We recommend using a toolbox environment to prevent system conflicts.
39
40## Creating a Fedora Toolbox Environment
41
42This guide focuses on Fedora hosts, but with small adjustments, it can work for other hosts. Using the Fedora Toolbox allows us to install the necessary packages without affecting the host system.
43
44**Note:** Toolbox is available for other systems, and even without Toolbox, it is possible to use Podman or Docker.
45
461. **Create a Fedora 41 Toolbox:**
47
48 ```bash
49 toolbox create --image registry.fedoraproject.org/fedora-toolbox:41 --container fedora-toolbox-41-cuda
50 ```
51
522. **Enter the Toolbox:**
53
54 ```bash
55 toolbox enter --container fedora-toolbox-41-cuda
56 ```
57
58 Inside the toolbox, you have root privileges and can install packages without affecting the host system.
59
60## Installing Essential Development Tools
61
621. **Synchronize the DNF Package Manager:**
63
64 ```bash
65 sudo dnf distro-sync
66 ```
67
682. **Install **Vim** the default text editor (Optional):**
69
70 ```bash
71 sudo dnf install vim-default-editor --allowerasing
72 ```
73
74 The `--allowerasing` flag will allow the removal of the conflicting `nano-default-editor` package.
75
763. **Install Development Tools and Libraries:**
77
78 ```bash
79 sudo dnf install @c-development @development-tools cmake
80 ```
81
82 This installs essential packages for compiling software, including `gcc`, `make`, and other development headers.
83
84## Adding the CUDA Repository
85
86Add the NVIDIA CUDA repository to your DNF configuration:
87
88```bash
89sudo dnf config-manager addrepo --from-repofile=https://developer.download.nvidia.com/compute/cuda/repos/fedora41/x86_64/cuda-fedora41.repo
90```
91
92After adding the repository, synchronize the package manager again:
93
94```bash
95sudo dnf distro-sync
96```
97
98## Installing Nvidia Driver Libraries
99
100First, we need to detect if the host is supplying the [NVIDIA driver libraries into the toolbox](https://github.com/containers/toolbox/blob/main/src/pkg/nvidia/nvidia.go):
101
102```bash
103ls -la /usr/lib64/libcuda.so.1
104```
105
106### If *`libcuda.so.1`* is missing:
107
108```
109ls: cannot access '/usr/lib64/libcuda.so.1': No such file or directory
110```
111
112**Explanation:**
113The host dose not supply the CUDA drivers, **install them now:**
114
115#### Install the Nvidia Driver Libraries on Guest:
116
117```bash
118sudo dnf install nvidia-driver-cuda nvidia-driver-libs nvidia-driver-cuda-libs nvidia-persistenced
119```
120
121### If *`libcuda.so.1`* exists:
122```
123lrwxrwxrwx. 1 root root 21 Mar 24 11:26 /usr/lib64/libcuda.so.1 -> libcuda.so.570.133.07
124```
125
126**Explanation:**
127The host is supply the CUDA drivers, **we need to update the guest RPM Database accordingly:**
128
129#### Update the Toolbox RPM Database to include the Host-Supplied Libraries:
130
131Note: we do not actually install the libraries, we just update the DB so that the guest system knows they are supplied by the host.
132
133##### 1. Download `nvidia-` parts that are supplied by the host RPM's (with dependencies)
134
135```bash
136sudo dnf download --destdir=/tmp/nvidia-driver-libs --resolve --arch x86_64 nvidia-driver-cuda nvidia-driver-libs nvidia-driver-cuda-libs nvidia-persistenced
137```
138
139##### 2. Update the RPM database to assume the installation of these packages.
140
141```bash
142sudo rpm --install --verbose --hash --justdb /tmp/nvidia-driver-libs/*
143```
144
145**Note:**
146
147- The `--justdb` option only updates the RPM database, without touching the filesystem elsewhere.
148
149##### Check that the RPM Database has been correctly updated:
150
151**Note:** This is the same command as in the *"Install the Nvidia Driver Libraries on Guest"* for if *`libcuda.so.1`* was missing.
152
153
154```bash
155sudo dnf install nvidia-driver-cuda nvidia-driver-libs nvidia-driver-cuda-libs nvidia-persistenced
156```
157
158*(this time it will not install anything, as the database things that these packages are already installed)*
159
160```
161Updating and loading repositories:
162Repositories loaded.
163Package "nvidia-driver-cuda-3:570.124.06-1.fc41.x86_64" is already installed.
164Package "nvidia-driver-libs-3:570.124.06-1.fc41.x86_64" is already installed.
165Package "nvidia-driver-cuda-libs-3:570.124.06-1.fc41.x86_64" is already installed.
166Package "nvidia-persistenced-3:570.124.06-1.fc41.x86_64" is already installed.
167
168Nothing to do.
169```
170
171## Installing the CUDA Meta-Package
172
173Now that the driver libraries are installed, proceed to install CUDA:
174
175```bash
176sudo dnf install cuda
177```
178
179This installs the CUDA toolkit and associated packages.
180
181## Configuring the Environment
182
183To use CUDA, add its binary directory to your system's `PATH`.
184
1851. **Create a Profile Script:**
186
187 ```bash
188 sudo sh -c 'echo "export PATH=\$PATH:/usr/local/cuda/bin" >> /etc/profile.d/cuda.sh'
189 ```
190
191 **Explanation:**
192
193 - We add to `/etc/profile.d/` as the `/etc/` folder is unique to this particular container, and is not shared with other containers or the host system.
194 - The backslash `\` before `$PATH` ensures the variable is correctly written into the script.
195
1962. **Make the Script Executable:**
197
198 ```bash
199 sudo chmod +x /etc/profile.d/cuda.sh
200 ```
201
2023. **Source the Script to Update Your Environment:**
203
204 ```bash
205 source /etc/profile.d/cuda.sh
206 ```
207
208 **Note:** This command updates your current shell session with the new `PATH`. The `/etc/profile.d/cuda.sh` script ensures that the CUDA binaries are available in your `PATH` for all future sessions.
209
210## Verifying the Installation
211
212To confirm that CUDA is correctly installed and configured, check the version of the NVIDIA CUDA Compiler (`nvcc`):
213
214```bash
215nvcc --version
216```
217
218You should see output similar to:
219
220```
221nvcc: NVIDIA (R) Cuda compiler driver
222Copyright (c) 2005-2025 NVIDIA Corporation
223Built on Fri_Feb_21_20:23:50_PST_2025
224Cuda compilation tools, release 12.8, V12.8.93
225Build cuda_12.8.r12.8/compiler.35583870_0
226```
227
228This output confirms that the CUDA compiler is accessible and indicates the installed version.
229
230## Conclusion
231
232You have successfully set up CUDA on Fedora within a toolbox environment using the Fedora 41 CUDA repository. By manually updating the RPM db and configuring the environment, you can develop CUDA applications without affecting your host system.
233
234## Troubleshooting
235
236- **Installation Failures:**
237
238 - If you encounter errors during installation, carefully read the error messages. They often indicate conflicting files or missing dependencies.
239 - You may use the `--excludepath` option with `rpm` to exclude conflicting files during manual RPM installations.
240
241- **Rebooting the Container:**
242
243 - Sometimes there may be a bug in the NVIDIA driver host passthrough (such as missing a shared library). Rebooting the container may solve this issue:
244
245 ```bash
246 # on the host system
247 podman container restart --all
248 ```
249
250- **Environment Variables Not Set:**
251 - If `nvcc` is not found after installation, ensure that `/usr/local/cuda/bin` is in your `PATH`.
252 - Run `echo $PATH` to check if the path is included.
253 - Re-source the profile script or open a new terminal session.
254
255## Additional Notes
256
257- **Updating CUDA in the Future:**
258
259 - Keep an eye on the official NVIDIA repositories for updates to your Fedora version.
260 - When an updated repository becomes available, adjust your `dnf` configuration accordingly.
261
262- **Building `llama.cpp`:**
263
264 - With CUDA installed, you can follow these [build instructions for `llama.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) to compile it with CUDA support.
265 - Ensure that any CUDA-specific build flags or paths are correctly set in your build configuration.
266
267- **Using the Toolbox Environment:**
268 - The toolbox environment is isolated from your host system, which helps prevent conflicts.
269 - Remember that system files and configurations inside the toolbox are separate from the host. By default the home directory of the user is shared between the host and the toolbox.
270
271---
272
273**Disclaimer:** Manually installing and modifying system packages can lead to instability of the container. The above steps are provided as a guideline and may need adjustments based on your specific system configuration. Always back up important data before making significant system changes, especially as your home folder is writable and shared with he toolbox.
274
275**Acknowledgments:** Special thanks to the Fedora community and NVIDIA documentation for providing resources that assisted in creating this guide.
276
277## References
278
279- [Fedora Toolbox Documentation](https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/)
280- [NVIDIA CUDA Installation Guide](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)
281- [Podman Documentation](https://podman.io/get-started)
282
283---