1# GGML-VirtGPU Backend Configuration
  2
  3This document describes the environment variables used by the ggml-virtgpu backend system, covering both the frontend (guest-side) and backend (host-side) components.
  4
  5## Environment Variables Overview
  6
  7The ggml-virtgpu backend uses environment variables for configuration across three main components:
  8- **Frontend (Guest)**: GGML applications running in VMs
  9- **Hypervisor**: Virglrenderer/APIR system
 10- **Backend (Host)**: Host-side GGML backend integration
 11
 12## Frontend (Guest-side) Configuration
 13
 14### GGML_REMOTING_USE_APIR_CAPSET
 15- **Location**: `ggml/src/ggml-virtgpu/virtgpu.cpp`
 16- **Type**: Boolean flag (presence-based)
 17- **Purpose**: Controls which virtio-gpu capability set to use for communication
 18- **Values**:
 19  - Set (any value): Use the APIR capset (long-term setup)
 20  - Unset: Use the Venus capset (easier for testing with an unmodified hypervisor)
 21- **Default**: Unset (Venus capset)
 22- **Usage**:
 23  ```bash
 24  export GGML_REMOTING_USE_APIR_CAPSET=1  # Use APIR capset
 25  # or leave unset for Venus capset
 26  ```
 27
 28## Hypervisor (Virglrenderer/APIR) Configuration
 29
 30These environment variables are used during the transition phase for
 31running with an unmodified hypervisor (not supporting the
 32VirglRenderer APIR component). They will be removed in the future, and
 33the hypervisor will instead configure VirglRenderer with the APIR
 34_Configuration Key_.
 35
 36### VIRGL_APIR_BACKEND_LIBRARY
 37- **Location**: `virglrenderer/src/apir/apir-context.c`
 38- **Configuration Key**: `apir.load_library.path`
 39- **Type**: File path string
 40- **Purpose**: Path to the APIR backend library that virglrenderer should dynamically load
 41- **Required**: Yes
 42- **Example**:
 43  ```bash
 44  export VIRGL_APIR_BACKEND_LIBRARY="/path/to/libggml-remotingbackend.so"
 45  ```
 46
 47### VIRGL_ROUTE_VENUS_TO_APIR
 48- **Location**: `virglrenderer/src/apir/apir-renderer.h`
 49- **Type**: Boolean flag (presence-based)
 50- **Purpose**: Temporary workaround to route Venus capset calls to APIR during hypervisor transition period
 51- **Status**: will be removed once hypervisors support APIR natively
 52- **Warning**: Breaks normal Vulkan/Venus functionality
 53- **Usage**:
 54  ```bash
 55  export VIRGL_ROUTE_VENUS_TO_APIR=1  # For testing with an unmodified hypervisor
 56  ```
 57
 58### VIRGL_APIR_LOG_TO_FILE
 59- **Location**: `virglrenderer/src/apir/apir-renderer.c`
 60- **Environment Variable**: `VIRGL_APIR_LOG_TO_FILE`
 61- **Type**: File path string
 62- **Purpose**: Enable debug logging from the VirglRenderer APIR component to specified file
 63- **Required**: No (optional debugging)
 64- **Default**: Logging to `stderr`
 65- **Usage**:
 66  ```bash
 67  export VIRGL_APIR_LOG_TO_FILE="/tmp/apir-debug.log"
 68  ```
 69
 70## Backend (Host-side) Configuration
 71
 72These environment variables are used during the transition phase for
 73running with an unmodified hypervisor (not supporting the
 74VirglRenderer APIR component). They will be removed in the future, and
 75the hypervisor will instead configure VirglRenderer with the APIR
 76_Configuration Key_.
 77
 78### APIR_LLAMA_CPP_GGML_LIBRARY_PATH
 79- **Location**: `ggml/src/ggml-virtgpu/backend/backend.cpp`
 80- **Environment Variable**: `APIR_LLAMA_CPP_GGML_LIBRARY_PATH`
 81- **Configuration Key**: `ggml.library.path`
 82- **Type**: File path string
 83- **Purpose**: Path to the actual GGML backend library (Metal, CUDA, Vulkan, etc.)
 84- **Required**: **Yes** - backend initialization fails without this
 85- **Examples**:
 86  ```bash
 87  # macOS with Metal backend
 88  export APIR_LLAMA_CPP_GGML_LIBRARY_PATH="/opt/llama.cpp/lib/libggml-metal.dylib"
 89
 90  # Linux with CUDA backend
 91  export APIR_LLAMA_CPP_GGML_LIBRARY_PATH="/opt/llama.cpp/lib/libggml-cuda.so"
 92
 93  # macOS or Linux with Vulkan backend
 94  export APIR_LLAMA_CPP_GGML_LIBRARY_PATH="/opt/llama.cpp/lib/libggml-vulkan.so"
 95  ```
 96
 97### APIR_LLAMA_CPP_GGML_LIBRARY_REG
 98- **Location**: `ggml/src/ggml-virtgpu/backend/backend.cpp`
 99- **Environment Variable**: `APIR_LLAMA_CPP_GGML_LIBRARY_REG`
100- **Configuration Key**: `ggml.library.reg`
101- **Type**: Function symbol name string
102- **Purpose**: Name of the backend registration function to call after loading the library
103- **Required**: No (defaults to `ggml_backend_init`)
104- **Default**: `ggml_backend_init`
105- **Examples**:
106  ```bash
107  # Metal backend
108  export APIR_LLAMA_CPP_GGML_LIBRARY_REG="ggml_backend_metal_reg"
109
110  # CUDA backend
111  export APIR_LLAMA_CPP_GGML_LIBRARY_REG="ggml_backend_cuda_reg"
112
113  # Vulkan backend
114  export APIR_LLAMA_CPP_GGML_LIBRARY_REG="ggml_backend_vulkan_reg"
115
116  # Generic fallback (default)
117  # export APIR_LLAMA_CPP_GGML_LIBRARY_REG="ggml_backend_init"
118  ```
119
120### APIR_LLAMA_CPP_LOG_TO_FILE
121- **Location**: `ggml/src/ggml-virtgpu/backend/backend.cpp:62`
122- **Environment Variable**: `APIR_LLAMA_CPP_LOG_TO_FILE`
123- **Type**: File path string
124- **Purpose**: Enable debug logging from the GGML backend to specified file
125- **Required**: No (optional debugging)
126- **Usage**:
127  ```bash
128  export APIR_LLAMA_CPP_LOG_TO_FILE="/tmp/ggml-backend-debug.log"
129  ```
130
131## Configuration Flow
132
133The configuration system works as follows:
134
1351. **Hypervisor Setup**: Virglrenderer loads the APIR backend library specified by `VIRGL_APIR_BACKEND_LIBRARY`
136
1372. **Context Creation**: When an APIR context is created, it populates a configuration table with environment variables:
138   - `apir.load_library.path` ← `VIRGL_APIR_BACKEND_LIBRARY`
139   - `ggml.library.path` ← `APIR_LLAMA_CPP_GGML_LIBRARY_PATH`
140   - `ggml.library.reg` ← `APIR_LLAMA_CPP_GGML_LIBRARY_REG`
141   - this step will eventually be performed by the hypervisor itself, with command-line arguments instead of environment variables.
142
1433. **Backend Initialization**: The backend queries the configuration via callbacks:
144   - `virgl_cbs->get_config(ctx_id, "ggml.library.path")` returns the library path
145   - `virgl_cbs->get_config(ctx_id, "ggml.library.reg")` returns the registration function
146
1474. **Library Loading**: The backend dynamically loads and initializes the specified GGML library
148
149## Error Messages
150
151Common error scenarios and their messages:
152
153- **Missing library path**: `"cannot open the GGML library: env var 'APIR_LLAMA_CPP_GGML_LIBRARY_PATH' not defined"`
154- **Missing registration function**: `"cannot register the GGML library: env var 'APIR_LLAMA_CPP_GGML_LIBRARY_REG' not defined"`
155
156## Example Complete Configuration
157
158Here's an example configuration for a macOS host with Metal backend:
159
160```bash
161# Hypervisor environment
162export VIRGL_APIR_BACKEND_LIBRARY="/opt/llama.cpp/lib/libggml-virtgpu-backend.dylib"
163
164# Backend configuration
165export APIR_LLAMA_CPP_GGML_LIBRARY_PATH="/opt/llama.cpp/lib/libggml-metal.dylib"
166export APIR_LLAMA_CPP_GGML_LIBRARY_REG="ggml_backend_metal_reg"
167
168# Optional logging
169export VIRGL_APIR_LOG_TO_FILE="/tmp/apir.log"
170export APIR_LLAMA_CPP_LOG_TO_FILE="/tmp/ggml.log"
171
172# Guest configuration
173export GGML_REMOTING_USE_APIR_CAPSET=1
174```