1# llama.cpp/examples/debug
 2
 3This is a utility intended to help debug a model by registering a callback that
 4logs GGML operations and tensor data. It can also store the generated logits or
 5embeddings as well as the prompt and token ids for comparision with the original
 6model.
 7
 8### Usage
 9
10```shell
11llama-debug \
12  --hf-repo ggml-org/models \
13  --hf-file phi-2/ggml-model-q4_0.gguf \
14  --model phi-2-q4_0.gguf \
15  --prompt hello \
16  --save-logits \
17  --verbose
18```
19The tensor data is logged as debug and required the --verbose flag. The reason
20for this is that while useful for a model with many layers there can be a lot of
21output. You can filter the tensor names using the `--tensor-filter` option.
22
23A recommended approach is to first run without `--verbose` and see if the
24generated logits/embeddings are close to the original model. If they are not,
25then it might be required to inspect tensor by tensor and in that case it is
26useful to enable the `--verbose` flag along with `--tensor-filter` to focus on
27specific tensors.
28
29### Options
30This example supports all standard `llama.cpp` options and also accepts the
31following options:
32```console
33$ llama-debug --help
34...
35
36----- example-specific params -----
37
38--save-logits                           save final logits to files for verification (default: false)
39--logits-output-dir PATH                directory for saving logits output files (default: data)
40--tensor-filter REGEX                   filter tensor names for debug output (regex pattern, can be specified multiple times)
41```
42
43### Output Files
44
45When `--save-logits` is enabled, the following files are created in the output
46directory:
47
48* `llamacpp-<model>[-embeddings].bin`        - Binary output (logits or embeddings)
49* `llamacpp-<model>[-embeddings].txt`        - Text output (logits or embeddings, one per line)
50* `llamacpp-<model>[-embeddings]-prompt.txt` - Prompt text and token IDs
51* `llamacpp-<model>[-embeddings]-tokens.bin` - Binary token IDs for programmatic comparison
52
53These files can be compared against the original model's output to verify the
54converted model.