GGUF

How it works

A GGUF file starts with the bytes GGUF, a version, and counts of tensors and metadata entries. Then come the metadata as typed key-value pairs, each tensor’s name, shape, type and position, and the tensor bytes.

Models are usually trained in PyTorch or another framework and converted to GGUF, often quantized on the way: the spec says the tensor data may differ from the original “due to quantization or other optimizations for inference”.

The metadata names the architecture and can carry the tokenizer and tokenizer.chat_template, “a Jinja template that specifies the input format expected by the model”. A runner such as llama.cpp can use that chat template to build the prompt. Hugging Face contrasts GGUF with tensor-only formats such as safetensors: GGUF “encodes both the tensors and a standardized set of metadata”.

Why it matters

A GGUF file is untrusted input, and both its parser and its chat template have been attack routes.

The parser. In January 2024 Neil Archibald of Databricks reported that GGML “performs insufficient validation on the input file”. He found “potentially exploitable” heap overflows, where a crafted file makes the parser write past its memory, that an attacker “could leverage” to run code (CVE-2024-25664 to 25668). Patches for them were merged six days after the report.

The template renderer. llama-cpp-python versions 0.2.30 to 0.2.71 rendered a file’s chat template without Jinja’s sandbox, the mode for “untrusted templates”, so a template could run code (CVE-2024-34359, fixed in 0.2.72).

A template needs no bug to do harm. In MITRE ATLAS case study AML.CS0064, an exercise, Pillar Security and Fujitsu Research of Europe poisoned GGUF chat templates so that on a trigger “the template injects attacker-controlled instructions into the context sent to the model”, without touching the weights. ATLAS maps the step where the engine runs that template to unsafe AI artifacts.

In practice

Checking the template on a model page may not check the file you download. Pillar reported that Hugging Face “only displays the template from the first file” in a repository, so a poisoned template can sit in another quantization such as Q4_K_M.gguf. Hugging Face replied in June 2025 that this was not a vulnerability “but rather how the UI displays chat templates”. LM Studio replied that users are responsible for downloading trusted models.

What helps: keep the runner patched, read the chat template in the exact file you run, and verify AI artifacts before loading them.

Questions and answers

What does GGUF stand for?

The GGUF specification does not expand the name. MITRE ATLAS and Pillar Security write it as GPT-Generated Unified Format. The specification describes GGUF as the successor to the GGML, GGMF and GGJT formats.

Is a GGUF file safe to download and run?

A GGUF file is untrusted input. In January 2024 a Databricks researcher reported potentially exploitable heap overflows in GGML's GGUF parser, patched that month. llama-cpp-python versions 0.2.30 to 0.2.71 rendered a file's chat template without a sandbox, which let a template run code; 0.2.72 fixed it. A chat template can also be poisoned to change the model's behaviour with no bug at all.

What is the difference between GGUF and safetensors?

Hugging Face describes safetensors as a tensor-only format and GGUF as encoding "both the tensors and a standardized set of metadata". In GGUF that metadata can include the tokenizer and a Jinja chat template.