{
  "dataset": "inference",
  "source": "https://gpucompat.com/data/inference.json",
  "about": "https://gpucompat.com/about-the-data/",
  "count": 11,
  "entries": [
    {
      "id": "exllamav2-archived",
      "engine": "exllamav2",
      "version": "0.3.2 (last release)",
      "releaseDate": "2025-07-13",
      "status": "archived",
      "cudaVersions": [
        "12.1"
      ],
      "notes": "ExLlamaV2's own README states up front: \"This project is archived for now. Development continues on ExLlamaV3\" -- v0.3.2 (2025-07-13) is its last tagged GitHub release and the repository has seen no further releases since. See the exllamav3 entry on this page for the actively maintained successor.\ncudaVersions (12.1) reflects the `cu121` tag on this project's documented example prebuilt-wheel filename, not a currently-maintained compatibility claim -- treat it as historical context, not installation guidance. ExLlamaV2 was CUDA/PyTorch-only: its setup.py does branch on `torch.version.hip` for an AMD/HIP code path, but no ROCm wheels or ROCm version support were ever documented for it, so rocmVersions is left unset here rather than guessed. Source: github.com/turboderp-org/exllamav2 (README.md, setup.py) and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "exllamav3-1-4-7",
      "engine": "exllamav3",
      "version": "1.4.7",
      "releaseDate": "2026-09-05",
      "status": "current",
      "cudaVersions": [
        "12.8",
        "13.2"
      ],
      "pythonVersions": [
        "3.10",
        "3.11",
        "3.12",
        "3.13",
        "3.14"
      ],
      "pytorchVersions": [
        "2.7.0",
        "2.8.0",
        "2.9.0",
        "2.10.0",
        "2.11.0"
      ],
      "notes": "The actively developed successor to the now-archived ExLlamaV2 (see that entry). Prebuilt wheels for this release are published only for CUDA 12.8 (\"cu128\") and CUDA 13.2 (\"cu132\"), each built across PyTorch 2.7.0 through 2.11.0 and Python 3.10 through 3.14 (enumerated from the v1.4.7 GitHub release's asset filenames, e.g. exllamav3-1.4.7+cu128.torch2.10.0-cp312-cp312-linux_x86_64.whl). Installing from PyPI or from source instead requires the CUDA Toolkit and a matching Torch build yourself -- requirements.txt pins `torch>=2.6.0` as a floor, not a ceiling, so newer combinations than the prebuilt wheels may also work unverified.\nNo AMD support: the project's own README lists \"ROCm support\" under \"What's missing?\" as a to-do item, so this is CUDA/NVIDIA-only today. rocmVersions is intentionally omitted here (confirmed absent, not merely unconfirmed). Source: github.com/turboderp-org/exllamav3 (README.md, requirements.txt) and the v1.4.7 release assets via the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "llama-cpp-current",
      "engine": "llama-cpp",
      "version": "rolling release (no numbered versions)",
      "releaseDate": "N/A -- commits are tagged with incrementing build numbers (e.g. b10826 as of 2026-09-06), not dated releases",
      "status": "current",
      "cudaVersions": [
        "12.4",
        "12.6",
        "12.8",
        "13.3",
        "13.4"
      ],
      "rocmVersions": [
        "6.1",
        "7.0"
      ],
      "notes": "llama.cpp doesn't publish a single \"supported CUDA version\" the way PyTorch or vLLM do -- it's compiled from source (or from AMD/community Docker images) against whatever CUDA Toolkit is installed, and the project's own CI simply tracks a moving target rather than pinning one. As of build b10826 (2026-09-06): its official Docker images build against CUDA 12.8.1 (the \"cuda12\" tag) and CUDA 13.3.0 (the \"cuda13\" tag) per .github/workflows/docker.yml; its Ubuntu CUDA CI job builds inside an nvidia/cuda:12.6.2-devel container per .github/workflows/build-cuda-ubuntu.yml; and its Windows CUDA CI job tests 12.4, 13.3, and 13.4 per .github/workflows/build-cuda-windows.yml. 12.4 through 13.4 is therefore a reasonable \"known-good\" range rather than a hard minimum or maximum -- older CUDA Toolkits (down to roughly 11.x) are widely reported by users to still work but aren't exercised by current CI.\nROCm: the same Ubuntu CI job's HIP build runs inside a rocm/dev-ubuntu-22.04:6.1.2 container (this project's own practical floor), while AMD's own ROCm documentation separately validates a specific llama.cpp build (b6652) against ROCm 7.0.0 for the Instinct MI325X/MI300X/MI210 -- see rocm.docs.amd.com/projects/llama-cpp/en/docs-26.02/install/llama-cpp-install.html. Both figures are included as rocmVersions since they represent two different (and both legitimate) \"what's actually been tested\" claims from two different sources, not a single official minimum.\nThis entry is a point-in-time snapshot, not a version history -- unlike PyTorch/vLLM, llama.cpp has no stable version numbers to track over time. pythonVersions is omitted: llama.cpp itself is a C++ CLI/server project with no Python version dependency of its own, distinct from the separately versioned `llama-cpp-python` PyPI bindings package, which isn't covered by this entry. Source: github.com/ggml-org/llama.cpp workflow files (.github/workflows/build-cuda-ubuntu.yml, build-cuda-windows.yml, docker.yml) and docs/build.md at the current master commit, plus AMD's ROCm documentation above, verified 2026-09-06.\n"
    },
    {
      "id": "ollama-0-33-3",
      "engine": "ollama",
      "version": "0.33.3",
      "releaseDate": "2026-09-02",
      "status": "current",
      "rocmVersions": [
        "7.0"
      ],
      "minComputeCapability": "5.0",
      "minNvidiaDriver": "550 (570+ required for compute capability 5.0-6.2 specifically)",
      "notes": "Ollama bundles its own CUDA/ROCm runtime libraries rather than requiring a separately-installed CUDA Toolkit, so its own docs express compatibility in terms of NVIDIA driver version and GPU compute capability rather than a CUDA Toolkit version: \"Ollama supports Nvidia GPUs with compute capability 5.0+ and driver version 550 and newer. Nvidia GPUs with compute capability 5.0 through 6.2 require driver version 570 or newer\" (docs/gpu.mdx).\nOn AMD, Ollama \"requires the AMD ROCm v7 driver on Linux\" and \"an AMD ROCm v7 / HIP7-capable driver stack on Windows\" (hence rocmVersions: [\"7.0\"]), with a broad supported-card list spanning Radeon RX 6800 through 9070 XT, Radeon PRO, Ryzen AI, and Instinct MI100 through MI350X (Linux list is broader than Windows). For AMD cards ROCm doesn't officially list, Ollama's docs describe the same HSA_OVERRIDE_GFX_VERSION community workaround covered on this site's /rocm-consumer-gpus/ page, with their own worked example: forcing an RX 5400 (gfx1034, unsupported) to identify as gfx1030 via HSA_OVERRIDE_GFX_VERSION=\"10.3.0\".\nversion/releaseDate is Ollama's latest non-prerelease GitHub release as of data-pull time (v0.34.0-rc1 existed but is a release candidate, not used here). Source: github.com/ollama/ollama docs/gpu.mdx and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "vllm-0-11-1",
      "engine": "vllm",
      "version": "0.11.1",
      "releaseDate": "2025-11-18",
      "status": "historical",
      "cudaVersions": [
        "12.9"
      ],
      "pythonVersions": [
        ">=3.10,<3.14"
      ],
      "notes": "Default CUDA_VERSION moved to 12.9.1 at this release (from 12.8.1, unchanged through v0.11.0) per docker/Dockerfile's ARG CUDA_VERSION. requires-python's floor also moved up to 3.10 at this release (from >=3.9 through v0.9.0/ v0.10.x). Source: github.com/vllm-project/vllm at tag v0.11.1 (docker/Dockerfile, pyproject.toml) and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "vllm-0-20-0",
      "engine": "vllm",
      "version": "0.20.0",
      "releaseDate": "2026-04-27",
      "status": "historical",
      "cudaVersions": [
        "13.0"
      ],
      "pythonVersions": [
        ">=3.10,<3.15"
      ],
      "notes": "First release built against CUDA 13 (default CUDA_VERSION 13.0.2, up from 12.9.1 through v0.19.1) per docker/Dockerfile's ARG CUDA_VERSION -- vLLM's jump to the CUDA 13 major line. Source: github.com/vllm-project/vllm at tag v0.20.0 (docker/Dockerfile, pyproject.toml) and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "vllm-0-28-0",
      "engine": "vllm",
      "version": "0.28.0",
      "releaseDate": "2026-08-26",
      "status": "historical",
      "cudaVersions": [
        "13.0"
      ],
      "rocmVersions": [
        "7.0",
        "7.2.1"
      ],
      "pythonVersions": [
        ">=3.10,<3.15"
      ],
      "notes": "Latest stable release as of 2026-09-06. Default CUDA_VERSION is 13.0.3 (docker/Dockerfile's ARG CUDA_VERSION; a patch bump from 13.0.2, unchanged since v0.20.0). On AMD, vLLM's own docs state it \"supports AMD GPUs with ROCm 6.3 or above,\" with pre-built ROCm wheels currently published for ROCm 7.0 and 7.2.1 -- rocmVersions here lists those two published-wheel versions specifically, not the full \"6.3 or above\" range the docs also describe as generally working. Source: github.com/vllm-project/vllm at tag v0.28.0 (docker/Dockerfile, pyproject.toml), docs.vllm.ai/en/stable/getting_started/installation/gpu/ for the ROCm support statement, and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "vllm-0-29-0",
      "engine": "vllm",
      "version": "0.29.0",
      "releaseDate": "2026-09-09",
      "status": "historical",
      "cudaVersions": [
        "13.0"
      ],
      "rocmVersions": [
        "7.0",
        "7.2.1"
      ],
      "pythonVersions": [
        ">=3.10,<3.15"
      ],
      "notes": "Default CUDA_VERSION is 13.0.3 (docker/Dockerfile's ARG CUDA_VERSION at tag v0.29.0, unchanged from 0.28.0). requires-python \">=3.10,<3.15\" from pyproject.toml at the same tag. On AMD, vLLM's docs at this tag state it \"supports AMD GPUs with ROCm 6.3 or above\" with pre-built wheels for ROCm 7.0 and 7.2.1; rocmVersions lists those two published-wheel versions. NVIDIA GPUs need compute capability 7.5 or higher (docs/getting_started/ installation/gpu.cuda.inc.md at this tag). Source: github.com/vllm-project/vllm at tag v0.29.0 and the GitHub Releases API (published 2026-09-09), verified 2026-09-27.\n"
    },
    {
      "id": "vllm-0-30-0",
      "engine": "vllm",
      "version": "0.30.0",
      "releaseDate": "2026-09-22",
      "status": "current",
      "cudaVersions": [
        "13.0"
      ],
      "rocmVersions": [
        "7.0",
        "7.2.1"
      ],
      "pythonVersions": [
        ">=3.10,<3.15"
      ],
      "notes": "Default CUDA_VERSION is 13.0.3 (docker/Dockerfile's ARG CUDA_VERSION at tag v0.30.0, unchanged from 0.28.0). requires-python \">=3.10,<3.15\" from pyproject.toml at the same tag. On AMD, vLLM's docs at this tag state it \"supports AMD GPUs with ROCm 6.3 or above\" with pre-built wheels for ROCm 7.0 and 7.2.1; rocmVersions lists those two published-wheel versions. NVIDIA GPUs need compute capability 7.5 or higher (docs/getting_started/ installation/gpu.cuda.inc.md at this tag). Source: github.com/vllm-project/vllm at tag v0.30.0 and the GitHub Releases API (published 2026-09-22), verified 2026-09-27.\n"
    },
    {
      "id": "vllm-0-8-5",
      "engine": "vllm",
      "version": "0.8.5",
      "releaseDate": "2025-04-28",
      "status": "historical",
      "cudaVersions": [
        "12.4"
      ],
      "pythonVersions": [
        ">=3.9,<3.13"
      ],
      "notes": "cudaVersions reflects the default CUDA_VERSION (12.4.1) baked into vLLM's own official Docker image build args at this tag (docker/Dockerfile, ARG CUDA_VERSION) -- not a full list of every CUDA version the wheel works with. vLLM's pip wheels auto-detect the installed CUDA via PyTorch/ nvidia-smi rather than pinning to one version the way PyTorch's own wheels do, so this figure represents \"what vLLM itself builds and tests against by default\" at this release, not a hard requirement. pythonVersions is this tag's pyproject.toml `requires-python`. Last release on the 12.4.1 default before it moved to 12.8.1 in v0.9.0. Source: github.com/vllm-project/vllm at tag v0.8.5 (docker/Dockerfile, pyproject.toml) and the GitHub Releases API, verified 2026-09-06.\n"
    },
    {
      "id": "vllm-0-9-0",
      "engine": "vllm",
      "version": "0.9.0",
      "releaseDate": "2025-05-15",
      "status": "historical",
      "cudaVersions": [
        "12.8"
      ],
      "pythonVersions": [
        ">=3.9,<3.13"
      ],
      "notes": "First release built with the default CUDA_VERSION bumped to 12.8.1 (from 12.4.1 in v0.8.5) per docker/Dockerfile's ARG CUDA_VERSION at this tag -- as with every entry in this collection, this is vLLM's own default build target, not an exhaustive compatibility list; the wheel auto-detects the installed CUDA rather than pinning to one version. Source: github.com/vllm-project/vllm at tag v0.9.0 (docker/Dockerfile, pyproject.toml) and the GitHub Releases API, verified 2026-09-06.\n"
    }
  ]
}