Jellyfin hardware acceleration: test your GPU, fix the render group, and see why it transcodes

Jellyfin does not detect your graphics card by itself. You pick the acceleration method and the codecs by hand in Dashboard → Playback → Transcoding, and a wrong choice or a missing permission quietly sends transcoding back to the processor. No error, no banner — just a hot CPU and fans you can hear from the hallway.

Transcode Doctor is a Jellyfin plugin that takes the guesswork out of this. It tests what your server can really do with Jellyfin's own ffmpeg, names the environment problem with the exact fix, applies the settings that passed with one click, and explains in plain words why each playback is being transcoded. This tutorial walks through the whole loop on a self-hosted server.

Direct play vs transcoding, in one paragraph

If the client can play the file as it is, Jellyfin streams it untouched (direct play) and the CPU barely notices. When it can't — the container, video codec, audio codec, subtitles, HDR or a bitrate limit get in the way — Jellyfin transcodes, and that work lands on either the GPU or the CPU. Getting hardware acceleration right is what keeps that second case cheap.

Prerequisites

• Jellyfin Server 12.2 or newer, with jellyfin-ffmpeg (the ffmpeg build that ships with Jellyfin).
• An NVIDIA, Intel or AMD graphics card with its driver, reachable by Jellyfin: for NVIDIA, a driver with nvidia-smi; for Intel and AMD, a /dev/dri render device the Jellyfin user can open, usually through the render group. In Docker, the device must be passed into the container.
• A MediaLab Works account with a license covering the plugin (subscription, lifetime within its update window, the Jellyfin bundle, or the free 30-day trial).

Step 1 — Install and activate the plugin

Install from your private plugin repository (Dashboard → Plugins → Repositories, then the catalog), from the public catalog, or manually from the zip. The user guide has the exact paths for package, Docker and Windows installs. After a Jellyfin restart, open the plugin page and click License, then paste your key and Activate with key — or start the 30-day free trial from the same page.

Step 2 — Run the hardware acceleration diagnosis

Open Transcode Doctor in the dashboard menu and click Run the tests. The plugin:

• checks ffmpeg, the NVIDIA card and the /dev/dri devices with the permissions of the Jellyfin process;
• generates a few short test clips with ffmpeg in Jellyfin's cache folder (transcode-doctor/) — your media is not touched;
• tries each method — NVIDIA NVENC, Intel QuickSync and VA-API (Intel/AMD) — at decoding H.264, HEVC, HEVC 10-bit, MPEG-2, VP9 and AV1 and at encoding H.264 and HEVC, with the measured speed.

Each test takes at most a minute. The table shows what works and how fast — that measured speed is the number you actually care about, not the marketing name on the box.

Step 3 — Read the problem and apply the exact fix

If something blocks acceleration, the page says what and how to fix it. The most common case is Jellyfin with no access to /dev/dri/renderD128. The plugin shows the command to run, which adds the Jellyfin user to the render group:

sudo usermod -aG render jellyfin

Restart Jellyfin, then run the tests again. Other cases the plugin flags include an NVIDIA card whose driver does not respond, and ffmpeg missing at the configured path. In Docker, the GPU device has to be passed into the container before any of this can pass.

Step 4 — Apply the recommendation (and know how to undo it)

When the tests are done, the page shows the method that passed the most tests, the decoding codecs, hardware encoding, and roughly how many 1080p transcodes can run at once — next to your current settings.

Apply the settings writes the recommendation into Dashboard → Playback → Transcoding: the acceleration method, the hardware decoding codecs that passed, HEVC 10-bit decoding, hardware encoding and the VA-API / QuickSync device. Nothing changes until you click. VC-1, VP8 and MPEG-4 decoding cannot be tested, so they stay as you set them; HEVC encoding is turned off if it failed and is never turned on by the plugin.

Before applying, the plugin saves your previous transcoding settings. Restore the previous settings puts them back exactly, which is what you want if a driver update or a Jellyfin upgrade changes the picture. You can always edit the settings by hand in Jellyfin too.

Step 5 — Find out why each playback transcodes

This is the part that saves the most time later. Why it transcodes lists recent transcoded playbacks — when, who, what — with Jellyfin's reasons explained in plain words and what can be done, plus whether the GPU or the CPU did the work. Up to 500 playbacks are kept.

Typical reasons you will see:

• Burned-in PGS subtitles. Image-based subtitles have to be drawn into the video frames, so the stream is re-encoded. A text-based subtitle track avoids it.
• HDR on an SDR screen. Tone mapping is a transcode by definition unless the client handles HDR itself.
• Audio codec. The client may accept the video but not the audio track, which still forces a transcode of the stream.
• The app's bitrate limit. A client set below the file's bitrate asks for a lower-quality stream.

"CPU (GPU not used)" on a playback while acceleration is configured means the card was not used for that one — worth a look at the method and codec combination.

Step 6 — Let the activity log watch for CPU fallback

When a video transcode runs on the CPU although hardware acceleration is configured, the plugin writes an entry in Jellyfin's activity log. The alert is on by default and only writes locally. It is the cheapest early warning you can have after a driver or Jellyfin upgrade.

Tips and troubleshooting

• Re-run the tests after any driver, Jellyfin or hardware change. A method that passed last month can fail after an update.
• Diagnosis says ffmpeg not found: install jellyfin-ffmpeg and set its path in the dashboard, then run the tests again.
• Docker: the render device must be mapped into the container; permission fixes on the host do nothing if the device never reaches Jellyfin.
• Platform notes: the checks are Linux-oriented (/dev/dri, nvidia-smi, the render group). NVIDIA NVENC is verified on a real Quadro M2000; Intel QSV and VA-API are detected and tested by the plugin but were not tested on real hardware by the developer. AMD AMF, Apple VideoToolbox and Rockchip are not supported.
• Privacy: the plugin downloads no test files — clips are generated locally — and nothing it stores leaves the server. The diagnosis runs only when you click the button; see Licensing and privacy in our plugins.
• Updates: keep to your private plugin repository so Jellyfin only offers versions your license allows, and use Roll back to x.y.z on the License page if an update misbehaves. Release notes are in the changelog.

Conclusion

Hardware acceleration in Jellyfin is not a checkbox you can guess your way through: the method, the codecs and the permissions all have to line up, and when they don't, the only symptom is a busy CPU. Transcode Doctor turns that into a measured answer — test with Jellyfin's own ffmpeg, fix the one thing that is actually broken, apply what passed with a single click and a one-click way back, and read the reason behind every transcode. If you get stuck, the knowledge base and FAQ are a good next stop, or contact us with the diagnosis report.