You have spent weeks on your setup. The Emby or Jellyfin library scans, the Nextcloud app installs, the website loads on your laptop. Then a friend opens it on a TV, a phone and a browser and something "sometimes" does not work. That is exactly the moment an independent check pays off.
This tutorial walks through how we run a QA audit at Quality assurance, written so you can either do the same checks yourself or hand the list over and buy a few hours of testing. The deliverable is always the same: a written report with what was tested, what was found and what to fix first. Think of it as a website testing checklist and a media server checklist in one pass.
What you need first
- A staging copy or a quiet window on the live system. The audit includes an update rehearsal on a copy of your server, so a copy is ideal, but a low-traffic evening works too.
- The list of things that matter: which clients people actually use (TVs, phones, browsers, client apps), which users and permissions exist, which ports are open to the internet.
- Access for the tester: an admin account on the media server or Nextcloud instance, and a normal user account, because permissions are part of the test.
- A way to reach us with the scope: the contact form, a ticket or Telegram. You get a written estimate with scope and hours free of charge before anything is bought.
Hours are bought on the product page with a minimum of one hour and are consumed in 15-minute steps. Unused hours stay in your account for 12 months.
Step 1: Write down the scope
Before touching anything, agree on what will be checked. The audit covers software (test plan, manual and automated tests, bug reports with steps to reproduce), media and mail servers (Emby, Jellyfin, Nextcloud or mailcow: configuration, users and permissions, security, performance, transcoding, mail delivery and backups), devices (TVs, phones, browsers, client apps: compatibility, subtitles, audio tracks, transcoding and remote access), websites (accessibility, SEO and performance with Lighthouse and manual checks) and changes (update rehearsal, backup restore test, load testing, basic security checks).
Pick the parts that apply to you. A Jellyfin operator with a small website needs a different mix than a Nextcloud admin. A short written scope keeps the hours focused and makes the final report easier to read.
Step 2: Build the playback matrix
This is the core of a media server check. Make a table with your clients as rows and the things that can break as columns.
Suggested rows:
- Living room TV with the native app
- Phone (at least one Android and one iOS device)
- Desktop browser
- Any other client app your users rely on
Suggested columns:
- Direct play of a typical file
- Transcoding of a file the client cannot play (check that it actually transcodes rather than failing)
- Subtitles: embedded and external, and whether they stay in sync
- Audio tracks: switching between them mid-playback
- Remote access from outside your network
Fill each cell with pass or fail and a one-line note. The failures are what the report will list, each with steps to reproduce and a severity. Do not chase every combination; a representative sample per client is enough to catch the usual suspects.
For Emby and Jellyfin, configuration and permissions belong in the same pass: check that a normal user cannot see libraries or admin areas they should not, and that transcoding settings match what your hardware can do.
Step 3: Check security exposure
This is where small setups most often have surprises. Stay on the configuration level: which ports are reachable from the internet, whether certificates are valid and renewing, which accounts exist and whether any default or unused admin accounts are still active. For Nextcloud, users and permissions are part of the media server check; for mailcow, mail delivery is checked as well.
Basic security testing is part of the service, so you do not need to run exploits yourself. What helps is a plain inventory: list every service you expose, why it is exposed and who should be able to reach it. Anything you cannot justify goes back behind the firewall or the VPN. If you run the audit yourself, this inventory alone is often worth the afternoon.
Step 4: Performance, SEO and accessibility for the website
If you also run a website, the audit includes accessibility, SEO and performance audits using Lighthouse plus manual checks, with priorities. Run Lighthouse on a representative page, not only the homepage, and note the three results that matter most to you. Manual checks catch what Lighthouse cannot: keyboard navigation, readable contrast, meaningful link text, sensible headings.
Keep this section short and factual in the report. "Largest contentful paint is slow on the pricing page, image is uncompressed" is more useful than a wall of scores. Fixes can be done by us as custom development or by your own team using the report.
Step 5: Rehearse the update and restore a backup
An update rehearsal on a copy of your server answers the question "will the next version break my plugins or clients?" before you find out the hard way. A backup restore test answers the scarier question: does the backup actually restore? Both are part of the service and both belong in the report as pass or fail with notes.
Load testing is also available if you expect a burst of users, for example after publishing a link.
Step 6: Turn it into a report you can act on
The report is the deliverable. Insist on this structure, whether you write it yourself or receive ours:
- What was tested, with the scope and the clients used.
- Findings, each with severity, steps to reproduce and, where useful, a screenshot.
- What to fix first, ordered by severity and effort.
- What passed, so you know what not to touch.
Severity is what makes a report useful. A failed login for normal users is not the same as a subtitle offset on one TV. Rank honestly and keep the list short enough to finish.
Step 7: Fix, then re-test
Fixes are re-tested as part of the service. This is the step people skip: a fix that is not verified is a hope, not a fix. Re-run the failing cells of the playback matrix, re-check the exposed ports you closed, and re-run Lighthouse on the pages you changed. Hours are consumed in 15-minute steps, so a short re-test is cheap.
Tips and troubleshooting
- "It sometimes doesn't work" is a symptom, not a bug. Note the client, the file, the time and what you saw. That note is the start of a reproduction.
- Test with a normal user account, not only the admin. Most permission problems hide there.
- Do not test on the live server during peak hours if you can avoid it; transcoding tests and load testing compete with real viewers.
- Keep the scope small. One hour of focused testing beats a vague "check everything" request.
- Write the report as you go. Reconstructing steps a week later wastes hours.
- Use the knowledge base and FAQ for setup questions before the audit, so the hours go to testing rather than configuration basics.
Conclusion
A QA audit is not about finding fault in your work; it is about replacing "it sometimes doesn't work" with a list you can fix in order. Build the playback matrix, inventory what you expose, run Lighthouse on the pages that matter, rehearse the update and the restore, and put everything in a short report with severities. If you would rather have someone independent do it, the scope and estimate are free, and the hours are yours to spend. Start at Quality assurance.