The first fully tested Fedora Firefox package

Mozilla testsuite running natively on Wayland. There is still a room for improvement

We hit a big milestone in Firefox deployment on Fedora with firefox-84.0.2 package. It’s the first fully tested Firefox package released to Fedora users. Let’s see what’s so exciting on it.

Mozilla has a large testsuite as a part of development and release process. When any new patch hits Firefox repository, it’s built and tested for functional and speed regressions. The testsuite is also a developers nightmare as it contains some old and outdated test environments and it may be difficult to pass patches through it.

So if you download a Mozilla Firefox binary you can be sure it’s generally working at least on X.org and Ubuntu 18.04 which is main test setup there. But what about distro builds?

There are may differences how Firefox is built by various distributions. The very first difference is used compiler. Mozilla compiles by custom patched Clang which is not available for distributions although they actually don’t want it.

GCC is the favorite compiler among Linux hackers and that brings extra maintenance burden to Firefox distro maintainers. Compiler crashes and code miscompilation are companions of every new GCC version (no to mention an extra fun with LTO).

Distros also tend to modify Firefox sources with various integration patches or patches from Nightly, run Firefox on Wayland backend or use system NSS so maintaining such package is like dancing in a minefield.

And there comes the Mozilla testsuite. If we manage to run it in our environment with our package modifications it greatly helps to reduce unwelcome surprises like Bug 1893474. And it needs to run in automated manner and test every Firefox build we may release to user on all arches.

And that’s done now. Almost. The latest Firefox Fedora builds (F32, F33) run the testsuite as a part of the build process and test results are generated as a new package (firefox-testresults).

If I say we’re almost there, I mean the most difficult work is done as the tests are run in Koji. There are still some failing tests which needs to be inspected and disabled or fixed. The testsuite needs to be run on Wayland – it checks X11 backend now. And it should be integrated to Fedora test infrastructure to show test results in some user-friendly way.