Chrisoft

It is not a secret that Google hates Linux distro packagers when it comes to Chromium. Copious amount of ink has been spilled over this topic, from back when the Google API debacle happened a few years ago like this one, to whatever this mess we have been experiencing ever since summed up nicely in this post. The patchset is brilliantly named, by the way.

Like many other distros, Gentoo's chromium ebuilds have experienced loads of challenges in the past few years. What made this worse is that Gentoo's chromium packaging matters are essentially an one-man show this entire time. It has changed hands, but this largely remained true. Whenever the sole maintainer goes on a vacation, falls sick, or otherwise does not have enough spare time for Gentoo matters, the ebuilds would become stale, since very few other people has the arcane knowledge of taming this wild beast.

I started maintaining my fork of the official chromium ebuilds towards the end of 2018, for this exact reason. The official ebuilds were out of date by multiple weeks, and those versions were hideously buggy. So I decided to take on the problem myself. How hard can it be besides bumping a bunch of version numbers?

It turned out, the answer to that was "a lot". Armed with virtually no knowledge of the Chromium code base or ebuild development whatsoever, it took me over a day of non-stop fiddling and multiple tries to get things working (mind you that back then everything was done on a terrible ULV dual-core i7, so even when I was asleep, my laptop was shitting its bed). A tiny bonus I got out of the deal was that since I was already applying user patches to Chromium at the time (most notably the vaapi patch from Debian), I was able to incorporate them into the ebuild. Sure I may have an entire ebuild to maintain now, but at the very least I don't have to use user patches any more!

I got so excited when this worked out that I tweeted about it. Sure I was doing nearly everything wrong: I started by dumping everything into /usr/local/portage without even setting up a git repository, and running emerge until it succeeded. I moved everything into git roughly a year later, but didn't learn the proper ebuild development workflow UNTIL LAST YEAR. Yes I didn't know ebuild is a command until 7 years into maintaining my own ebuild files, and that is the reason why the git history is littered with stupid one-line changes.

Now that's enough background information. What's the deal this time?

I will admit something I did wrong up front. I probably shouldn't have tried doing two things at the same time. I shouldn't have tried building crubit and moving to the "lite" source code tarballs at once. You will understand what this means as I explain on.

Google has been pushing crubit within the chromium code base. It seems like the crubit binary shipped with the official tarball requires the bundled rust toolchain to function, so it simply wouldn't work with my existing packaging script. Unlike the official ebuilds, mine never gained the ability to use the bundled toolchain as my ebuilds hard diverged from the official ones in 2024, soon after a new maintainer took over. So I had to somehow work around this. Fortunately for me, up until M153, this was relatively easy. Just set a few gn variables and I'd be all set.

But somewhere in M154 things changed. If I rip out crubit, gn dependencies for something called "cbor" would break. If I rip out that one too (or rather disabling its rust components) something buried deeply inside the browser UI code would break. I was running out of time and I figured I might as well try to somehow make crubit work. In that process, I discovered a few prerequisites to make crubit work. But I truly ran out of time at that point and decided I'm just going to give in and use the bundled rust toolchain for this version. So I hastily patched that into my ebuild and I was done with it for that week.

Another week rolls around and the dev branch is now at M155. I decided to try the crubit thing again without using the bundled toolchain. And here is where luck started going against me: when I tried to download the latest official tarball for the dev branch (155.0.8048.0), it 404'ed. I took a peek at Google's publish tarball CI job, and it was stuck in attempting to publish for a few ancient versions from 2022, as well as bizarre, non-existent versions like 137.0.7141.1450312. Reminded that the official ebuilds have switched to a much smaller "lite" version of the tarball generated by a script that I can also run myself, I decided to give that a go. "How bad can it be," I thought to myself, "it's not like I'm going to use the bundled toolchains anyway."

By this time you have probably realized that whenever the main character says "how bad can it be", it usually goes really badly for them. I ran into some insanity within the third_party directory, most of which I've never seen when I was using the full official tarball. The lite tarballs must have some other critical differences from the full ones that I do not know of. At this point it was 2 in the morning, so I decided to retreat to the version that has an official tarball (155.0.8040.2) and proceed with the crubit stuff.

To make crubit work, I found the following requirements all have to be met:

• //build/rust/std/rules/BUILD.gn must be regenerated for the precise rust toolchain version used to build chromium. This process uses crubit and something called "gnrt", whose source code is included in the chromium tarball at //tools/crates/gnrt and must be built beforehand.
• Crubit itself, being a rustc plugin, must be built against the precise rust toolchain version used to build chromium too.
• Crubit also only supports very recent rustc versions (nightlies and two recent stable releases).
• The exact revision of crubit listed in the file //tools/rust/update_rust.py must be used. Otherwise expect trouble.
• When using crubit to generate gn files for the rust standard library with the command provided in the official documentation, it loads the rust standard library source code from third_party/rust-src. However in the building process, the rust standard library source code inside third_party/rust-toolchain/lib/rustlib/src/rust will be used instead. Their content must match.
• And none of this will work at all if I tell the Chromium build system "OK here is my system rust toolchain. Please use this instead of the bundled one." The support simply isn't there. Instead what I had to do is to replace all the rust stuff shipped in the third_party directory with files from the system toolchain to trick the build system into thinking I'm still using the bundled ones. The entirety of everything above basically boils down to performing the full "roll rust" routine, but to a version matching the system toolchain.
• To make things less tedious, use a rust toolchain that supports unstable features (-Z and such). I use the latest stable release but built with rust.channel=nightly for this and that has been working fine for me. If you decide to try the rustc_nightly_capability gn variable instead, all I can say is good luck.

And don't forget ebuilds don't have access to the network, so all dependencies of crubit and gnrt must be declared earlier and fetched like other Gentoo rust packages. But all this must have been completed before the src_configure phase so half the facilities provided by cargo.eclass wouldn't work and everything just got even more complicated for no good reason.

After all this nonsense, the build did end up succeeding. But I lost an entire night of sleep over it, which is absolutely not worth it. Each failure that's glossed over in this post took ages to manifest, sometimes over an hour. Relevant documentation was long-winded, likely written with the assistance of an LLM and is probably not intended to be read by a human but rather an "AI agent". This shit almost broke me. I can now see how maintaining a distro package for chromium is easily a full-time job that nobody wants to pay for. I might as well follow the steps of countless former maintainers of this abomination and just quit. Mind you that I never even attempted to upstream my ebuilds because I can't test them on any setup beyond my own. I simply don't have the time to do so. This is despite the fact that I now own a 48-core workstation where a full build doesn't take over 10 hours to complete unlike when I first started.

This is just pure misery.

Fuck you, Google.