icculus / fatelf Goto Github PK

I'd like to suggest that as part of the embedded record header, it would be nice to also be able to specify the shared libc used. VERSION 1 FORMAT doesn't seem to allow that right now. This could be achieved by following the initial 16bit int and padded two bytes by for example an int indicating the libc .so file name length, and then the actual .so file name, preceding still the actual embedded ELF binary. Alternately, the spec could be changed such that having two identical records is allowed if they both link a different shared libc, although it would then be considerably harder for the OS loader to figure out which one to pick. This would be very useful to cover scenarios like Alpine Linux where one would likely want to build for shared musl libc, vs mainstream Linux distros where one would likely want to build for shared glibc.

(To my understanding, static musl libc is unusable in most cases for graphical applications and therefore not really an alternate solution here. Also, from what I've gathered the only musl libc compatibility layer that exists for running shared glibc binaries without setting up an entire container environment like flatpak or docker seems to be pretty hacky. Like, in a way where it introduces additional race conditions and attack vectors for the programs run, so not really what one would want to rely on instead. Don't quote me on that though, only what my impression of it was.)

Hello @icculus (and anybody else reading this),

It was disappointing how your initial (well though-out) FatELF proposal didn't gain any traction at the time and kind of withered on the vine.

The frustrating thing is that Apple has already proven this approach to be the best one for smooth transitions between architectures from the perspective of the customers who just want to continue to run their software. Apple applied this concept three times, in the form of the "Fat Binary" format when they migrated from m68k to PowerPC, as a well as what they later called "Universal Apps" when they moved from PowerPC to x86, and then again during their currently still ongoing transition to Apple Silicon (ARM64). (It's even four times, when you also count their transition from 32-bit x86 to x86_64!) This is a well-proven concept now, and it's kind of crazy that Microsoft hasn't developed something similar in their PE executable format.

Going back to the lack of Linux adoption at the time: perhaps the time just wasn't right for it yet. Back in 2009, the x86_64 architecture reigned supreme, and it provided native backwards compatibility with 32-bit x86 (IA-32) code, negating the need for something like FatELF. The iPhone had been on the market for just 2 years. It was just 1 year after Android was introduced. Mobile gaming, although existent, was a much smaller slice of the market.

Obviously, a lot has changed over the last 14 years. With the rapid emergence of both the ARM64/AArch64 and RISC-V architectures lately, perhaps the time is now right to dust off this idea and resubmit FatELF for consideration.

I believe that the backing of at least one prominent organization in the Linux space would be enough for it to be widely adopted by the Linux kernel developers as well as the major distros. Two notable companies that I believe would be interested in championing FatELF would be the Raspberry Pi Foundation and Valve.

The Raspberry Pi Foundation has been very careful to maintain downwards compatibility in Raspberry Pi releases, going back to the original Raspberry Pi that had a very outdated ARMv6 CPU. They now have to maintain separate 32-bit and 64-bit versions of Raspberry Pi OS. Having the bulk of the .deb files built in a FatELF binary format with support for both 32-bit and 64-bit ARM would alleviate things for them considerably, I would reckon. Also, given that they are a Strategic Member of the RISC-V Foundation, they must already be thinking about how to manage a possible transition from ARM to RISC-V at some point in the future, or at least how to maintain Raspberry PI OS for two completely different and mutually incompatible instruction set architectures.

Then there is Valve, which has had a lot of success recently on the hardware front in the form of the Steam Deck. Granted, that portable game console is built around an AMD64/x86_64 SoC for out-of-the-box compatibility with the vast existing library of x86 games, but given the rising popularity of mobile phones as a gaming platform, Apple moving to ARM, and a continuing consumer demand for ever longer battery life and the best possible price/performance ratio, I cannot image Valve not thinking about wanting to hedge their bets between processor architectures in the long term. The x86 architecture will not be dominant forever.

And perhaps Google would also like to give FatELF another look for use in Android phones and Chromebooks, especially since they recently started added RISC-V support to the Android code base. With FatELF, the number of compatible apps would likely grow a lot quicker.

Once development toolchains get mature support for FatELF, publishers of games and other software will be able to provide support for multiple CPU architectures practically for free. Basically, all they would have to do is add runners with different CPU architectures to their CI/CD pipelines. QA engineers (presumably) already spin up the test builds on a wide range of different hardware, so they might as well include some systems with diverse architectures, just to double-check that everything is functional.

I've also read about how the AppImage developers would very much like to see FatELF being embraced and adopted the Linux ecosystem, so multi-arch AppImages would become feasible.

Icculus, I know you may personally have been burned out by the earlier rejection of your FatELF proposal, especially after you had put so much effort into it earlier. But as I said, a lot has changed over the last decade, and the circumstances today are quite different and, I believe, much more amenable to an idea like this these days.

I believe it might be a good time to resurrect the FatELF idea. At the very least, why not ask around the community, raise some buzz over it and see what happens? I'd be happy to help out with that. 🙂