The "Magic" of Custom ROMs: Prelude

I am going to start a blog series on porting custom ROMs for Android devices. This idea comes from recent chats with friends on the topic of choosing new phones to buy and custom ROMs -- that the whole notion of being able to compile an entire operating system from scratch and make it running on a given device seems pretty magic-like to many. Pretty much all of these friends of mine are fairly tech-savvy, and many of them are developers themselves and use Linux as their daily driver OS on their PCs. But when it comes to phones, somehow, running a customized OS begins to sound like magic to them, and most of them just sit around and wait for someone to port a custom ROM to the phone they are using, or alternatively, they check if there is already somebody working on a port before deciding to switch to a new phone.

To be clear, I am not expecting everybody to learn to make everything on their own. Frankly, working with something like a custom ROM has never been something too enjoyable except for people that like to stare at a terminal window rolling for hours and exclaim when the phone even just lights up the screen with a boot animation, to say the least. However, it seemed to me that people do not refrain from trying just because they do not like the process. Instead, most of them stop trying long before they even get a glance into the actual process itself. They stop because there is not enough guidance on where to begin and how to start their first step into the rabbit hole of custom ROM development.

Now, there do exist quite a few documentations and guides on porting custom ROMs, on some forums like xda-developers or on wiki pages of custom ROMs like LineageOS. But they either only describe a process that worked for one instance with no actual expalanation and insights on the rationale behind each step, or just being too vague to actually guide anybody in the first place. Here is a quote from a LineageOS wiki article

Ok, so if nobody is making headway on your device, where do you go from here? Consider that the majority of the device maintainers for LineageOS have significantly different day jobs than Android device maintenance (or even programming, for that matter). If you are passionate enough about getting LineageOS up and running on your device, you can make it happen. Start easy; buy an old, but well supported device and try compiling your first ROM. Once you’re running software you compiled yourself, start investigating the device configuration files. Tweak, and then tweak some more. Eventually, see what you’re able to accomplish on your device of interest!

...yeah, it is not great.

I am not going to quote any guides on XDA like this. Although they are more detailed than this one from LineageOS, they seldom work except for one or two devices and, as previously mentioned, do not actually tell you the rationale behind the steps. If you do not have the exact same environment and device as used in those guides, it is almost certain that you will run into some issue not covered by them at all. I have seen people following those guides and being turned away by an error early on which can be fixed easily if the rationales were actually explained instead of simply listing commands for people to copy and run.

The custom ROM community is, for the least, more obscure than many of other FOSS communities I've interacted with, not in the way that the source code is not visible (otherwise it cannot be called FOSS), but, hopefully evident from what I just described, the fact that there is such a lack of either guidance or documentation for people to get started. Android Open Source Project (AOSP) does have an official documentation page, but that is more for OEMs instead of FOSS developers, and even that is probably not quite complete if you compare it with what OEMs actually get from Google. This can not only hurt the community in the sense that less developers are available, but also by the possibility of creating and enabling some so-called buildbots or snake-oil projects to survive -- since there is no easy way for an educated user to quickly check for information. These questionable developers are normally those that managed to get a build working (probably by following those guides with steps but without explanation) but failed to obtain more information to start actual development. Still desiring the "status" of a reputable developer, they resort to making bold, probably questionable claims and pretend to know what they are actually doing. Back to the users' side, if the only source of information is what developers claim, then listening to a "reputable" developer does not really feel that different from listening to a random developer that happens to make bold claims. I have been involved in one of such dramas, and while I totally regret taking part in one and mocking these uninformed developers, it did make me realize how many more people could have become actual Android ROM developers if more information was available in an easy-to-acquire manner.

In addition, custom ROMs are not ported to new devices based on the number of requests (as LineageOS Wiki states); instead, it's only devices with developers capable of porting custom ROMs that get supported. Therefore, more developers also means potentially more supported devices, bringing in more users, which in turn increases publicity of custom ROM projects. The higher the publicity is, the harder it will be for Google and mobile phone vendors to screw us up, which has been happening since several years ago with the introduction of SafetyNet and the proprietarization of many formerly open-source Android components, for example, the Dialer and SMS apps. To me, it seems particularly important that the FOSS Android ROM community should have more developers to combat the recent tendency of proprietarization of the once open-source Android ecosystem, yet it is not capable of doing so because of all the aforementioned obscurity.

To make it absoluetly clear, being obscure is most certainly not intentional, either by Google or by the community. The whole process of porting an open-soruce Android ROM to a device does include many, many difficulties and differs significantly depending on what sort of device you are working on. The Android Open Source Project documentation was intended for OEMs because that's what it was desigend for from the very beginning: for people or organizations that have at least some control over the hardware, for example, the partition layout, or the trusted execution environment (TEE). But, obviously, we, as average comsumers, do not. That the Android custom ROM community thrives on almost blackbox hardware nobody in the community knows any implementation details of is already incredible, especially before there was a thing called Project Treble that significantly reduced the workload of such development.

I know I would not be able to change the situation very much. After all, I am just yet another Android ROM developer that shares a lot of problems everybody has. I do not have time either to answer every question about ROM development just like everybody else. Neither am I a very good writer that writes easily comprehensible articles. Heck, I am not even an English native speaker. But I want to at least make an attempt to do something different. I want to share part of my experience as an Android custom ROM developer for all these years, not just a series of steps that even a sophiscated script may be able to execute. I would like to paint a general picture of what is "porting", what do we do when we say "porting", and, most importantly, how to figure out solutions to problems on your own when trying to do so -- even StackOverflow is seldom useful when attempting to port an Android ROM.

So, here I start a new blog series. I am not sure yet what it will be, but let us just try and see.