Please know that this is advanced stuff – if you’ve never compiled a kernel before, you should follow the guide “How to Build a Custom Kernel” linked above, and this guide will involve cherry-picking and merging commits from the latest Linux-stable kernel with your Android kernel before you compile it. Upstreaming your Android kernel to the latest Linux stable has a lot of positive benefits, such as being up-to-date with the latest security commits and bugfixes – we’ll explain some of the pros and cons later in this guide.
What is Linux-Stable kernel?
Linux-stable as the name implies is the stable arm of the Linux kernel. The other arm is known as “mainline”, which is the master branch. All of the Linux kernel development happens in the mainline, and generally follows this process:
What are LTS kernels?
Every year, Greg will pick one kernel and maintain it for either two years (LTS) or six years (extended LTS). These are designed to have products that need stability (like Android phones or other IOT devices). The process is the exact same as above, it just happens for a longer time. There are currently six LTS kernels (which can always be viewed on the kernel.org releases page):
What are the benefits of upstreaming my Android kernel to Linux Stable?
When important vulnerabilities are disclosed/fixed, the stable kernels are the first ones to get them. Thus, your Android kernel will be a lot safer against attacks, security flaws, and just bugs in general.
The Linux stable includes fixes for a lot of drivers my Android device doesn’t use, isn’t this mostly unnecessary?
Yes and no, depending on how you define “mostly”. The Linux kernel may include a lot of code that goes unused in the Android system, but that doesn’t guarantee there will be no conflicts from those files when merging new versions! Understand that virtually nobody builds every single part of the kernel, not even the most common Linux distros like Ubuntu or Mint. This doesn’t mean you shouldn’t be taking these fixes because there ARE fixes for drivers you DO run. Take arm/arm64 and ext4 for example, which are the most common Android architecture and file system respectively. In 4.4, from 4.4.78 (version of the latest Oreo CAF tag) to 4.4.121 (latest upstream tag), these are the following numbers for the commits of those systems: The most time-consuming part is the initial bring up; once you are all the way up to date, it takes no time at all to merge in a new release, which usually contains no more than 100 commits. The benefits that this brings (more stability and better security for your users) should necessitate this process though.
How to Merge Linux Stable Kernel into an Android Kernel
First you need to figure out what kernel version your Android device is running. As trivial as this seems, it is necessary to know where you need to begin. Run the following command in your kernel tree: It will return back the version you’re on. The first two numbers will be used to figure out the branch you need (e.g. linux-4.4.y for any 4.4 kernel) and the last number will be used to determine what version you need to start with merging (e.g. if you are on 4.4.21, you’ll merge 4.4.22 next).
Grab the latest kernel source from kernel.org
kernel.org houses the latest kernel source in the linux-stable repository. At the bottom of that page, there will be three fetch links. In my experience, Google’s mirror tends to be the fastest but your results may vary. Run the following commands:
Decide if you want to merge the entire kernel or cherry-pick the commits
Next, you will need to choose if you want to merge the commits or cherry-pick. Here’s the pros and cons of each and when you may want to do them. NOTE: If your kernel source is in the form of a tarball, you will most likely need to cherry-pick, otherwise you will get thousands of file conflicts because git is populating the history based purely on upstream, not what the OEM or CAF has changed. Just skip to step 4.
Cherry-picking:
Pros:
Easier to resolve conflicts as you know exactly what conflict is causing an issue.Easier to rebase as each commit is on its own.Easier to bisect if running into issues
Cons:
It takes longer as each commit has to be individually picked.Little more difficult to tell if commit is from upstream on first glance
Merge
Pros:
It’s faster as you do not have to wait for all of the clean patches to merge.It’s easier to see when a commit is from upstream as you will not be the committer, the upstream maintainer will be.
Cons:
Resolving conflicts can be a bit more difficult as you will need to look up which commit is causing the conflict using git log/git blame, it will not directly tell you.Rebasing is difficult as you cannot rebase a merge, it will offer to cherry-pick all of the commit individually. However, you should not be rebasing often, instead using git revert and git merge where possible.
I would recommend doing a cherry-pick to figure out any problem conflicts initially, doing a merge, then revert the problem commits afterwards so updating is easier (as merging is quicker after being up to date).
Add the commits to your source, one version at a time
The most important part of this process is the one version at a time part. There MAY be a problem patch in your upstream series, which could cause a problem with booting or break something like sound or charging (explained in the tips and tricks section). Doing incremental version changes is important for this reason, it’s easier to find an issue in 50 commits than upwards of 2000 commits for some versions. I would only recommend doing a full merge once you know all of the problem commits and conflict resolutions.
Cherry-picking
Format: Example: git cherry-pick v3.10.73..v3.10.74
Merge
Format: Example: git merge v3.10.74 I recommend keeping track of the conflicts in merge commits by removing the # markers.
How to Resolve Conflicts
We can’t give a step-by-step guide for resolving every single conflict, as it involves a good knowledge of C language, but here’s a few hints. If you are merging, figure out what commit is causing the conflict. You can do this one of two ways:
Figure out if you already have the commit. Some vendors like Google or CAF will attempt to look upstream for critical bugs, like the Dirty COW fix, and their backports could conflict with upstream’s. You can run git log –grep=”<part_of_commit_message>” and see if it returns anything. If it does, you can skip the commit (if cherry-picking using git reset –hard && git cherry-pick –continue) or ignore the conflicts (remove the ««« and everything between the ====== and »»»).Figure out if there has been a backport that is messing up resolution. Google and CAF like to backport certain patches that stable wouldn’t. Stable will often need to adapt the resolution of the mainline commit to the abscence of certain patches that Google opts to backport. You can look at the mainline commit by running git show
Otherwise, it may just be a result of a CAF/Google/OEM addition, in which case you just need to shuffle some things around. Here is a mirror of the linux-stable kernel.org repository on GitHub, which can be easier for looking up commit lists and diffs for conflict resolution. I recommend going to the commit list view first and locating the problem commit to see the original diff to compare it to yours. Example URL: https://github.com/nathanchance/linux-stable/commits/linux-3.10.y/arch/arm64/mm/mmu.c You can also do it via the command line: Solving resolutions is all about context. What you should ALWAYS do is make sure your final diff matches upstream’s by running the following commands in two separate windows:
Enable rerere
Git has a feature called rerere (stands for Reuse Recorded Resolution), meaning that when it detects a conflict, it will record how you resolved it so you can reuse it later. This is especially helpful for both chronic rebasers with both merging and cherry-picking as you will just need to run git add . && git <merge|cherry-pick> –continue when redoing the upstream bringup as the conflict will be resolved how you previously resolved it. It can be enabled by running the following command in your kernel repo:
How to git bisect when running into a compiler or runtime error
Given that you will be adding a sizable number of commits, it’s very possible for you to introduce a compiler or runtime error. Instead of just giving up, you can use git’s built-in bisect tool to figure out the root cause of the issue! Ideally, you will be building and flashing every single kernel version as you add it so bisecting will take less time if needed but you can bisect 5000 commits without any issues. What git bisect will do is take a range of commits, from where the issue is present to where it wasn’t present, and then start halving the commit range, allowing you to build and test and let it know if it is good or not. It will continue this until it spits out the commit causing your issue. At that point, you can either fix it or revert it. NOTE: Mergers will need to temporarily run git rebase -i <good_sha> to apply all the patches to your branch for proper bisecting, as bisecting with the merges in place will often times checkout onto the upstream commits, meaning you have none of the Android specific commits. I can go into more depth on this upon request but trust me, it is needed. Once you have identified the problem commit, you can revert or rebase it into the merge.
Do NOT squash upstream updates
A lot of new developers are tempted to do this as it is “cleaner” and “easier” to manage. This is terrible for a few reasons:
Authorship is lost. It’s unfair to other developers to have their credit stipped for their work.Bisecting is impossible. If you squash a series of commits and something is an issue in that series, it’s impossible to tell what commit caused an issue in a squash.Future cherry-picks are harder. If you need to rebase with a squashed series, it is difficult/impossible to tell where an conflict results from.
Subscribe to the Linux Kernel mailing list for timely updates
In order to get notified whenever there is an upstream update, subscribe to the linux-kernel-announce list. This will allow you to get an email every time a new kernel is released so you can update and push as quick as possible.
Linux Mint 20 “Ulyana” An All 64-Bit Linux OS Based On Ubuntu 20.04 Stable…Dirty Pipe: The Latest Serious Linux Kernel Vulnerability is Being PatchedHow to Learn More About the Linux Kernel from Your Own MachineLatest Stable Chrome Web Browser Version 86 Brings Security Enhancements And…
