Linux kernel backports

https://git.kernel.org/pub/scm/linux/kernel/git/backports/backports.git

https://git.kernel.org/pub/scm/linux/kernel/git/backports/backports.git

All changes go into the git repository. We track the changes, who made the changes, when the changes were made. Other than this we also track who merged the change. We have two key maintainers:

Hauke Mehrtens Luis R. Rodriguez

The repository is shared on kernel.org between both of these maintainers, they coordinate via IRC and e-mail about when one can / should merge changes.

The project mirrors Linux's own release mechanisms. The project relies on linux-next to generate snapshot releases based on the latest development efforts both on Linux and on the Linux backports project, these are the linux-next based snapshots. Once Linus releases a kernel as a release candidate the backports project follows with a respective release candidate for it. Once Linus blesses a final release a respective backports project follows through with a respective final release.

During the development phase of Linux we have daily linux-next based snapshots, these are dated, the latest one being based on linux-next tag next-20160122, and so the respective backports release is:

https://www.kernel.org/pub/linux/kernel/projects/backports/2016/01/22/backports-20160122.tar.gz

Packaged releases are also made to mirror Linux stable releases, so for instance we have the v3.10-rc1 release:

https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.10-rc1/

Then the v3.10 final release:

https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.10/

After that we have follow up fixes to upstream Linux, and then a respective backports release for it, there were 3 extra version releases for v3.10:

https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.10.4/ https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.10.17/ https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.10.19/

If the need ever arises to make a backports release update between the same snapshot of Linux a postfix digit is used to annotate this. For instance, we had two v3.8-rc2 releases, -1 and -2:

https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.8-rc2/compat-drivers-3.8-rc2-1.tar.bz2 https://www.kernel.org/pub/linux/kernel/projects/backports/stable/v3.8-rc2/compat-drivers-3.8-rc2-2.tar.bz2

Git is used:

https://git.kernel.org/cgit/linux/kernel/git/backports/backports.git/

The release tags for backports mirror the Linux kernel release scheme. When releases are made based on linux-next, the respective tag is used but instead of "next", "backports" is used. So for instance, a release based on linux-next next-20160122 would have a tag "backports-20160122".

For stable releases the same Linux versioning scheme is followed, starting with release candidates (v3.10-rc1-1, v3.10-rc1-2), to final version (v3.10-1, v3.10-2), to extra version stable updates (v3.10.4-1, v3.10.19-1):

v3.10-1 v3.10-2 v3.10-rc1-1 v3.10-rc1-2 v3.10.17-1 v3.10.17-2 v3.10.19-1 v3.10.4-1

Since this project builds on Linux, we needed a way to distinguish between backport specific releases. This is accounted for the extra dash followed by the release number, so v3.10-1, v3.10-2, both -1 and -2 are backport specific annotations.

Here's an example release using tag backports-20160122

https://git.kernel.org/cgit/linux/kernel/git/backports/backports.git/tag/?h=backports-20160122

Likewise, here's a stable release using tag v4.4.2-1:

https://git.kernel.org/cgit/linux/kernel/git/backports/backports.git/tag/?h=v4.4.2-1

Since the backports project mirrors Linux the ChangeLog of Linux is referred to for changes, backport specific updates are too many to list, but major updates are referenced when a release is made, and the respective project README is updated to account for subsystems that are backported. This is available here:

https://git.kernel.org/cgit/linux/kernel/git/backports/backports.git/tree/README

When a new device driver is added it is mentioned only in the commit logs, we could do a better job at not only annotating subsystems but also device drivers, this becomes tricky however as we backport over 700 device drivers now. We will discuss what things we can do better to improve on this area.

The backports project builds on top of Linux, the amount of backports specific code accounts for about 1%-2% of the code used. Although there is a risk of a security issue within backports, the major attack surface consists of non-backports related code: device drivers, or libraries carried over. Security fixes from Linux are carried over, we do not explicitly mirror the changelog of Linux, instead refer people to Linux' own changelog for security fixes on Linux.

This is documented here:

https://backports.wiki.kernel.org/index.php/Documentation/reporting-bugs https://backports.wiki.kernel.org/index.php/Bugzilla

The "Backports project" section of bugzilla is used.

Example of latest fixes:

https://bugzilla.kernel.org/show_bug.cgi?id=105921 https://bugzilla.kernel.org/show_bug.cgi?id=71501 https://bugzilla.kernel.org/show_bug.cgi?id=66601 https://bugzilla.kernel.org/show_bug.cgi?id=65881

Sadly most reports are bogus. Most issues are actually reported on the mailing list and addressed there. There is no metric to address the number of issues mentioned on the mailing list and fixed there. We however do have an archive.

The mailing list is used for this and requests to add new drivers considered and discussed.

Bugzilla is used:

https://backports.wiki.kernel.org/index.php/Bugzilla

The project also has mailing list and that is archivedi

https://backports.wiki.kernel.org/index.php/Mailing_list

https://backports.wiki.kernel.org/index.php/Reporting-vulnerabilities

This explains the process to follow to report vulnerabilities.

It is requested private vulnerabilities are reported via e-mail directly to the Linux backports maintainers. This is documented here:

https://backports.wiki.kernel.org/index.php/Reporting-vulnerabilities

We have not had any vulnerabilities reported.

Since Linux backports mirrors Linux, the build testing / automatic builds are borrowed, so linux-next daily builds are done by IBM, likewise, Intel's 0-day test infrastructure . There is no home page to linux-next daily builds tests, but its known that this merges all development trees and there is a compile test done daily. 0-day's home page:

https://01.org/lkp

Other than this currently backports build tests are done prior to each release, since all supported kernels need to be tested against, currently 24 kernels need to be compile tested against, compile testing with all features enabled takes a long time. HP, Linux Foundation, and SUSE donated a server to help with this, we grant access to key developers to help build test prior to making a release, or when incorporating a patch to test the build.

We will work on automating builds though this may be a bit complex given the overhead.

All required tools are free and open source software. The same tools to build Linux are the same tools to build backports Linux releases, with the addition of Coccinelle needed to generate packages for users. Only developers need coccinelle. Needed tools to build Linux backports packages:

make gcc python perl

When using integration the kernel is built as you typically would otherwise build the Linux kernel.

All required tools are the same to build Linux, its obvious Linux can only be built by open source tools.

Intel uses Linux backports for their own builds / release for users of 802.11 wifi. They have a series of run time tests against the 802.11 Intel wifi driver. This test the 802.11 subsystem and Intel's specific device driver against specific kernels.

Its not okay to not have a generic run time automation test suite, this however is a bit complex though as it would mean testing at run time Linux backports builds on each supported 24 kernels for each release. There is also the issue of needing emulated devices to load and test drivers. This is a major undertaking. Some form of automating run time testing is desirable. As it stands we rely on independent stakeholders to do their own run time testing of supported device drivers. The rest of the drivers are supported as best effort.

The 0-day bot does testing of every commit before it is merged into the main repository.

Since Linux kernel code is used we match the same practice and use the same tools to help look for issues. The same build flags are also used. The kernel build process enables a lot of warning flags, and it also provides the tool 'sparse' to check for many other coding problems. And a full test suite of coccineele scripts is integrated into the kernel source tree itself.

Where ever practical, all warnings are fixed.