Reproducible Builds in June 2019

← View all our monthly reports


Welcome to the June 2019 report from the Reproducible Builds project! In our reports we outline the most important things that we have been up to over the past month.

In order that everyone knows what this is about, whilst anyone can inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries. The motivation behind the reproducible builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

In June’s report, we will cover:

  • Media coverage — Lego bricks, pizza and… Reproducible Builds‽
  • Upstream news — Is Trusting Trust close to a ‘rebuttal’?
  • Events — What happened at MiniDebConf Hamburg, the OpenWrt Summit, etc.
  • Software development — Patches patches patches, etc.
  • Misc news — From our mailing list…
  • Getting in touch… and how to contribute.

Media coverage

  • The Prototype Fund, an initiative to “aid software developers, hackers and creatives in furthering their ideas from concept to demo” produced a video featuring Holger Levsen explaining Reproducible Builds… using Lego bricks and pizza!

One key motivation for reproducible builds is to enable peak efficiency for the build caches used in modern build systems.

Upstream news


Events

There were a number of events that included or incorporated members of the Reproducible Builds community this month. If you know of any others, please do get in touch. In addition, a number of members of the Reproducible Builds project will be at DebConf 2019 in Curitiba, Brazil and will present on the status of their work.

MiniDebConf Hamburg 2019

Holger Levsen, Jelle van der Waa, kpcyrd and Alexander Couzens attended MiniDebConf Hamburg 2019 and worked on Reproducible Builds. As part of this, Holger gave a status update on the Project with a talk entitled Reproducible Builds aiming for bullseye, referring to the next Debian release name:


Jelle van der Waa kindly gifted Holger with a Reproducible Builds display:

In addition, Lukas Puehringer gave a talk titled Building reproducible builds into apt with in-toto:

As part of various hacking sessions:

  • Jelle van der Waa:

    • Improved the reproducible_json.py script to generate distribution-specific JSON, leading to the availability of an ArchLinux JSON file.
    • Investigated why the Arch Linux kernel package is not reproducible, finding out that KBUILD_BUILD_HOST and KGBUILD_BUILD_TIMESTAMP should be set. The enabling of CONFIG_MODULE_SIG_ALL causes the kernel modules to be signed with a (non-deterministic) build-time key if none is provided, leading to unreproducibility.
    • keyutils was fixed with respect to it embedding the build date in its binary. […]
    • nspr was made reproducible in Arch Linux. […]
  • kpcyrd:
    • Created various Jenkins jobs to generate Alpine build chroots, schedule new packages and to ultimately build them. […][…][…]
    • Created an Alpine reproducible testing overview page.
    • Provided a proof of concept SOURCE_DATE_EPOCH patch for abuild to fix timestamp issues in Alpine packages. […]
  • Alexander Couzens:
    • Rewrote the database interaction routines for OpenWrt.
    • Migrated the OpenWrt package parser to use Python 3.x as Python 2.x will be reaching end-of-life at the end of this year.
    • Setup a test environment using a new README.development file.

Holger Levsen was on-hand to review and merge all the above commits, providing support and insight into the codebase. He additionally split out a README.development from the regular, more-generic README file.

OpenWrt summit

The OpenWrt project is a Linux operating system targeting embedded devices, particularly wireless network routers. In June, they hosted a summit that took place from 10th to 12th of the month.

Here, Holger participated in the discussions regarding .buildinfo build-attestation documents. As a result of this, Paul Spooren (aparcar) made a pull request to introduce/create a feeds.buildinfo (etc) for reproducibility in OpenWrt.

Software development

buildinfo.debian.net

Chris Lamb spent significant time working on buildinfo.debian.net, his experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them. This included:

  • Started making the move to Python 3.x (and Django 2.x) […][…][…][…][…][…][…] additionally performing a large number of adjacent cleanups including dropping the authentication framework […], fixing a number of flake8 warnings […], adding a setup.cfg to silence some warnings […], moving to __str__ and str.format(...) over %-style interpolation and u"Unicode" strings […], etc.

  • Added a number of (as-yet unreleased…) features, including caching the expensive landing page queries. […]

  • Took the opportunity to start migrating the hosting from its current GitHub home to a more-centralised repository on salsa.debian.org, moving from the Travis to the GitLab continuous integration platform, updating the URL to the source in the footer […] and many other related changes […].

  • Applied the Black “uncompromising code formatter” to the codebase. […]

Project website

There was a significant amount of effort on our website this month.

  • Chris Lamb:

    • Moved the remaining site to the newer website design. This was a long-outstanding task (#2) and required a huge number of changes, including moving all the event and documentation pages to the new design […] and migrating/merging the old _layouts/page.html into the new design […] too. This could then allow for many cleanups including moving/deleting files into cleaner directories, dropping a bunch of example layouts […] and dropping the old “home” layout. […]

    • Added reports to the homepage. (#16)

    • Re-ordered and merged various top-level sections of the site to make the page easier to parse/navigate […][…] and updated the documentation for SOURCE_DATE_EPOCH to clarify that the alternative -r call to date(1) is for compatibility with BSD variants of UNIX […].

    • Made a large number of visual fixups, particularly to accommodate the principles of responsive web design. […][…][…][…][…]

    • Updated the lint functionality of the build system to check for URIs that are not using /foo/-style relative URLs. […]

  • Jelle van der Waa updated the Events page to correct invalid Markdown […] and fixed a typo of “distribution” on a previous event page […].

  • Thomas Vincent added a huge number of videos and slides to the Resources page […][…][…][…][…][…] etc. as well as added a button to link to subtitles […] and fixing a bug when displaying metadata links […].

In addition, Atharva Lele added the Buildroot embedded Linux project to the “Who’s involved” page. […]

Test framework

We operate a comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. The following changes were done in the last month:

  • Alexander Couzens (OpenWrt):
  • Holger Levsen:
    • Show Alpine-related jobs on the job health page. […]
    • Alpine needs the jq command-line JSON processor for the new scheduler. […]
    • Start a dedicated README.development file. […]
    • Add support for some nodes running Debian buster already. […]
  • Jelle van der Waa:
    • Change Arch Linux and Alpine BLACKLIST status to blacklist […] and GOOD to reproducible […] respectfully.
    • Add a Jenkins job to generate Arch Linux HTML pages. […]
    • Fix the Arch Linux suites in the reproducible.ini file. […]
    • Add an Arch JSON export Jenkins job. […]
    • Create per-distribution reproducible JSON files. […]

  • kpcyrd (Alpine):

    • Start adding an Alpine theme. […]
    • Add an Alpine website. […][…][…][…]
    • Add #alpine-reproducible to the KGB chat bot. […]
    • Use the apk version instead of vercmp. […]
    • Install/configure various parts of the chroot including passing in Git options […], adding the abuild group onto more servers […][…], installing GnuPG […]
    • Build packages using its own scheduler. […] […][…]
    • Misc maintenance and fixups. […][…]
  • Mattia Rizzolo:
    • Adjust the setup_pbuilder script to use [check-valid-until=no] instead of Acquire::Check-Valid-Until (re. (#926242)). […]

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Distribution work

In Debian, 39 reviews of packages were added, 3 were updated and 8 were removed this month, adding to our knowledge about identified issues.

Chris Lamb also did more work testing of the reproducibility status of Debian Installer images. In particular, he was working around and patching an issue stemming from us testing builds far into the “future”. (#926242)

In addition, following discussions at MiniDebConf Hamburg, Ivo De Decker reviewed the situation around Debian bug #869184 again (“dpkg: source uploads including _amd64.buildinfo cause problems”) and updated the bug with some recommendations for the next Debian release cycle.

Bernhard M. Wiedemann posted his monthly Reproducible Builds status update for the openSUSE distribution.

Other tools

In diffoscope (our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues) Chris Lamb documented that run_diffoscope should not be considered a stable API […] and adjusted the configuration to build our Docker image from the current Git checkout, not the Debian archive […]

Lastly, Chris Lamb added support for the clamping of tIME chunks in .png files […] to strip-nondeterminism, our tool to remove specific non-deterministic results from a completed build.

Misc news

On our mailing list this month Lars Wirzenius continued conversation regarding various questions about reproducible builds and their bearing on building a distributed continuous integration system which received many replies (thread index for May & June). In addition, Sebastian Huber asked whether anyone has attempted a reproducible build of a GCC compiler itself.

If you are interested in contributing the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:


This month’s report was written by Alexander Borkowski, Arnout Engelen, Bernhard M. Wiedemann, Chris Lamb, heinrich5991, Holger Levsen, Jelle van der Waa, kpcyrd & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.



← View all our monthly reports