Reproducible Builds: Weekly report #20

What happened in the reproducible builds effort this week:

Media coverage

Motherboard published an article on the project inspired by the talk at the Chaos Communication 15. Journalists sadly rarely pick their headlines. The sensationalist “How Debian Is Trying to Shut Down the CIA” got started a few rants here and there. One from OpenBSD developper Ted Unangst lead to a good email contact and some thorough comments.

Toolchain fixes

• Emmanuel Bourg uploaded maven-ant-helper/7.11 which improved the reproducibility of the Javadoc by removing the timestamps and using the English locale.
• Thomas Schmitt uploaded libisoburn/1.4.0-2 which adds to the ISO image creator xorriso new flags for -alter_date to avoid update ctimes. Report by Daniel Kahn Gillmor.
• Florian Schlichting uplodaded libmodule-build-perl/0.421400-2 which makes linked file ordering deterministic. Original patch by Niko Tyni.

The modified version of gettext has been removed from the experimental toolchain. Fixing individual package seems a better approach for now.

Chris Lamb sent two patches for abi-compliance- checker: one to drop the timestamp from generated HTML reports and another to make umask and timestamps deterministic in the abi tarball.

Bugs submitted by Dhole lead to a discussion on the best way to adapt pod2man now that we have SOURCE_DATE_EPOCH specified. There is really a whole class of issues that are currently undiscovered waiting for tests running on a different date. This is likely to should happen soon.

Chris Lamb uploaded a new version of debhelper in the “reproducible” repository, cherry-picking a fix for interactions between ddebs and udebs.

Packages fixed

The following packages became reproducible due to changes in their build dependencies: aspic, django- guardian, erlang- sqlite3, etcd, libnative-platform- java, mingw- ocaml, nose2, oar, obexftp, py3cairo, python- dugong, python- secretstorage, python- setuptools, qct, qdox, recutils, s3ql, wine.

The following packages became reproducible after getting fixed:

• bochs/2.6-4 by Santiago Vila.
• codec2/0.4-3 by A. Maitland Bottoms.
• coquelicot/0.9.4-1 by Lunar.
• criticalmass/1:1.0.0-3 by Santiago Vila.
• ekg/1:1.9~pre+r2855-3 by Santiago Vila.
• eterm/0.9.6-3 by Santiago Vila.
• fbi/2.10-2 by Moritz Muehlenhoff.
• fsvs/1.2.6-2 by Santiago Vila.
• glhack/1.2-3 by Santiago Vila.
• httraqt/1.4.6-2 by Anton Gladky.
• libapache-authznetldap-perl/0.07-6 by gregor herrmann, original patch by Dhole.
• libkinosearch1-perl/1.01-3 uploaded by Florian Schlichting, original patch by Niko Tyni.
• liblucy-perl/0.3.3-6 uploaded by Florian Schlichting, original patch by Niko Tyni.
• slony1-2/2.2.4-1 by Christoph Berg.
• slrn/1.0.2-3 uploaded by Moritz Muehlenhoff, original patch by Dmitry Bogatov.
• svtplay-dl/0.10.2015.08.24-1 uploaded by Olof Johansson, fixed upstream.
• swh-plugins/0.4.15+1-8 uploaded by Jaromír Mikeš, original patch by Chris Lamb.
• sysstat/11.1.6-2 uploaded by Robert Luberda, original patch by Chris Lamb.
• uhd/3.9.0-3 by A. Maitland Bottoms.
• volk/1.1-3 by A. Maitland Bottoms.
• yadifa/2.1.3-2 uploaded by Markus Schade, original patch by Santiago Vila.

Some uploads fixed some reproducibility issues but not all of them:

• dict-jargon/4.4.7-3 uploaded by Ruben Molina, original patch by Dhole.
• ferret-vis/6.9.3-3 uploaded by Alastair McKinstry, original patch by Chris Lamb.

Patches submitted which have not made their way to the archive yet:

• #798366 on lilo by Dmitry Bogatov: remove usage of __TIME__ and __DATE__ macros.
• #798557 on libapache-dbi-perl by Dhole: set date of the manpage to the latest debian/changelog entry.
• #798776 on testdisk by upstream!

reproducible.debian.net

The configuration of all remote armhf and amd64 nodes in now finished. The remaining reproducibility tests running on the Jenkins host has been removed. armhf results and graphs are now visible in dashboard. We can now test the whole archive in 2-3 weeks using the current 12 amd64 jobs and 3 months using the current 6 armhf builders. We will be looking at improving the armhf sitation, maybe using more native systems or via arm64. (h01ger)

The Jenkins UI is now more responsive since all jobs building packages have been moved to remote hosts. (h01ger)

A new job has been added to collect information about build nodes to be included in the variation table. (h01ger)

The “currently scheduled” page has been split for amd64 and armhf. They now give an overview (refreshed every minute, thanks to Chris Lamb) of the packages currently being tested. (h01ger)

Several cleanup and bugfixes have been made, especially in the remote building and maintenance scripts. They should now be more robust against network problems. The automatic scheduler is now also run closer to when schroots and pbuilders are updated. (h01ger, mapreri)

Package reviews

16 reviews have been removed, 54 added and 55 updated this week.

Santiago Vila renamed lc_messages_randomness with the more descriptive different_pot_creation_date_in_gettext_mo_files.

New issues added this week: timestamps_in_reports_generated_by_abi_compliance_checker, umask_and_timestamp_variation_in_tgz_generated_by_abi_compliance_checker, and timestamps_added_by_blast2.

23 new FTBFS bugs have been filled by Chris Lamb, and Niko Tyni.

Misc.

Red Hat developper Mike McLean had a talk at Flock 2015 about reproducible builds in Koji. Slides and video recording are available. Koji is the build infrastructure used by Fedora, Red Hat and other distributions. It already keeps track of the environment used for a given build, so the required changes for handling the environment are smaller than the ones in Debian. Fedora is still missing a team effort to fix non-determinism in the package builds, but it is great to see Fedora moving forward.