Reproducible Builds: Weekly report #12

What happened in the reproducible builds effort this week:

Toolchain fixes

Eric Dorlan uploaded automake-1.15/1:1.15-2 which makes the output of mdate-sh deterministic. Original patch by Reiner Herrmann.

Kenneth J. Pronovici uploaded epydoc/3.0.1+dfsg-8 which now honors SOURCE_DATE_EPOCH. Original patch by Reiner Herrmann.

Chris Lamb submitted a patch to dh- python to make the order of the generated maintainer scripts deterministic. Chris also offered a fix for a source of non-determinism in dpkg- shlibdeps when packages have alternative dependencies.

Dhole provided a patch to add support for SOURCE_DATE_EPOCH to gettext.

Packages fixed

The following 78 packages became reproducible in our setup due to changes in their build dependencies: chemical-mime- data, clojure- contrib, cobertura-maven- plugin, cpm, davical, debian-security- support, dfc, diction, dvdwizard, galternatives, gentlyweb- utils, gifticlib, gmtkbabel, gnuplot- mode, gplanarity, gpodder, gtg- trace, gyoto, highlight.js, htp, ibus- table, impressive, jags, jansi- native, jnr- constants, jthread, jwm, khronos- api, latex-coffee- stains, latex- make, latex2rtf, latexdiff, libcrcutil, libdc0, libdc1394-22, libidn2-0, libint, libjava-jdbc- clojure, libkryo- java, libphone-ui- shr, libpicocontainer- java, libraw1394, librostlab- blast, librostlab, libshevek, libstxxl, libtools-logging- clojure, libtools- macro-clojure, litl, londonlaw, ltsp, macsyfinder, mapnik, maven-compiler- plugin, mc, microdc2, miniupnpd, monajat, navit, pdmenu, pirl, plm, scikit- learn, snp- sites, sra- sdk, sunpinyin, tilda, vdr-plugin- dvd, vdr-plugin- epgsearch, vdr-plugin- remote, vdr-plugin- spider, vdr-plugin- streamdev, vdr-plugin- sudoku, vdr-plugin- xineliboutput, veromix, voxbo, xaos, xbae.

The following packages became reproducible after getting fixed:

• analog/2:6.0-21 uploaded by Andreas Beckmann, original patch by Dhole.
• base-passwd/3.5.38 uploaded by Colin Watson, original patch by Juan Picca.
• debconf/1.5.57 uploaded by Colin Watson, original patch by Lunar.
• ipband/0.8.1-4 by Mats Erik Andersson.
• kfreebsd-10/10.1~svn274115-7 by Steven Chamberlain.
• libcommons-cli-java/1.3.1-1 by tony mancill.
• libpsl/0.7.1-1 by Daniel Kahn Gillmor.
• maven-archiver/2.6-3 by Emmanuel Bourg.
• mtink/1.0.16-9 by Graham Inggs.
• ocamlweb/1.39-2 uploaded by Mehdi Dogguy, original patch by Chris Lamb.
• rbdoom3bfg/1.0.3+repack1+git20150625-1 by Tobias Frost.
• spatialite-tools/4.2.1~rc1-2 by Bas Couwenberg.
• task/2.4.4+dfsg-1 by Sebastien Badia.

Some uploads fixed some reproducibility issues but not all of them:

• bullet/2.83.4+dfsg-1 by Markus Koschany.
• cdo/1.6.6+dfsg.1-2 by Alastair McKinstry.
• fish/2.2.0-1 uploaded by Tristan Seligmann, original patch by Chris Lamb.
• sympy/0.7.6-3 by Sergey B Kirpichev.
• xtables-addons/2.7-1 uploaded by Dmitry Smirnov, original patch by Reiner Herrmann.

Patches submitted which have not made their way to the archive yet:

• #792178 on gunroar by Reiner Herrmann: use C locale when sorting source files.
• #792181 on tth by Reiner Herrmann: remove timestamps from generated HTML files.
• #792285 on pkgconf by Juan Picca: set LC_ALL=C when running sort.
• #792319 on jsmath-fonts by Chris Lamb: set TZ=UTC when calling unzip.
• #792424 on swh-plugins by Chris Lamb: sort inputs in Makefile.
• #792525 on ruby-standalone by Reiner Herrmann: use UTC and C locale when formatting the manpage date for the documentation.
• #792528 on dict-foldoc by Reiner Herrmann: use C locale when formatting the date for the documentation.
• #792529 on tomatoes by Reiner Herrmann: use date from debian/changelog in version string.
• #792593 on lives by Dhole: process a Perl hash in stable order.
• #792596 on jsmath by Dhole: set TZ=UTC when calling unzip.
• #792597 on jsmath-fonts-sprite by Dhole: set TZ=UTC when calling unzip.
• #792598 on libreoffice-canzeley-client by Dhole: set TZ=UTC when calling unzip.
• #792599 on openthesaurus by Dhole: set TZ=UTC when calling unzip.
• #792602 on fonts-stix by Dhole: set TZ=UTC when calling unzip.
• #792667 on jack-audio-connection-kit by use date from debian/changelog in manpages.
• #792668 on pyhoca-gui by remove date from package version number.
• #792671 on apertium-dbus by remove *.pyo and *.pyc from binary package.
• #792673 on bup by use date from debian/changelog when generating version strings.
• #792684 on cain by Chris Lamb: ensure stable permissions when creating source tarball.
• #792709 on dict-jargon by Dhole: set timestamp in archive using the latest entry of debian/changelog.
• #792727 on libaqbanking by Micha Lenk (upstream): sort source files in documentation.
• #792763 on docbook-dsssl by Chris Lamb: sort input files when creating changelog.
• #792770 on lynx-cur by Reiner Herrmann: use C locale when sorting configuration files.
• #792771 on mu-cade by Reiner Herrmann: use C locale when sorting source files.
• #792772 on titanion by Reiner Herrmann: use C locale when sorting source files.
• #792783 on linuxlogo by Reiner Herrmann: use C locale when sorting source files.
• #792821 on pkg-config by Juan Picca: use C locale when sorting source files.
• #792828 on tiger by Daniel Kahn Gillmor: use C locale when listing soure files.

reproducible.debian.net

The statistics on the main page of reproducible.debian.net are now updated every five minutes. A random unreviewed package is suggested in the “look at a package” form on every build. (h01ger)

A new package set based new on the Core Internet Infrastructure census has been added. (h01ger)

Testing of FreeBSD has started, though no results yet. More details have been posted to the freebsd-hackers mailing list. The build is run on a new virtual machine running FreeBSD 10.1 with 3 cores and 6 GB of RAM, also sponsored by Profitbricks.

strip-nondeterminism development

Andrew Ayer released version 0.009 of strip- nondeterminism. The new version will strip locales from Javadoc, include the name of files causing errors, and ignore unhandled (but rare) zip64 archives.

debbindiff development

Lunar continued its major refactoring to enhance code reuse and pave the way to fuzzy-matching and parallel processing. Most file comparators have now been converted to the new class hierarchy.

In order to support for archive formats, work has started on packaging Python bindings for libarchive. While getting support for more archive formats with a common interface is very nice, libarchive is a stream oriented library and might have bad performance with how debbindiff currently works. Time will tell if better solutions need to be found.

Documentation update

Lunar started a Reproducible builds HOWTO intended to explain the different aspects of making software build reproducibly to the different audiences that might have to get involved like software authors, producers of binary packages, and distributors.

Package reviews

17 obsolete reviews have been removed, 212 added and 46 updated this week.

15 new bugs for packages failing to build from sources have been reported by Chris West (Faux), and Mattia Rizzolo.

Presentations

Lunar presented Debian efforts and some recipes on making software build reproducibly at Libre Software Meeting 2015. Slides and a video recording are available.

Misc.

h01ger, dkg, and Lunar attended a Core Infrastructure Initiative meeting. The progress and tools mode for the Debian efforts were shown. Several discussions also helped getting a better understanding of the needs of other free software projects regarding reproducible builds. The idea of a global append only log, similar to the logs used for Certificate Transparency, came up on multiple occasions. Using such append only logs for keeping records of sources and build results has gotten the name “Binary Transparency Logs”. They would at least help identifying a compromised software signing key. Whether the benefits in using such logs justify the costs need more research.