Reproducible Builds: Weekly report #59

What happened in the Reproducible Builds effort between June 5th and June 11th 2016:

Media coverage

Ed Maste gave a talk at BSDCan 2016 on reproducible builds (slides, video).

GSoC and Outreachy updates

Weekly reports by our participants:

• Scarlett Clark worked on making some packages reproducible, focusing on KDE backend and utility programs.
• Ceridwen published an initial design for the interface for reprotest, including a discussion on different types of build variations and the difficulties of specifying certain types of variations.
• Valerie Young improved documentation for building our tests website, began migrating Debian-specific pages into a new namespace, and planned future work around its navigation.

Documentation update

• Ximin Luo proposed a modification to our SOURCE_DATE_EPOCH spec explaining FORCE_SOURCE_DATE.

  Some upstream build tools (e.g. TeX, see below) have expressed a desire to control which cases of embedded timestamps should obey SOURCE_DATE_EPOCH. They were not convinced by our arguments on why this is a bad idea, so we agreed on an environment variable FORCE_SOURCE_DATE for them to implement their desired behaviour - named generically, so that at least we can set it centrally. For more details, see the text just linked. However, we strongly urge most build tools not to use this, and instead obey SOURCE_DATE_EPOCH unconditionally in all cases.

Toolchain fixes

• TeX Live 2016 released with SOURCE_DATE_EPOCH support for all engines except LuaTeX and original TeX.
• Continued discussion (alternative archive) with TeX upstream, about SOURCE_DATE_EPOCH corner cases, eventually resulting in the FORCE_SOURCE_DATE proposal from above.
• gcc-5/5.4.0-4 by Matthias Klose now avoids storing -fdebug-prefix-map in DW_AT_producer, thanks to #819176 by Daniel Kahn Gillmor.
• sphinx/1.4.3-1 by Dmitry Shachnev now drops Debian-specific patches relating to SOURCE_DATE_EPOCH applied upstream, #820895 by Alexis Bienvenüe.
• asciidoctor/1.5.4-2 by Cédric Boutillier now supports SOURCE_DATE_EPOCH, thanks to #820435 by Alexis Bienvenüe.
• dh-python/1.5.4-2 by Piotr Ożarowski now behaves better in some cases, thanks to #804339 by Chris Lamb.

Packages fixed

The following 16 packages have become reproducible due to changes in their build-dependencies: apertium-dan-nor apertium-swe-nor asterisk-prompt-fr-armelle blktrace canl-c code-saturne coinor-symphony dsc-statistics frobby libphp-jpgraph paje.app proxycheck pybit spip tircd xbs

The following 5 packages are new in Debian and appear to be reproducible so far: golang-github-bowery-prompt golang-github-pkg-errors golang-gopkg-dancannon-gorethink.v2 libtask-kensho-perl sspace

The following packages had older versions which were reproducible, and their latest versions are now reproducible again after being fixed:

• excellent-bifurcation/0.0.20071015-8 by Vincent Cheng, #826427 by Reiner Herrmann.
• gnurobbo/0.68+dfsg-2 by Stephen Kitt, #826424 by Reiner Herrmann.

The following packages have become reproducible after being fixed:

• gdbm/1.8.3-14 by Matthias Klose, #774394 by Jérémy Bobbio.
• miceamaze/4.2.1-3 by Sarah COUDERT, #825634 by Reiner Herrmann.
• netcdf/1:4.4.1~rc2-1~exp3 by Bas Couwenberg.
• osmo-bts/0.4.0-2 by Ruben Undheim.
• pd-hcs/0.1-3 by IOhannes m zmölnig.
• pd-hid/0.7-2 by IOhannes m zmölnig.
• python-certbot/0.8.0-1 by Harlan Lieberman-Berg, #824452 by Chris Lamb.
• python-csb/1.2.3+dfsg-3 by Sascha Steinbiss.
• python-osprofiler/1.3.0-2 by Thomas Goirand.
• usb-modeswitch-data/20160112-3 by Didier Raboud, #826343 by intrigeri.

Some uploads have fixed some reproducibility issues, but not all of them:

• bzr/2.7.0-7 by Jelmer Vernooij.
• clanlib/1.0~svn3827-5 by Stephen Kitt, #790357 by Chris Lamb.
• dwarfutils/20160507+git20160523.9086738-1 by Fabian Wolff.
• fakeroot/1.20.2-2 by Clint Adams, #795861.
• fastqtl/2.184+dfsg-2 by Dylan Aïssi, #826209 by Chris Lamb.
• gnuplot/5.0.3+dfsg3-3 by Anton Gladky.
• lazarus/1.6+dfsg-3 by Paul Gevers.
• python-pygit2/0.24.0-3 by Ondřej Nový.

Patches submitted that have not made their way to the archive yet:

• #806331 against xz-utils by Ximin Luo: make the selected POSIX shell stable across build environments
• #806494 against gnupg by intrigeri: Make man pages not embed a build-time dependent timestamp
• #806945 against bash by Reiner Herrmann and Ximin Luo: Use the system man2html, and set PGRP_PIPE unconditionally.
• #825857 against python-setuptools by Anton Gladky: sort libs in native_libs.txt
• #826408 against brainparty by Reiner Herrmann: Sort object files for deterministic linking order
• #826416 against blockout2 by Reiner Herrmann: Sort the list of source files
• #826418 against xgalaga++ by Reiner Herrmann: Sort source files to get a deterministic linking order
• #826423 against kraptor by Reiner Herrmann: Sort source files for deterministic linking order
• #826431 against traceroute by Reiner Herrmann: Sort lists of libraries/source/object files
• #826544 against doc-debian by intrigeri: make the created files stable regardless of the locale
• #826676 against python-openstackclient by Chris Lamb: make the build reproducible
• #826677 against cadencii by Chris Lamb: make the build reproducible
• #826760 against dctrl-tools by Reiner Herrmann: Sort object files for deterministic linking order
• #826951 against slicot by Alexis Bienvenüe: please make the build reproducible (fileordering)
• #826982 against hoichess by Reiner Herrmann: Sort object files for deterministic linking order

Package reviews

68 reviews have been added, 19 have been updated and 28 have been removed in this week. New and updated issues:

• cryptographic_signature
• timestamps_in_maven_version_files
• ftbfs_build-indep_not_build_on_some_archs
• timestamps_added_by_xbean_spring
• timestamps_in_maven_metadata_local_xml_files
• timestamps_in_documentation_generated_by_asciidoctor

26 FTBFS bugs have been reported by Chris Lamb, 1 by Santiago Vila and 1 by Sascha Steinbiss.

diffoscope development

• Mattia Rizzolo uploaded diffoscope/54 to jessie-backports.

strip-nondeterminism development

• Mattia uploaded strip-nondeterminism/0.018-1 to jessie-backports, to support a debhelper backport.
• Andrew Ayer uploaded strip-nondeterminism/0.018-2 fixing #826700, a packaging improvement for Multi-Arch to ease cross-build situations.
• 2 days later Andrew released strip-nondeterminism/0.019; now strip-nondeterminism is able to:
  • recursively normalize JAR files embedded within JAR files (#823917)
  • clamp the timestamp, the same way tar >=1.28-2.2 can (for now available only for gzip archives)

disorderfs development

• Andrew Ayer released disorderfs/0.4.3, fixing a issue with umask handling (#826891)

tests.reproducible-builds.org

• Valerie Young namespaced the Debian-specific pages to /debian/ namespace, with redirects to for the previous URLs.
• Holger Levsen improved the reliability of build jobs: the availability of both build nodes (for a given build) is now being tested when a build job is started, to better cope when one of the 25 build nodes go down for some reason.
• Ximin Luo improved the index of identified issues to include the total popcon scores of each issue, which is now also used for sorting that page.

Misc.

Steven Chamberlain submitted a patch to FreeBSD’s makefs to allow reproducible builds of the kfreebsd installer.

Ed Maste committed a patch to FreeBSD’s binutils to enable deterministic archives by default in GNU ar.

Helmut Grohne experimented with cross+native reproductions of dash with some success, using rebootstrap.

This week’s edition was written by Ximin Luo, Chris Lamb, Holger Levsen, Mattia Rizzolo and reviewed by a bunch of Reproducible builds folks on IRC.