Reproducible Builds: Weekly report #60

What happened in the Reproducible Builds effort between June 12th and June 18th 2016:

Media coverage

• Sune Vuorela blogged about making Qt documentation reproducible while spending good times in the Alps.

GSoC and Outreachy updates

Weekly reports by our participants:

• Scarlett Clark
• Satyam Zode

Toolchain fixes

• texlive-bin/2016.20160513.41080-3 has been uploaded to unstable, featuring support for FORCE_SOURCE_DATE. See the last post for details on it.
• doxygen/1.4.4-1 has been uploaded to unstable, fixing #822197 (upstream bug), which caused some generated html file to contain unreproducible memory addresses of Python objects used at build time.
• debhelper/9.20160618 has been uploaded to unstable, fixing #824490, which instructs ant to not save the username of the build user in the generated files. Original patch by Emmanuel Bourg.
• HW42 reported a long-known (although only internally) bug in our dpkg-buildinfo. this particular bug doesn’t affect our current infrastructure, but it’s a blocker for having .buildinfo support merged upstream.
• epydoc/3.0.1+dfsg-13 and 3.0.1+dfsg-14 have been uploaded by Kenneth J. Pronovici which fixes nondeterministic ordering issues in generated documentation and removes memory addresses. Original patches (#825968 and #827416) by Sascha Steinbiss.

With this upload of texlive-bin we decided to stop keeping our patched fork of as most of the patches for SOURCE_DATE_EPOCH support had been integrated upstream already, and the last one (making FORCE_SOURCE_DATE default to 1) had been refused. So, we are now going to let the archive be rebuilt against unstable’s texlive-bin and see how many packages will become unreproducible with this change; once enough data will be collected we will ponder whether FORCE_SOURCE_DATE should be exported by helper tools (such as debhelper) or manually exported by every package that needs it.

(For those wondering: we still recommend to follow SOURCE_DATE_EPOCH always and don’t recommend other projects to implement FORCE_SOURCE_DATE…)

With the drop of texlive-bin we now have only three modified packages in our experimental repository.

Reproducible work in other projects

• Ed Maste sent a patch to have ELF Tool Chain’s elfcopy support SOURCE_DATE_EPOCH.
• Ed Maste changed FreeBSD’s ar(1) to have a reproducible output by default when invoked with -s.
• Ed Maste merged changes from NetBSD’s makefs to FreeBSD’s to add a command line option for setting the timestamp.
• Ed Maste reported on FreeBSD package reproducibility details from the investigation for his BSDCan talk, including diffoscope results for the non-reproducible packages.
• Holger Levsen spent some time thinking and documenting how to store package notes and issues in a multi-distribution friendly manner, so that we can share these issues and notes between reproducible efforts across different projects. Thoughts we had about this topic during the Athens meeting last December are included in the updated README.

Packages fixed

The following 12 packages have become reproducible due to changes in their build dependencies: django-floppyforms flask-restful hy jets3t kombu llvm-toolchain-3.8 moap python-bottle python-debtcollector python-django-debug-toolbar python-osprofiler stevedore

The following packages have become reproducible after being fixed:

• adios/1.9.0-10 by Alastair McKinstry.
• airstrike/0.99+1.0pre6a-8 by Markus Koschany, #826421 by Reiner Herrmann.
• apt-dater/1.0.3-1 by Patrick Matthäi, #797211 by Chris Lamb.
• bitlbee/3.4.2-1 by Jelmer Vernooij.
• dar/2.5.5-1 by Laszlo Boszormenyi.
• fastjet/3.0.6+dfsg-2 by Mattia Rizzolo, #787865 by Maria Valentina Marin.
• libretro-beetle-pce-fast/0.9.38.7+git20160609-1 by Sérgio Benjamim, #827147 by Reiner Herrmann.
• libretro-beetle-psx/0.9.38.6+git20151019-2 by Sérgio Benjamim, #827148 by Reiner Herrmann.
• mx/1.99.4-1 by Ying-Chun Liu.
• python-coverage/4.1+dfsg.1-1 by Ben Finney.
• shiro/1.2.5-1 by Tony Mancill, #797296 by Chris Lamb.
• splix/2.0.0+svn315-5 by Didier Raboud.
• u-boot/2016.07~rc1+dfsg1-3 by Vagrant Cascadian, debugging and patch by HW42, patches submitted upstream.

Some uploads have fixed some reproducibility issues, but not all of them:

• gcc-mingw-w64/18 by Stephen Kitt.
• gnuplot/5.0.3+dfsg3-6 by Anton Gladky, #827197 by Alexis Bienvenüe.

Uploads with reproducibility fixes that currently fail to build:

• ruby2.3/2.3.1-3 by Christian Hofstaedtler, avoids unreproducible rbconfig.rb files by always using bash for building.

Patches submitted that have not made their way to the archive yet:

• #827109 against asciijump by Reiner Herrmann: sort source files for deterministic linking order.
• #827112 against boswars by Reiner Herrmann: sort source files for deterministic linking order.
• #827114 against overgod by Reiner Herrmann: use C locale for sorting source files.
• #827115 against netpbm by Alexis Bienvenüe: honour SOURCE_DATE_EPOCH while generating output.
• #827124 against funguloids by Reiner Herrmann: use C locale for sorting files.
• #827145 against scummvm by Reiner Herrmann: don’t embed extra fields in zip archive and build with proper host architecture.
• #827150 against netpanzer by Reiner Herrmann: sort source files for deterministic linking order.
• #827172 against reaver by Alexis Bienvenüe: sort object files for deterministic linking order.
• #827187 against latex2html by Alexis Bienvenüe: iterate deterministic over Perl hashes; honour SOURCE_DATE_EPOCH for output; strip username from output; sort index keys.
• #827313 against cherrypy3 by Sascha Steinbiss: prevent memory addresses in output.
• #827361 against matplotlib by Alexis Bienvenüe: honour SOURCE_DATE_EPOCH in output and sort keys while iterating over dict.
• #827382 against dwarfutils by Reiner Herrmann: fix array size, which caused memory from outside a table to be embedded into output.
• #827384 against skytools3 by Sascha Steinbiss: use stable sorting order and remove timestamps from documentation.
• #827419 against ldaptor by Sascha Steinbiss: sort list of input files and prevent home directory from leaking into documentation.
• #827546 against git-buildpackage by Sascha Steinbiss: replace timestamps in documentation with changelog date; prevent temporary paths in documentation.
• #827572 against xprobe by Reiner Herrmann: sort list of object files in static library archives.

Package reviews

36 reviews have been added, 12 have been updated and 31 have been removed in this week.

17 FTBFS bugs have been reported by Chris Lamb, Santiago Vila and Dominic Hargreaves.

diffoscope development

Satyam worked on argument completion (#826711) for diffoscope.

strip-nondeterminism development

Mattia Rizzolo uploaded strip-nondeterminism 0.019-1~bpo8+1 to jessie-backports.

reprotest development

Ceridwen filed an Intent To Package (ITP) bug for reprotest as #827293.

tests.reproducible-builds.org

• Mattia Rizzolo uploaded pbuilder 0.225 to unstable, providing built-in support for eatmydata. We’re planning to use it in armhf and i386 builders where we don’t build in tmpfs, to increase the build speed some more.
• Valery Young reworked the appearance of the package page, hopefully making them more intuitive and usable. In the process she changed the script generating them to use a real templating system, thus improving maintenance for the future.
• Holger adjusted the scheduler to reschedule packages in state ‘depwait’ after two days instead of three.
• Mattia added the bug title next to the bug numbers in the notes.

Misc.

This week’s edition was written by Mattia Rizzolo, Reiner Herrmann, Ed Maste and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.