← View all weekly reports

Reproducible Builds: Weekly report #1

Published: May 4, 2015.

Debian Jessie has been released on April 25th, 2015. This has opened the Stretch development cycle. Reactions to the idea of making Debian build reproducibly have been pretty enthusiastic. As the pace is now likely to be even faster, let’s see if we can keep everyone up-to-date on the developments.

Before the release of Jessie

The story goes back a long way but a formal announcement to the project has only been sent in February 2015.

Since then, too much work has happened to make a complete report, but to give some highlights:

  • New variations are now tested: umask, kernel version, domain name, and timezone. We might only be missing CPU type and current date now.
  • Many improvements to the test system on jenkins.debian.net and the pages showing the results.
  • Now not only packages from unstable are tested but also those in testing and experimental.
  • When rescheduling packages for testing, the build products can be kept and the IRC channel gets a notification when its over.
  • binutils version 2.25-6 is now built with the --enable-deterministic-archives flag. Making ar, strip and others create deterministic static libraries.
  • Number of identified issues has grown from about 80 to 123 today.

Lunar did a pretty improvised lightning talk during the Mini-DebConf in Lyon.

This past week

It seems changes were pilling behind the curtains given the amount of activity that happened in just one week.

Toolchain fixes

  • Niels Thykier uploaded debhelper/9.20150501 which includes fixes to dh_makeshlibs (#774100), dh_icons (#774102), dh_usrlocal (#775020). Patches written by Lunar.
  • Helmut Grohne uploaded doxygen/1.8.9.1-3 which will not generate timestamps in HTML by default. Kudos to akira for bringing the issue upstream.
  • Kenneth J. Pronovici uploaded epydoc/3.0.1+dfsg-6 adding a --no-include-build-time option. Patch by Jelmer Vernooij.
  • David Prévot uploaded php-apigen/2.8.1+dfsg-2 which now has reproducible output.
  • Cédric Boutillier uploaded ruby-prawn/2.0.1+dfsg-1 which now produce a deterministic output when using gradients. Patch by Lunar.
  • Jelmer Vernooij uploaded samba/2:4.1.17+dfsg-4 which contains a patch by Matthieu Patou making the output of pidl (from libparse-pidl-perl) reproducible.
  • Dmitry Shachnev uploaded sphinx/1.3.1-1 in experimental which should produce deterministic output. The original patch from Chris Lamb has inspired the upstream fix.
  • gregor herrmann uploaded libextutils-depends-perl/0.404-1 which makes ExtUtils::Depends output deterministic. Original patch by Reiner Herrmann.
  • Niko Tyni uploaded perl/5.20.2-4 which makes the output of Pod::Man reproducible. Nice team work visible on #780259.

We also rebased the experimental version of debhelper twice to merge the latest set of changes.

Lunar submitted a patch to add a -creation- date to genisoimage.

Reiner Herrmann opened #783938 to request making -notimestamp the default behavior for javadoc.

Juan Picca submitted a patch to add a --use-date flag to texi2html.

Packages fixed

The following packages became reproducible due to changes of their build dependencies: apport, batctl, cil, commons- math3, devscripts, disruptor, ehcache, ftphs, gtk2hs- buildtools, haskell-abstract- deque, haskell-abstract- par, haskell-acid- state, haskell- adjunctions, haskell- aeson, haskell-aeson- pretty, haskell- alut, haskell-ansi- terminal, haskell- async, haskell- attoparsec, haskell- augeas, haskell-auto- update, haskell-binary- conduit, haskell- hscurses, jsch, ledgersmb, libapache2-mod-auth- mellon, libarchive- tar-wrapper-perl, libbusiness-onlinepayment-payflowpro- perl, libcapture-tiny-perl, libchi-perl, libcommons-codec- java, libconfig-model- itself-perl, libconfig-model-tester-perl, libcpan-perl-releases-perl, libcrypt-unixcrypt- perl, libdatetime- timezone-perl, libdbd- firebird-perl, libdbix- class-resultset-recursiveupdate-perl, libdbix-profile- perl, libdevel-cover- perl, libdevel-ptkdb- perl, libfile-tail- perl, libfinance-quote- perl, libformat-human- bytes-perl, libgtk2-perl, libhibernate- validator-java, libimage-exiftool-perl, libjson-perl, liblinux-prctl- perl, liblog-any- perl, libmail-imapclient- perl, libmocked- perl, libmodule-build-xsutil- perl, libmodule- extractuse-perl, libmodule-signature-perl, libmoosex-simpleconfig-perl, libmoox-handlesvia- perl, libnet-frame- layer-ipv6-perl, libnet-openssh-perl, libnumber-format-perl, libobject-id-perl, libpackage-pkg-perl, libpdf-fdf-simple-perl, libpod-webserver-perl, libpoe-component-pubsub-perl, libregexp-grammars-perl, libreply-perl, libscalar-defer-perl, libsereal-encoder-perl, libspreadsheet-read-perl, libspring-java, libsql- abstract-more-perl, libsvn-class-perl, libtemplate-plugin-gravatar-perl, libterm-progressbar- perl, libterm-shellui- perl, libtest-dir- perl, libtest-log4perl- perl, libtext-context- eitherside-perl, libtime-warp-perl, libtree- simple-perl, libwww-shorten- simple-perl, libwx- perl-processstream-perl, libxml-filter-xslt-perl, libxml-writer-string-perl, libyaml-tiny-perl, mupen64plus-core, nmap, openssl, pkg-perl- tools, quodlibet, r-cran- rjags, r-cran- rjson, r-cran- sn, r-cran- statmod, ruby- nokogiri, sezpoz, skksearch, slurm- llnl, stellarium.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues but not all of them:

Patches submitted which did not make their way to the archive yet:

Improvements to reproducible.debian.net

Mattia Rizzolo has been working on compressing logs using gzip to save disk space. The web server would uncompress them on-the-fly for clients which does not accept gzip content.

Mattia Rizzolo worked on a new page listing various breakage: missing or bad debbindiff output, missing build logs, unavailable build dependencies.

Holger Levsen added a new execution environment to run debbindiff using dependencies from testing. This is required for packages built with GHC as the compiler only understands interfaces built by the same version.

debbindiff development

Version 17 has been uploaded to unstable. It now supports comparing ISO9660 images, dictzip files and should compare identical files much faster.

Documentation update

Various small updates and fixes to the pages about PDF produced by LaTeX, DVI produced by LaTeX, static libraries, Javadoc, PE binaries, and Epydoc.

Package reviews

Known issues have been tagged when known to be deterministic as some might unfortunately not show up on every single build.

For example, two new issues have been identified by building with one timezone in April and one in May. RD and help2man add current month and year to the documentation they are producing.

1162 packages have been removed and 774 have been added in the past week. Most of them are the work of proper automated investigation done by Chris West.

Summer of code

Finally, we learned that both akira and Dhole were accepted for this Google Summer of Code. Let’s welcome them!

They have until May 25th before “coding” officialy begins. Now is the good time to help them feel more comfortable by sharing all these little bits of knowledge on how Debian works.

← View all weekly reports