Reproducible Builds: Weekly report #34

What happened in the reproducible builds effort between December 13th to December 19th:

Infrastructure

Niels Thykier started implementing support for .buildinfo files in dak. A very preliminary commit was made by Ansgar Burchardt to prevent .buildinfo files from being removed from the upload queue.

Toolchain fixes

• Niels Thykier uploaded debhelper/9.20151219 which sorts files read by dh_installinit. Patch by Reiner Herrmann.
• Jo Shields uploaded mona/4.2.1.102+dfsg2-4 which lands upstream changes making GUID reproducible in unstable.
• Niko Tyni uploaded perl/5.22.1-1 which makes support for SOURCE_DATE_EPOCH in podlators available in unstable.

Mattia Rizzolo rebased our experimental debhelper with the changes from the latest upload.

New fixes have been merged by OCaml upstream.

Packages fixed

The following 39 packages have become reproducible due to changes in their build dependencies: apache-mime4j, avahi-sharp, blam, bless, cecil- flowanalysis, cecil, coco- cs, cowbell, cppformat, dbus-sharp- glib, dbus- sharp, gdcm, gnome-keyring- sharp, gudev- sharp-1.0, jackson- annotations, jackson- core, jboss- classfilewriter, jboss- jdeparser2, jetty8, json- spirit, lat, leveldb- sharp, libdecentxml- java, libjavaewah- java, libkarma, mono.reflection, monobristol, nuget, pinta, snakeyaml, taglib- sharp, tangerine, themonospot, tomboy- latex, widemargin, wordpress, xsddiagram, xsp, zeitgeist- sharp.

The following packages became reproducible after getting fixed:

• 4ti2/1.6.7+ds-2 uploaded by Jerome Benoit, original patch by Reiner Herrmann.
• clblas/2.8+ds1-3 by Ghislain Antony Vaillant.
• config-manager/0.4-2.2 by Mattia Rizzolo.
• cvs-fast-export/1.35-1 by Anthony Fok.
• dgedit/0~git20151217-1 by Víctor Cuadrado Juan.
• genometools/1.5.7-7 by Sascha Steinbiss.
• gnome-blog/0.9.1-7 uploaded by Frederic Peters, based on an NMU by Mattia Rizzolo.
• guava-libraries/19.0-1 by Emmanuel Bourg.
• kwave/0.9.0-1-1 uploaded by Pino Toscano, fixed upstream.
• libandroid-json-org-java/20121204-20090211-2 by Emmanuel Bourg.
• libchado-perl/1.23-5 uploaded by Andreas Tille, original patch by Niko Tyni.
• libffi/3.2.1-4 by Matthias Klose.
• mksh/52-1 by Thorsten Glaser.
• monit/1:5.15-3 uploaded by Sergey B Kirpichev, original patch by Chris Lamb.
• ntlmaps/0.9.9.0.1-11.5 by Mattia Rizzolo.
• perl/5.22.1~rc3-2 by Niko Tyni.
• picolisp/15.11-1 by Kan-Ru Chen.
• qelectrotech/1:0.5-1 uploaded by Denis Briand, fixed upstream.
• tomcat8/8.0.30-1 by Emmanuel Bourg.
• xfonts-ayu/1:1.7a-1 by Hideki Yamane with a patch from Chris Lamb.
• xfonts-kaname/1.1-10 by Hideki Yamane with a patch from Chris Lamb.
• xfonts-kappa20/1:0.396-1 by Hideki Yamane with a patch from Chris Lamb.

Some uploads fixed some reproducibility issues, but not all of them:

• bup/0.27-2 uploaded by Robert Edmonds, patch by Chris Lamb.
• cain/1.10+dfsg-1 uploaded by Andreas Tille, original patch by Chris Lamb.
• liblouisutdml/2.5.0-2 by Samuel Thibault.
• mariadb-10.0/10.0.22-5 by Otto Kekäläinen.

Patches submitted which have not made their way to the archive yet:

• #807837 on lxc by Reiner Herrmann: use time of latest debian/changelog entry for LXC_GENERATE_DATE.
• #807838 on graphite2 by Reiner Herrmann: tell dblatex to use a static path.
• #808032 on python-genpy by Chris Lamb: sort list of generated modules.
• #808388 on buzztrax by Chris Lamb: implement support for SOURCE_DATE_EPOCH.

reproducible.debian.net

Packages in experimental are now tested on armhf. (h01ger)

Arch Linux packages in the multilib and community repositories (4,000 more source packages) are also being tested. All of these test results are better analyzed and nicely displayed together with each package. (h01ger)

For Fedora, build jobs can now run in parallel. Two are currently running, now testing reproducibility of 785 source packages from Fedora 23. mock/1.2.3-1.1 has been uploaded to experimental to better build RPMs. (h01ger)

Work has started on having automatic build node pools to maximize use of armhf build nodes. (Vagrant Cascadian)

diffoscope development

Version 43 has been released on December 15th. It has been dubbed as “ epic! ” as it contains many contributions that were written around the summit in Athens.

Baptiste Daroussin found that running diffoscope on some Tar archives could overwrite arbitrary files. This has been fixed by using libarchive instead of Python internal Tar library and adding a sanity check for destination paths. In any cases, until proper sandboxing is implemented, don’t run diffosope on unstrusted inputs outside an isolated, throw-away system.

Mike Hommey identified that the CBFS comparator would needlessly waste time scanning big files. It will now not consider any files bigger than 24 MiB—8 MiB more than the largest ROM created by coreboot at this time. An encoding issue related to Zip files has also been fixed. (Lunar)

New comparators have been added: Android dex files (Reiner Herrmann), filesystem images using libguestfs (Reiner Herrmann), icons and JPEG images using libcaca (Chris Lamb), and OS X binaries (Clemens Lang). The comparator for Free Pascal Compilation Unit will now only be used when the unit version matches the compiler one. (Levente Polyak)

A new multi-file HTML output with on-demand loading of long diffs is available through the --html-dir option. On-demand loading requires jQuery which path can be specified through the --jquery option. The diffs can also be simply browsed for non-JavaScript users or when jQuery is not available. (Joachim Breitner)

Portability toward other systems has been improved: old versions of GNU diff are now supported (Mike McQuaid), suggestion of the appropriate locale is now the more generic en_US.UTF-8 (Ed Maste), the --list-tools option can now support multiple systems (Mattia Rizzolo, Levente Polyak, Lunar).

Many internal changes and code clean-ups have been made, paving the way for parallel processing. (Lunar)

Version 44 was released on December 18th fixing an issue affecting .deb lacking a md5sums file introduced in a previous refactoring (Lunar). Support has been added for Mozilla optimized Zip files. (Mike Hommey). The HTML output has been optimized in size (Mike Hommey, Esa Peuha, Lunar), speed (Lunar), and will now properly number lines (Mike Hommey). A message will always be displayed when lines are ignored at the end of a diff (Lunar). For portability and consistency, Python os.walk() function is now used instead of find to perform directory listing. (Lunar)

Documentation update

Package reviews

143 reviews have been removed, 69 added and 22 updated in the previous week.

Chris Lamb reported 12 new FTBFS issues.

News issues identified this week: random_order_in_init_py_generated_by_python- genpy, timestamps_in_copyright_added_by_perl_dist_zilla, random_contents_in_dat_files_generated_by_chasen- dictutils_makemat, timestamps_in_documentation_generated_by_pandoc.

Chris West did some improvements on the scripts used to manage notes in the misc repository.

Misc.

Accounts of the reproducible builds summit in Athens were written by Thomas Klausner from NetBSD and Hans-Christoph Steiner from The Guardian Project.

Some openSUSE developers are working on a hackweek on reproducible builds which was discussed on the opensuse-packaging mailing-list.