Reproducible Builds: Weekly report #181

Here’s what happened in the Reproducible Builds effort between Sunday October 7 and Saturday October 13 2018:

• Holger gave a talk about diffoscope at the MirageOS hacker retreat. This resulted in three bugs being filed (#910540 (backtrace when using diffoscope on MacOS, #910542 (Please add better support for OCaml object files), & #910541 (Filing bugs on diffoscope is cumbersome for non-Debian contributors).

• In the upcoming weeks, Chris Lamb and kpcyrd will be attending New York University to work with students from the Application Security course on the topic of reproducible builds.

• Richard Purdie reported that the Yocto Project now offers reproducible builds for arbitrary paths and build users. This is only tested with core-image-sato, an X-based graphical image but means the core system components are reproducible, providing a solid base that further improvements can be made from.

• Mes (a Scheme-based compiler for our “sister” bootstrappable builds effort) announced their 0.18 release.

• Mattia Rizzolo updated the Contribute section on our website, removing some references from the now-deprecated Alioth Debian service. He also added a handy link to file a new bug report on the diffoscope.org website.

• disorderfs (our FUSE-based filesystem that deliberately introduces non-determinism into filesystem metadata) version 0.5.4-2 was uploaded to Debian unstable by Holger Levsen. It included contributions already covered in previous weeks but also include some fixes to the the debian/watch file (1, 2 & 3) and was migrated to use the new debhelper-compat (= 11) notation.

• Chris Lamb reviewed 25 Debian packages and 28 were removed in this week, adding to our knowledge about identified issues.Chris also added a new timestamps_in_episode_files_generated_by_jxc toolchain issue and expanded the comments for translations_missing_in_desktop_files.

• strip-nondeterminism version 0.043-2 was uploaded to Debian unstable by Mattia Rizzolo. It included contributions already covered in previous weeks as well using the new debhelper-compat (= 11) notation and fixing the debian/watch file after moving from the now-deprecated Alioth Debian infrastructure.

Another brief reminder that another Reproducible Builds summit will be taking place between 11th—13th December 2018 in Mozilla’s offices in Paris. If you are interested in attending please send an email to holger@layer-acht.org. More details can also be found on the corresponding event page of our website.

diffoscope development

diffoscope is our in-depth “diff-on-steroids” utility which helps us diagnose reproducibility issues in packages) was updated this week, including contributions from:

• Chris Lamb:
  • Use str.startswith(...) over str.index(...) == 0 in the Macho comparator to prevent tracebacks if text cannot be found on the line. (#910540).
• Marek Marczykowski-Górecki:
  • Handle FAT filesystems with the fsimage comparator.
  • Add a test for FAT fsimage comparator.
• Mattia Rizzolo:
  • Adjust some build dependencies: Add linux-image-amd64 [amd64] to allow the SquashFS tests to run and linux-image-generic.
  • Adjust “autopkgtest” dependencies: Add linux-image-amd64 as well as the optional Python modules.
  • Move the list of optional Python packages to setup.py and have dh_python3 check the generated `requires.txt.
  • Add a Build-Conflicts against graphicsmagick-imagemagick-compat.
  • Temporarily drop {Build,Test}-Depends for apktool and oggvideotools; they are not available in buster at the moment.
  • Rename procyon-decompiler to procyon,

Packages reviewed and fixed, and bugs filed

• Bernhard M. Wiedemann:
  • hub (Makefile fix)
  • imagemagick (noarch diff in png + gif)
  • ninja (discovered unreleased fix for FTBFS-2038)
  • perl-MooseX-Role-Cmd (make noarch)
  • perl-Term-Size-Perl (date)
  • pocl (fixed, compile-time CPU-detection)
  • python-keystoneauth1 (randomness in docs)
  • ronn (date)
• Chris Lamb:
  • #910490 filed against libhandy.
  • #910655 filed against fast5.
  • #910752 filed against syndie.
  • #910789 filed against opari2.

Test framework development

There were a large number of updates to our Jenkins-based testing framework that powers tests.reproducible-builds.org by Holger Levsen this week, including:

• Arch Linux-specific changes:
  • Temporarily disable building with build2 and build as the jenkins user. This was an attempt to diagnose a path-variation issue.
  • Also call printenv before the builds to diagnose path issue.
  • Always force PATH to match makechrootpkg, else Arch Linux builds vary as the include the current PATH in the build results.
  • Increase timeout for running diffoscope to one hour.
  • Trap “exit” so we properly end the schroot sessions on failures.
  • Include the number of recent builds on the recent_builds.html page.
  • Drop an unused variable.

• Debian GNU/Linux-specific changes:

  • Re-add 8 i386 builders and 8 arm64 builders as the relevant build nodes are not using all the memory they have available to them.

• Misc/generic changes:

  • Add a new job health page (1, 2, 3, 4, etc.)
  • Warn on files greater than 8G in /var/log and fail maintenance job if files greater than 32G are found.
  • Add health-check and maintenance jobs for the pb10 node.
  • Only check for outdated running kernels once per day; the maintenance job also checks this.
  • Fix path to /var/spool/squid/ for compatibility with stretch.
  • Perform some node maintenance. (1, 2, 3, etc.).

In addition, Mattia Rizzolo performed some node administration (1, 2).

Misc.

This week’s edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.