Reproducible Builds: Weekly report #180

Here’s what happened in the Reproducible Builds effort between Sunday September 30 and Saturday October 6 2018:

• Yet another reminder that the fourth Reproducible Builds summit will be taking place between 11th—13th December 2018 in Paris at Mozilla’s offices. If you are interested in attending please send an email to holger@layer-acht.org. More details can also be found on the corresponding event page of our website.

• In addition, Mariusz Zaborski will present a talk on reproducible builds on the 13th October at Security BSides Warsaw 2018.

• Eric Myhre gave a talk titled Path-agnostic binaries, co-installable libraries, and How To Have Nice Things at All Systems Go 2018 in Berlin, Germany featuring “content addressable splays” that benefit from reproducible builds.

• Bernhard M. Wiedemann wrote autoprovenance to help to understand how and where unreproducible files are created. Whilst it is meant to be distribution-agnostic it currently only works with rpmbuild-strace and as such is more useful for RPM-based distros.

• Marek Marczykowski-Górecki reported on Qubes OS progress on achieving reproducible installation ISO images.

• Mattia Rizzolo uploaded diffoscope version 103 to Debian unstable incorporating a fix by Chris Lamb to support gnumeric 1.12.43.

• Guillem Jover provided a patch to strip-nondeterminism to remove some old Date::Parse code which was subsequently uploaded to Debian unstable by Chris Lamb.

• Chris Lamb reviewed 35 Debian packages. In addition, 6 were updated and 15 were removed in this week, adding to our knowledge about identified issues. Chris Lamb also added two issue toolchain issues (path_in_texi_figure_documentation & randomness_in_python_changelog_references) and Peter Wu updated the documentation for cmake.

• Vagrant Cascadian gave an impromptu talk about reproducible builds at CAT Barcamp.

Packages reviewed and fixed, and bugs filed

• Bernhard M. Wiedemann:

  • bedtools (drop environment.pickle)
  • bowtie2 (date, hostname)
  • dealii (fixed, date / copyright year)
  • gap-ace (drop date)
  • gdb (includes testresults)
  • gettext-runtime (date)
  • gismo (CPU-detection)
  • gpg2 (date from patch updating mtime)
  • libcbor (bug, parallelism-race)
  • llvm/clang (fix FTBFS-2030)
  • obs-build (disable ext2/3/4 dir_index to have more stable readdir)
  • pocl (CPU detection)
  • python-pyproj (merged, sort python glob / readdir(2))
  • python-scipy (merged, sort glob / readdir(2))
  • python-service_identity (workaround parallelism-induced ordering issues in sphinx-build; found that it is not a race condition)
  • vpp (fixed bug: FTBFS-j1, upstream)
  • wcalc (use convert -strip)
  • xine-ui/xine-lib (bug: filesystem ordering causes randomly missing entries)
• Chris Lamb:
  • #910354 filed against pjproject.
  • #909936 filed against python-changelog (merged upstream)
• Marek Marczykowski-Górecki:
  • createrepo (timestamps in repomd.xml)
  • lorax (multiple reproducibility issues)
  • pungi (merged, multiple reproducibility issues)

Test framework development

There were a huge number of updates to our Jenkins-based testing framework that powers tests.reproducible-builds.org by Holger Levsen this week, including:

• Arch Linux-specific changes:

  • Add new page with scheduled builds.
  • Add new page with the recent builds from the last day.
  • Explain how many packages were scheduled.
  • Delete the first build’s artifacts if the second build fails.
  • Reduce the minimum date for rescheduling to 10 days.
  • Add comment explaining why we might delete packages that are building.
  • Shorten IRC message if longer then 256 characters.

• Debian GNU/Linux-specific changes:

  • Make all arm64 nodes use codethink16 as an APT proxy. (#909838)
  • Allow squid access from other codethink-hosted nodes.
  • Install debootstrap from stretch-backports on the Ubuntu nodes as debootstrap 1.0.78+nmu1ubuntu1.6 cannot install on unstable anymore.
  • Don’t show graphs with oldest stretch builds on performance stats page anymore.
  • Schedule more new packages.
  • Increase log level when running pbuilder --create.

• OpenWrt-specific changes:

  • Now that the projects have re-joined, merge the LEDE work back in so that we only have OpenWrt tests (1, 2, 3, 4, 5, 6)
  • Create a landing page at tests.reproducible-builds.org/openwrt/
  • Use upstream Git repository (instead of lynxis’s fork).
  • Document that every build is done under a umask(2) of 0022, because this is enforced by the OpenWrt build system.
  • Add more console output.

• Misc/generic changes:

  • Automatically mark “sick” nodes as offline after an hour of problems, not three hours.
  • Distinguish between automatically and manually offline nodes in health overview.
  • Update FreeBSD test system to version 11.2.
  • Update some timeouts, including increasing the timeout to 24 hours for some longer-running jobs and adding a timeout of 12 hours for general reproducible jobs & 15 minutes for healthcheck jobs.
  • Be move verbose when IRC message sending fails.

In addition, Alexander Couzens added a comment regarding OpenWrt/LEDE which was subsequently amended by Holger.

Misc.

This week’s edition was written by Bernhard M. Wiedemann, Chris Lamb, heinrich5991, Holger Levsen, Marek Marczykowski-Górecki, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.