Reproducible Builds: Weekly report #74

Here is what happened in the Reproducible Builds effort between Sunday September 18 and Saturday September 24 2016:

Outreachy

We intend to participate in Outreachy Round 13 and look forward for new enthusiastic applications to contribute to reproducible builds. We’re offering four different areas to work on:

• Improve test and debugging tools.
• Improving reproducibility of Debian packages.
• Improving Debian infrastructure.
• Help collaboration across distributions.

Reproducible Builds World summit #2

We are planning e a similar event to our Athens 2015 summit and expect to reveal more information soon. If you haven’t been contacted yet but would like to attend, please contact holger.

Toolchain development and fixes

Mattia uploaded dpkg/1.18.10.0~reproducible1 to our experimental repository. and covered the details for the upload in a mailing list post.

The most important change is the incorporation of improvements made by Guillem Jover (dpkg maintainer) to the .buildinfo generator. This is also in the hope that it will speed up the merge in the upstream.

One of the other relevant changes from before is that .buildinfo files generated from binary-only builds will no longer include the hash of the .dsc file in Checksums-Sha256 as documented in the specification.

Even if it was considered important to include a checksum of the source package in .buildinfo, storing it that way breaks other assumptions (eg. that Checksums-Sha256 contains only files part of that are part of a single upload, whereas the .dsc might not be part of that upload), thus we look forward for another solution to store the source checksum in .buildinfo.

Bugs filed

• #838713 filed against python-xlib by Chris Lamb.
• #838754 filed against golang-google-grpc by Chris Lamb.
• #838188 filed against ocaml by Johannes Schauer.
• #838785 filed against funnelweb by Reiner Herrmann.

Reviews of unreproducible packages

250 package reviews have been added, 4 have been updated and 4 have been removed in this week, adding to our knowledge about identified issues.

4 issue types have been added:

• captures_users_gecos issue
• timestamps_in_org_mode_html_output toolchain issue.
• varnish_vmodtool_random_file_id
• gpg_keyring_magic_bytes_differ

3 issue types have been updated:

• timestamps_in_org_mode_html_output more generally.
• timestamps_in_documentation_generated_by_texi2html
• randomness_in_ocaml_preprocessed_files

Weekly QA work

FTBFS bugs have been reported by:

• Chris Lamb (11)
• Santiago Vila (2)

Documentation updates

h01ger created a new Jenkins job so that every commit pushed to the master branch for the website will update reproducible-builds.org.

diffoscope development

• Mattia Rizzolo:
  • Skip rlib tests if the “nm” tool is missing
• Ximin Luo:
  • Give better advice about what envvar to set to make the console work
  • tests/basic-command-line: check exit code and use a more complex example
  • Add a script to check sizes of dependencies
  • Don’t use unicode quotes to avoid breakage under LC_ALL=C

strip-nondeterminism development

• Chris Lamb:
  • .perltidyrc: Add from lintian.
  • .perltidyrc: We use tabs, not spaces.
  • Run perltidy

reprotest development

• Ximin Luo uploaded reprotest 0.3 and 0.3.1 to unstable with these changes:
  • Make some variations more reliable, so tests don’t fail
  • Add a safety device to guard against typos
  • Address lintian warnings
  • Remove any existing artifact, in case the build script doesn’t overwrite it
  • Fix the logic of some tests, and don’t vary fileordering on Debian buildds
  • Use the magic of VIRTUALENV_DOWNLOAD=no, seen in tox’s own autopkgtest tests
  • Flush so subprocess output is guaranteed to appear later
  • Don’t error if the build command generates stderr
  • Default tests to run on “null” only since it takes effort to set up the others
  • hey dawg i herd u liek tests so i put some tests in ur tests so u can test while u test
  • Make no_clear_on_error optional; we don’t want to pass it in everywhere e.g. tests
  • Output a nice big obvious summary at the end when successful
  • Don’t repeat documentation in two different places, move it all to –help
  • More reliable build artifact pattern matching, and update docs
  • Use a list comprehension for slightly more idiomatic python than map/lambda
  • Make multi-component artifact patterns like ‘*.deb *.changes’ work correctly
  • Add a –no-clean-on-error option so you can analyse what went wrong
  • More help text for virtual_server_args
  • Support shell patterns as the build_artifact
  • Use sys.exit inside main(), not check() as it’s more idiomatic
  • Pass kind=’build’ to check_exec so it doesn’t time out after 100s
  • adt_testbed: add stdout/stderr to the “auxverb failed” error message, if available
  • If no virtual_server is given then use “null”

tests.reproducible-builds.org

• The full rebuild of all packages in unstable (for all tested archs) with the new build path variation has been completed. This has had the result that we are down to ~75% reproducible packages in unstable now. In comparison, for testing (where we don’t vary the build path) we are still at ~90%. IRC notifications for unstable have been enabled again. (Holger)
• Make the notes job robust about bad data (see #833695 and #833738). (Holger)
• Setup profitbricks-build7 running stretch as testing reproducible builds of F-Droid need to use a newer version of vagrant in order to support running vagrant VMs with kvm on kvm. (Holger)
• The misbehaving ‘opi2a’ armhf node has been replaced with a Jetson-TK1 board kindly donated by NVidia. This machine is using an NVIDIA tegra-k1 (cortex-a15) quad-core board. (vagrant and Holger)

Misc.

This week’s edition was written by Chris Lamb, Holger Levsen and Mattia Rizzolo and reviewed by a bunch of Reproducible Builds folks on IRC.