Reproducible Builds: Weekly report #107

Here’s what happened in the Reproducible Builds effort between Sunday May 7 and Saturday May 13 2017:

Report from Reproducible Builds Hamburg Hackathon

We were 16 participants from 12 projects: 7 Debian, 2 repeatr.io, 1 ArchLinux, 1 coreboot + LEDE, 1 F-Droid, 1 ElectroBSD + privoxy, 1 GNU R, 1 in-toto.io, 1 Meson and 1 openSUSE. Three people came from the USA, 3 from the UK, 2 Finland, 1 Austria, 1 Denmark and 6 from Germany, plus we several guests from our gracious hosts at the CCCHH hackerspace as well as a guest from Australia…

We had four presentations:

• “Reproducible Builds everywhere” by h01ger
• https://in-toto.io by Justin Cappos
• http://repeatr.io by Eric Myhre
• http://mesonbuild.com by Jussi Pakkanen

Some of the things we worked on:

• h01ger did orga stuff for this very hackathon, discussed tests.r-b.o with various non-Debian contributors, filed some bugs and restarted the policy discussion in #844431. He also did some polishing work on tests.r-b.o which shall be covered in next issue of our weekly blog.
• Justin Cappos involved many of us in interesting discussions and started to write an academic paper about Reproducible Builds of which he shared an early beta on our mailinglist.
• Chris Lamb (lamby) filed a number of patches for individual packages, worked on diffoscope, merged many changes to strip-nondeterminism and also filed #862073 against dak to upload buildinfo files to external services.
• Maria Glukhova (siamezzze) fixed a bug with plots on tests.reproducible-builds.org and worked on diffoscope test coverage.
• Lynxis worked on a new squashfs upstream release improving support for reproducible squashfs filesystems and also had some time to hack on coreboot and show others how to install coreboot on real hardware.
• Michael Poehn worked on integrating F-Droid builds into tests.reproducible-builds.org, on the F-Droid verification utility and also ran some app reproducibility tests.
• Bernhard worked on various unreproducible issues upstream and submitted fixes for curl, bzr, ant.
• Erin Myhre worked on bootstrapping cleanroom builds of compiler components in Repeatr sandboxes.
• Calvin Behling merged improvements to reppl for a cleaner storage format and better error handling and did design work for next version of repeatr pipeline execution. Calvin also lead the reproducibility testing of restaurant mood lighting.
• Eric and Calvin also claim to have had all sorts of useful exchanges about the state of other projects, and learned a lot about where to look for more info about debian bootstrap and archive mirroring from steven and lamby :)
• Phil Hands came by to say hi and worked on testing d-i on jenkins.debian.net.
• Chris West (Faux) worked on extending misc.git:has-only.py, and started looking at Britney.

We had a Debian focused meeting where we discussed a number of topics:

• IRC meetings: yes, we want to try again to have them, monthly, a poll for a good date is being held.
• Debian tests post Stretch: we’ll add tests for stable/Stretch.
• .buildinfo files, how forward: we need sourceful uploads for any arch:all packages. dak should send .buildinfo files to buildinfo.debian.net.
• (pre?) Stretch release press release: we should do that, esp. as our achievements are largely unrelated to Stretch.
• Reproducible Builds Summit 3: yes, we want that.
• what to do (in notes.git) with resolved issues: keep the issues.
• strip-nondeterminism quo vadis: Justin reminded us that strip-nondeterminism is a workaround we want to get rid off.

And then we also had a lot of fun in the hackerspace, enjoying some of their gimmicks, such as being able to open physical doors with ssh or controlling light and music with an webbrowser without authentication (besides being in the right network).

(This wasn’t the hackathon per-se, but some of us appreciated these sights and so we thought you would too.)

Many thanks to:

• Debian for sponsoring food and accommodation!
• Dock Europe for providing us with really nice accommodation in the house!
• CCC Hamburg for letting us use their hackerspace for >3 days non-stop!

News and media coverage

openSUSE has had a security breach in their infrastructure, including their build services. As of this writing, the scope and impact are still unclear, however the incident illustrates that no one should rely on being able to secure their infrastructure at all times. Reproducible Builds help mitigate this by allowing independent verification of build results, by parties that are unaffected by the compromise.

(Whilst this can happen to anyone. Kudos to openSUSE for being open about it. Now let’s continue working on Reproducible Builds everywhere!)

On May 13th Chris Lamb gave a talk on Reproducible Builds at OSCAL 2017 in Tirana, Albania.

Toolchain bug reports and fixes

• Chris Lamb:
  • Fixed a long-standing toolchain issue (#842635, #858389) in docbook-to-man by adopting the package.
  • #862003 filed against debhelper.
  • #862073 filed against ftp.debian.org, to upload buildinfo files to buildinfo.debian.net.
• Steven Chamberlain:
  • #862059 filed against sbuild, for signing buildinfo files.
• Ximin Luo:
  • #862112 filed against r-base.
  • #862113 filed against gcc-6.
  • #862116 filed against dpkg.

Packages’ bug reports

• Chris Lamb:
  • #862088 filed against compass-h5bp-plugin.
  • #862140 filed against ofxstatement-plugins.
  • #862179 filed against acct.
  • #862183 filed against libjgroups-java.
  • #862195 filed against sendip.
  • #862451 filed against wammu, forwarded upstream.
  • #862484 filed against seqan2.
  • #862553 filed against vim-command-t.
  • #862588 filed against tkhtml1.
  • #862592 filed against taskcoach.
• Chris West:
  • #862252 filed against dns-root-data.

Reviews of unreproducible packages

11 package reviews have been added, 2562 have been updated and 278 have been removed in this week, adding to our knowledge about identified issues. Most of the updates were to move ~1800 packages affected by the generic catch-all captures_build_path (out of ~2600 total) to the more specific gcc_captures_build_path, fixed by our proposed patches to GCC.

5 issue types have been updated:

• Updated docbook_to_man_one_byte_delta
• Added ocaml_captures_build_path
• Added gcj_captures_build_path
• Added gcc_captures_build_path
• Re-added docbook_to_man_one_byte_delta

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

• Adrian Bunk (1)
• Chris Lamb (2)
• Chris West (1)

diffoscope development

diffoscope development continued on the experimental branch:

• Maria Glukhova:
  • Code refactoring and more tests.
• Chris Lamb:
  • Add safeguards against unpacking recursive or deeply-nested archives. (Closes: #780761)

strip-nondeterminism development

• strip-nondeterminism 0.033-1 and -2 were uploaded to unstable by Chris Lamb. It included contributions from:

• Bernhard M. Wiedemann:
  • Add cpio handler.
  • Code quality improvements.
• Chris Lamb:
  • Add documentation and increase verbosity, in support of the long-term aim of removing the need for this tool.

reprotest development

• reprotest 0.6.1 and 0.6.2 were uploaded to unstable by Ximin Luo. It included contributions from:

• Ximin Luo:
  • Add a documentation section on “Known bugs”.
  • Move developer documentation away from the man page.
  • Mention release instructions in the previous changelog.
  • Preserve directory structure when copying artifacts. Otherwise hash output on a successful reproduction sometimes fails, because find(1) can’t find the artifacts using the original artifact_pattern.
• Chris Lamb
  • Add proper release instructions and a keyring.

trydiffoscope development

• Chris Lamb:
  • Uses the diffoscope from Debian experimental if possible.

Misc.

This week’s edition was written by Ximin Luo, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.