Reproducible Builds
Status Update

Chris Lamb • Holger Levsen • Maria Glukhova •
Steven Chamberlain • Vagrant Cascadian •
Valerie Young • Ximin Luo

DebConf17
Montréal, Canada

The incomplete team, with apologies to $YOU

akira • Alexis Bienvenüe • Alexander Couzens • Andrew Ayer • Asheesh Laroia • Bernhard M. Wiedemann • Boyuan Yang • Ceridwen • Chris Lamb • Chris West • Christoph Berg • Clint Adams • Dafydd Harries • Daniel Kahn Gillmor • Daniel Shahaf • Daniel Stender • David Suarez • Dhole • Drew Fisher • Emmanuel Bourg • Emanuel Bronshtein • Esa Peuha • Fabian Wolff • Guillem Jover • Hans-Christoph Steiner • Harlan Lieberman-Berg • Helmut Grohne • Holger Levsen • HW42 • Intrigeri • Jelmer Vernooij • josch • Juan Picca • Justin Cappos • Lunar • Maria Glukhova • Mathieu Bridon • Mattia Rizzolo • Nicolas Boulenguez • Niels Thykier • Niko Tyni • Paul Wise • Peter De Wachter • Philip Rinn • Reiner Herrmann • Robbie Harwood • Santiago Vila • Sascha Steinbiss • Satyam Zode • Scarlett Clark • Stefano Rivera • Stéphane Glondu • Steven Chamberlain • Tom Fitzhenry • Vagrant Cascadian • Valerie Young • Valentin Lorentz • Wookey • Ximin Luo

What is the goal of Reproducible Builds?

Prove binary came from source code.

Why do we want to prove this?

The binary could have been:

  • ...compiled by a malicious actor.
  • ...compiled with a compromised compiler.

How do we achieve Reproducible Builds?

In Debian, two branches of work:

  • 1. Compilation of binary program should be deterministic.
  • 2. Build environment of any binary program should be reproducible.

How far we've come..!

First rebuild in 201324% packages reproducible
June 201794% packages reproducible

How far we'll need to go..!

sometime100% packages reproducible
sometimetools to actually verify that in practice

Check the progress

tests.reproducible-builds.org

(reproducible.debian.net)

Technical & other security benefits

Predictable OpenID secret

# Build.PL
$build->config_data(OpenIDConsumerSecret=>int(1e15*rand()));

# /usr/share/perl5/GBrowse/ConfigData.pm
{
 'OpenIDConsumerSecret' => '639098210478536',
 'cgibin' => '/usr/lib/cgi-bin/gbrowse',
 'conf' => '/etc/gbrowse',
 [..]
},
  • Every installation shares the same secret!

#833885 (gbrowse)

Random chars in manpages

-This manual page documents the usageoof WikipediaFS.
+This manual page documents the usage of WikipediaFS.

memcpy(&buf[1], &buf[2], strlen(buf)-1);
memcpy(3): The memory areas must not overlap
  • " n\\011" → "\111" → maps to capital "I"
- memcpy(&buf[1], &buf[2], strlen(buf)-1);
+ memmove(&buf[1], &buf[2], strlen(buf)-1);

#842635 (docbook2man)

Fails to build 0.46% of the time

x = f(u('abc'), 16)
y = f(u('abc'), 16)
self.assertEqual(sorted(set(x)), [u('a'), u('b'), u('c')])

AssertionError: Lists differ: [u'a', u'b'] != [u'a', u'b', u'c']
  • (3C2)*(2/3)16 – (3C1)*(1/3)16 =~ 0.46%

#844233 (python-passlib)

Recent updates

Reproducible Builds Summit

December 2016, Berlin

Who Attended?

  • Software Freedom Conservancy
  • Bazel

reproducible-builds.org/docs/definition

A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts.
...

Reproducible Builds Summit 2016

Other work

  • .buildinfo files for RPMs
  • Cross Distro Collaboration

tests.r-b.org expansion

  • increased amd64+i386 ressources
  • added arm64
  • expanded armhf, up to 29 boards!
  • some arm64 boards building armhf, with issues
  • some more projects, more collaboration

OpenSuSE on board

  • Bernhard submitting a lot of patches upstream, pick some examples

Examples of more active distros/projects

  • Guix, Fdroid, LEDE, Coreboot, but also in-toto, Tails…

Updates on build path

  • GCC -fdebug-prefix-map, DW_AT_producer, etc
  • golang -trimpath: golang/go#16860
  • rustc --remap-path-prefix: rust-lang/rust#41555, #34902
  • BUILD_PATH_PREFIX_MAP:
    https://reproducible-builds.org/specs/build-path-prefix-map/
  • GCC patch fixed 1800 packages but getting some resistance from upstream...

Reproducibility tools

reprotest - overview and updates

  • What: run commands under varying build environments and check their output for reproducibility. Features:
    • Running inside virtual containers (e.g. ...)
    • Presets for convenience, atm only Deb packages
  • Reduce diff with autopkgtest, aim to deduplicate (ximin)
  • Make it distro-independent, first ArchLinux (santiago)

diffoscope - overview

diffoscope.org
  • What exactly makes two files different?
  • Recursively unpacks archives, decompresses PDF files, disassembles binaries etc
  • Converts various file formats to human-readable form
  • Reports differences in form of plain text, HTML, RST, JSON or Markdown
  • Try it online: try.diffoscope.org

diffoscope - updates 1/2

  • Now works better with huge diffs (like GCC)
    • possible to control how detailed the output gets
    • reuse previously generated output saved in JSON format
  • 10s of speed optimisations (via Tails)
    • from 3 hours → 8 minutes
  • Progress bar displayed when diffoscope runs in terminal
  • --exclude, --max-container-depth and other ways to control behaviour

diffoscope - updates 2/2

  • Better logging and debugging utilities
  • New formats supported for comparison: APK, OGG, .dtb, R object files (.rds, .rdb), PGP files, .docx, .odt, ...
  • New output formats: RST, Markdown, JSON
  • Visual comparison of images (JPEG, ICO, PNG, GIF)

Future directions

FTP archive and dak

  • Publish buildinfo files in the official archive, requires some dak patches.
  • Then, run tests.r-b.org rebuilds against actual Debian binaries, and encourage third parties to do the same.
  • steven began to work in this area with repro-build.pl (the topic of his "Fun with .buildinfo" talk yesterday)

Debian buster

  • We had not been testing against actual archive binaries, hope to fix this soon.
  • Recently, required packages NMU (by ximin), now all reproducible except GCC.
  • Next, begin our wider NMU campaign, for packages with long-pending patches.
  • Eventually aim for build-essential and key-packages.

Policy

  • "Packages should be reproducible" (#844431).
    • we'll need to define reproducibilty
    • reproducibile in a fairly controlled way / sane environment - not everywhere
    • define requirements / exceptions: same buld environment + same options + same path
    • mention .buildinfo files and missing processes+tools
  • Should we have this in policy now? Do we agree that Debian is ready for this, as a should which still needs work and non complying is a normal bug for now…

User interfaces

  • UI/workflow for APT to notify users about unreproducible packages (#863622).
  • sbuild, pbuilder

How can I help?

  • Join our lovely team!
  • Check your packages on qa.debian.org/developer.php
  • Merge patches & push them upstream
  • Fix toolchain issues (Java, TeX, dvips, graphviz, etc.)

#debian-reproducible
#reproducible-builds

reproducible-builds.org
reproducible-builds@lists.alioth.debian.org

Thanks to our sponsors: