Reproducible Builds: Weekly report #69

What happened in the Reproducible Builds effort between Sunday August 14 and Saturday August 20 2016:

Fasten your seatbelts

Important note: we enabled build path variation for unstable now, so your package(s) might become unreproducible, while previously it was said to be reproducible… given a specific build path it probably still is reproducible but read on for the details below in the tests.reproducible-builds.org section! As said many times: this is still research and we are working to make it reality.

Media coverage

Daniel Stender blogged about python packaging and explained some caveats regarding reproducible builds.

Toolchain developments

Thomas Schmitt uploaded xorriso which now obeys SOURCE_DATE_EPOCH. As stated in its man pages:

ENVIRONMENT
[...]
SOURCE_DATE_EPOCH  belongs to the specs of reproducible-builds.org.  It
is supposed to be either undefined or to contain a decimal number which
tells the seconds since january 1st 1970. If it contains a number, then
it is used as time value to set the  default  of  --modification-date=,
--gpt_disk_guid,  and  --set_all_file_dates.  Startup files and program
options can override the effect of SOURCE_DATE_EPOCH.

Packages reviewed and fixed, and bugs filed

The following packages have become reproducible after being fixed:

• cadencii/3.3.9+svn20110818.r1732-5 by Ying-Chun Liu, #826677 by Chris Lamb.
• cfengine3/3.7.4-3 by Antonio Radici.
• findbugs/3.1.0~preview2-1 by Emmanuel Bourg.
• gcpegg/5.1-14 by Bdale Garbee, #777416 by Chris Lamb.
• gnunet-gtk/0.10.1-4 by Bertrand Marc, #834111 by Chris Lamb.
• konwert/1.8-12 by Yann Dirson, #778945 by Chris Lamb.
• link-grammar/5.3.8-2 by Jeremy Bicha, #829011 by Chris Lamb.
• metastudent-data/2.0.1-1 by Sascha Steinbiss.
• pixmap/2.6pl4-20 by Paul Slootman, #793709 by Chris Lamb Maria Valentina Marin a.k.a. Akira.
• python-afl/0.5.4-2 by Daniel Stender.
• tf5/5.0beta8-6 by Russ Allbery, #834276 by Chris Lamb.
• triplane/1.0.8-2 by Markus Koschanyover, #776603 by Chris Lamb.
• vips/8.3.3-1 by Laszlo Boszormenyi, #834758 by Chris Lamb.
• xzgv/0.9.1-4 by Theodore Y. Ts’o, #777274 by Chris Lamb.

The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.)

• vulkan/1.0.21.0+dfsg1-1 by Timo Aaltonen.

The following 2 packages were not changed, but have become reproducible due to changes in their build-dependencies: tagsoup tclx8.4.

Some uploads have addressed some reproducibility issues, but not all of them:

• cappuccino/0.5.1-4 by Breno Leitao, #799330 by Chris Lamb.
• darkice/1.3-0.1 by Nicolas Bouleng.
• flask-restful/0.3.5-1 by Ondřej Nový, #809780 by Chris Lamb.
• kodi/16.1+dfsg1-2 by Balint Reczey, #825285 by Lukas Rechberger.
• libaws/3.3.2-3 by Nicolas Bouleng.
• liblog4ada/1.3-1 by Nicolas Bouleng.
• libxmlezout/1.06.1-8 by Nicolas Bouleng.
• ruby-rmagick/2.15.4+dfsg-2 by Antonio Terceir, #795484 from Chris Lamb.
• taggrepper/0.05-2 by Kumar Appaiah, #777725 and #793726 from Maria Valentina Marin.
• xotcl/1.6.8-2 by Stefan Sobernig, #797543 by Chris Lamb.

Patches submitted that have not made their way to the archive yet:

• #834755 against apt-cacher-ng by Chris Lamb.
• #834976 against auto-apt-proxy by Chris Lamb.
• #834549 against botan1.10 by Chris Lamb.
• #834861 against cookiecutter by Chris Lamb.
• #834779 against dicom3tools by Chris Lamb.
• #834859 against echoping by Chris Lamb.
• #834983 against eyed3 by Chris Lamb.
• #834735 against filepp by Chris Lamb.
• #834957 against flashrom by Chris Lamb.
• #834773 against gkrellm by Chris Lamb.
• #834292 against gr-radar by Chris Lamb.
• #834956 against ircd-irc2 by Chris Lamb.
• #834897 against jpy by Chris Lamb.
• #834452 against libdc0 by Chris Lamb.
• #834324 against libnss-ldap by Reiner Herrmann.
• #834945 against libquvi by Chris Lamb.
• #834534 against librep by Chris Lamb.
• #834537 against mozvoikko by Chris Lamb.
• #834857 against nagios-nrpe by Chris Lamb.
• #834316 against openocd by Reiner Herrmann.
• #834993 against oss4 by Reiner Herrmann.
• #834302 against pd-flite by Reiner Herrmann.
• #834754 against phpab by Chris Lamb.
• #834279 against phpldapadmin by Chris Lamb.
• #834277 against pybit by Chris Lamb.
• #834553 against pyicu by Chris Lamb.
• #834541 against ruby-pg by Chris Lamb.
• #835051 against sheepdog by Chris Lamb.
• #834896 against ttf-tiresias by Chris Lamb.
• #834780 against tuxpaint-config by Chris Lamb.
• #834988 against twitter-bootstrap3 by Chris Lamb.
• #834776 against uhub by Chris Lamb.
• #835061 against varnish by Chris Lamb.

Bug tracker house keeping:

• Chris Lamb pinged 164 bugs he filed more than 90 days ago which have a patch and had no maintainer reaction.

Reviews of unreproducible packages

55 package reviews have been added, 161 have been updated and 136 have been removed in this week, adding to our knowledge about identified issues.

2 issue types have been updated:

• Added user_in_documentation_generated_by_gsdoc
• Added locale_differences_in_pom

Weekly QA work

FTBFS bugs have been reported by:

• Chris Lamb (16)
• Santiago Vila (2)

diffoscope development

Chris Lamb, Holger Levsen and Mattia Rizzolo worked on diffoscope this week.

Improvements were made to SquashFS and JSON comparison, the https://try.diffoscope.org/ web service, documentation, packaging, and general code quality.

diffoscope 57, 58, and 59 were uploaded to unstable by Chris Lamb. Versions 57 and 58 were both broken, so Holger set up a job on jenkins.debian.net to test diffoscope on each git commit. He also wrote a CONTRIBUTING document to help prevent this from happening in future.

From these efforts, we were also able to learn that diffoscope is now reproducible even when built across multiple architectures:

< h01ger> | https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/diffoscope.html shows these packages were built on amd64:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> | and on i386:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb
< h01ger> | and on armhf:
< h01ger> |  bd21db708fe91c01ba1c9cb35b9d41a7c9b0db2b 62288 diffoscope_59_all.deb
< h01ger> |  366200bf2841136a4c8f8c30bdc87057d59a4cdd 20146 trydiffoscope_59_all.deb

And those also match the binaries uploaded by Chris in his diffoscope 59 binary upload to ftp.debian.org, yay! Eating our own dogfood and enjoying it!

tests.reproducible-builds.org

Debian related:

• show percentage of results in the last 24/48h (h01ger)
• switch python database backend to SQLAlchemy (Valerie)
• vary build path varitation for unstable and experimental for all architectures. (h01ger)

The last change probably will have an impact you will see: your package might become unreproducible in unstable and this will be shown on tracker.debian.org, while it will still be reproducible in testing.

We’ve done this, because we think reproducible builds are possible with arbitrary build paths. But: we don’t think those are a realistic goal for stretch, where we still recommend to use ´.buildinfo´ to record the build patch and then do rebuilds using that path.

We are doing this, because besides doing theoretical groundwork we also have a practical goal: enable users to independently verify builds. And if they only can do this with a fixed path, so be it. For now :)

To be clear: for Stretch we recommend that reproducible builds are done in the same build path as the “original” build.

Finally, and just for our future references, when we enabled build path variation on Saturday, August 20th 2016, the numbers for unstable were:

Misc.

Ximin Luo updated our git setup scripts to make it easier for people to write proper descriptions for our repositories.

This week’s edition was written by Ximin Luo and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.