Reproducible Builds in August 2025

← View all our monthly reports


Welcome to the August 2025 report from the Reproducible Builds project!

Welcome to the latest report from the Reproducible Builds project for August 2025. These monthly reports outline what we’ve been up to over the past month, and highlight items of news from elsewhere in the increasingly-important area of software supply-chain security. If you are interested in contributing to the Reproducible Builds project, please see the Contribute page on our website.

In this report:

  1. Reproducible Builds Summit 2025
  2. Reproducible Builds and live-bootstrap at WHY2025
  3. DALEQ Explainable Equivalence for Java Bytecode
  4. Reproducibility regression identifies issue with AppArmor security policies
  5. Rust toolchain fixes
  6. Distribution work
  7. diffoscope
  8. Website updates
  9. Reproducibility testing framework
  10. Upstream patches

Reproducible Builds Summit 2025

Please join us at the upcoming Reproducible Builds Summit, set to take place from October 28th — 30th 2025 in Vienna, Austria!**

We are thrilled to host the eighth edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin, Hamburg and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort.

During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving.

If you’re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. Registration is open until 20th September 2025, and we are very much looking forward to seeing many readers of these reports there!


Reproducible Builds and live-bootstrap at WHY2025

WHY2025 (What Hackers Yearn) is a nonprofit outdoors hacker camp that takes place in Geestmerambacht in the Netherlands (approximately 40km north of Amsterdam). The event is “organised for and by volunteers from the worldwide hacker community, and knowledge sharing, technological advancement, experimentation, connecting with your hacker peers, forging friendships and hacking are at the core of this event”.

At this year’s event, Frans Faase gave a talk on live-bootstrap, an attempt to “provide a reproducible, automatic, complete end-to-end bootstrap from a minimal number of binary seeds to a supported fully functioning operating system”.

Frans’ talk is available to watch on video and his slides are available as well.


DALEQ Explainable Equivalence for Java Bytecode

Jens Dietrich of the Victoria University of Wellington, New Zealand and Behnaz Hassanshahi of Oracle Labs, Australia published an article this month entitled DALEQ — Explainable Equivalence for Java Bytecode which explores the options and difficulties when Java binaries are not identical despite being from the same sources, and what avenues are available for proving equivalence despite the lack of bitwise correlation:

[Java] binaries are often not bitwise identical; however, in most cases, the differences can be attributed to variations in the build environment, and the binaries can still be considered equivalent. Establishing such equivalence, however, is a labor-intensive and error-prone process.

Jens and Behnaz therefore propose a tool called DALEQ, which:

disassembles Java byte code into a relational database, and can normalise this database by applying Datalog rules. Those databases can then be used to infer equivalence between two classes. Notably, equivalence statements are accompanied with Datalog proofs recording the normalisation process. We demonstrate the impact of DALEQ in an industrial context through a large-scale evaluation involving 2,714 pairs of jars, comprising 265,690 class pairs. In this evaluation, DALEQ is compared to two existing bytecode transformation tools. Our findings reveal a significant reduction in the manual effort required to assess non-bitwise equivalent artifacts, which would otherwise demand intensive human inspection. Furthermore, the results show that DALEQ outperforms existing tools by identifying more artifacts rebuilt from the same code as equivalent, even when no behavioral differences are present.

Jens also posted this news to our mailing list.


Reproducibility regression identifies issue with AppArmor security policies

Tails developer intrigeri has tracked and followed a reproducibility regression in the generation of AppArmor policy caches, and has identified an issue with the 4.1.0 version of AppArmor.

Although initially tracked on the Tails issue tracker, intrigeri filed an issue on the upstream bug tracker. AppArmor developer John Johansen replied, confirming that they can reproduce the issue and went to work on a draft patch. Through this, John revealed that it was caused by an actual underlying security bug in AppArmor — that is to say, it resulted in permissions not (always) matching what the policy intends and, crucially, not merely a cache reproducibility issue.

Work on the fix is ongoing at time of writing.


Rust toolchain fixes

Rust Clippy is a linting tool for the Rust programming language. It provides a collection of lints (rules) designed to identify common mistakes, stylistic issues, potential performance problems and unidiomatic code patterns in Rust projects. This month, however, Sosthène Guédon filed a new issue in the GitHub requesting a new check that “would lint against non deterministic operations in proc-macros, such as iterating over a HashMap”.


Distribution work

In Debian this month:

Lastly, Bernhard M. Wiedemann posted another openSUSE monthly update for their work there.


diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb made the following changes, including preparing and uploading versions, 303, 304 and 305 to Debian:

  • Improvements:

    • Use sed(1) backreferences when generating debian/tests/control to avoid duplicating ourselves. […]
    • Move from a mono-utils dependency to versioned mono-devel | mono-utils dependency, taking care to maintain the [!riscv64] architecture restriction. […]
    • Use sed over awk to avoid mangling dependency lines containing = (equals) symbols such as version restrictions. […]

  • Bug fixes:

    • Fix a test after the upload of systemd-ukify version 258~rc3. […]
    • Ensure that Java class files are named .class on the filesystem before passing them to javap(1). […]
    • Do not run jsondiff on files over 100KiB as the algorithm runs in O(n^2) time. […]
    • Don’t check for PyPDF version 3 specifically; check for >= 3. […]

  • Misc:

    • Update copyright years. […][…]

In addition, Martin Joerg fixed an issue with the HTML presenter to avoid crash when page limit is None […] and Zbigniew Jędrzejewski-Szmek fixed compatibility with RPM 6 […]. Lastly, John Sirois fixed a missing requests dependency in the trydiffoscope tool. […]


Website updates

Once again, there were a number of improvements made to our website this month including:


Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In August, however, a number of changes were made by Holger Levsen, including:

  • reproduce.debian.net-related:

    • Run 4 workers on the o4 node again in order to speed up testing. […][…][…][…]
    • Also test trixie-proposed-updates and trixie-updates etc. […][…]
    • Gather seperate statistics for each tested release. […]
    • Support sources from all Debian suites. […]
    • Run new code from the prototype database rework branch for the amd64-pull184 pseudo-architecture. […][…]
    • Add a number of helpful links. […][…][…][…][…][…][…][…][…]
    • Temporarily call debrebuild without the --cache argument to experiment with a new version of devscripts. […][…][…]
    • Update public TODO. […]

  • Installation tests:

    • Add comments to explain structure. […]
    • Mark more old jobs as old or “dead”. […][…][…]
    • Turn the maintenance job into a no-op. […]

  • Jenkins node maintenance:

    • Increase penalties if the osuosl5 or ionos7 nodes are down. […]
    • Stop trying to fix network automatically. […]
    • Correctly mark ppc64el architecture nodes when down. […]
    • Upgrade the remaining arm64 nodes to Debian trixie in anticipation of the release. […][…]
    • Allow higher SSD temperatures on the riscv64 architecture. […]

  • Debian-related:

    • Drop the armhf architecture; many thanks to Vagrant for physically hosting the nodes for ten years. […][…]
    • Add Debian forky, and archive bullseye. […][…][…][…][…][…][…]
    • Document the filesystem space savings from dropping the armhf architecture. […]
    • Exclude i386 and armhfr from JSON results. […]
    • Update TODOs for when Debian trixie and forky have been released. […][…]

  • tests.reproducible-builds.org-related:

  • Misc:

    • Detect errors with openQA erroring out. […]
    • Drop the long-disabled openwrt_rebuilder jobs. […]
    • Use qa-jenkins-dev@alioth-lists.debian.net as the contact for jenkins.debian.net. […]

    • Redirect reproducible-builds.org/vienna25 to reproducible-builds.org/vienna2025. […]

    • Disable all OpenWrt reproducible CI jobs, in coordination with the OpenWrt community. […][…]
    • Make reproduce.debian.net accessable via IPv6. […]
    • Ignore that the megacli RAID controller requires packages from Debian bookworm. […]

In addition,

  • James Addison migrated away from deprecated toplevel deb822 Python module in favour of debian.deb822 in the bin/reproducible_scheduler.py script […] and removed a note on reproduce.debian.net note after the release of Debian trixie […].

  • Jochen Sprickerhof made a huge number of improvements to the reproduce.debian.net statistics calculation […][…][…][…][…][…] as well as to the reproduce.debian.net service more generally […][…][…][…][…][…][…][…].

  • Mattia Rizzolo performed a lot of work migrating scripts to SQLAlchemy version 2.0 […][…][…][…][…][…] in addition to making some changes to the way openSUSE reproducibility tests are handled internally. […]

  • Lastly, Roland Clobus updated the Debian Live packages after the release of Debian trixie. […][…]


Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:



Finally, if you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:



← View all our monthly reports