Stable order for inputs

If building your software requires processing several inputs at once, make sure the order is stable across builds.

A typical example is creating an archive from the content of a directory. Most filesystems do not guarantee that listing files in a directory will always result in the same order.

Example Makefile

The following Makefile will result in unreproducible builds¹:

SRCS = $(wildcard *.c)
tool: $(SRCS:.c=.o)
	$(CC) -o $@ $^

Solutions:

a) List all inputs explicitly and ensure they will be processed in that order.

SRCS = util.c helper.c main.c
tool: $(SRCS:.c=.o)
	$(CC) -o $@ $^

b) Sort inputs:

SRCS = $(sort $(wildcard *.c))
tool: $(SRCS:.c=.o)
	$(CC) -o $@ $^

Watch out for locale-related issues

When sorting inputs, one must ensure that the sorting order is not affected by the system locale settings. Some locales will not distinguish between uppercase and lowercase characters.

For example, tar will by default use the filesystem order when descending directories:

$ tar -cf archive.tar src

A solution is to use find and sort but the following might still have differences when run under different locales:

$ find src -print0 | sort -z |
    tar --no-recursion --null -T - -cf archive.tar

The locale used to sort files must be specified to avoid any surprises:

$ find src -print0 | LC_ALL=C sort -z |
    tar --no-recursion --null -T - -cf archive.tar

This might not be the only change required for Tar and other archive formats as they usually embed more metadata problems.

← Documentation index

Documentation home

Introduction

• Definitions
• History
• Buy-in
• Making plans
• Academic publications

Achieve deterministic builds

• Commandments of reproducible builds
• Variations in the build environment
• SOURCE_DATE_EPOCH
• Deterministic build systems
• Volatile inputs can disappear
• Stable order for inputs
• Stripping of unreproducible information
• Value initialization
• Version information
• Timestamps
• Timezones
• Locales
• Archive metadata
• Stable order for outputs
• Randomness
• Build path
• System images
• JVM

Define a build environment

• What's in a build environment?
• Recording the build environment
• Definition strategies
• Proprietary operating systems

Distribute the environment

• Building from source
• Virtual machine drivers
• Formal definition

Verification

• Cryptographic checksums
• Embedded signatures
• Sharing certifications

Specifications

• SOURCE_DATE_EPOCH
• BUILD_PATH_PREFIX_MAP (WIP)