Collaborative Working Sessions - Metrics

Reproducible Builds Summit 2022

Notes

We defined 4 columns:

• distributions ( ex. debian )
• projects ( ex. tor browser )
• compilers ( ex. gcc )
• ecosystem ( ex. go libraries ) this is of course related to a language

For each column we defined :

• qualitative metrics QL ( yes / no / maybe ) that can be manually sourced from a website ( with citation ) and updated from time to time.
• quantitative metrics QN ( a number ) that should be scraped from an external source and updated regularly and automatically

For distributions:

• QN number of reproducible packages / vs n. of total packages ( for each release )
• QN number of reproducible packages / vs n. of total packages ( for each arch )
• QL does the distribution has a reproducible base system ( bootstrap )
• QL what kind of reproducibility test is used ?
• QN how many “reproducible friendly compilers” are shipped by this distro ?

For Projects:

• QL does the project ships reproducible binanries / images / things ?
• QN how many of the project dependencies are reproducible ?
• what kind of reproducible test is used ?

For Compilers:

• QL does the compiler repruducibly bootstrap itself
• QL does the compiler produce reproducible binaries ( under defined cirmumstances )

For ecosystems:

• QN How many reproducible packages / total number of installable packages ?

yaml file formats:

Compiler yaml file format:

name of the compiler: string
language: string
build systems: [ string ]
reproducible binaries on differen archs: [string, bool]
bootstrap reproducibly: bool
compile reproducible binaries: bool

Distribution yaml file format:

name of the distribution: string
type of reproducibility (reprotest/rebuild): string
reproducible status: [architecture]
reproducible status of build essential dependencies: [architecture]

architecture:

name: string
reproducible: float
unreproducible: float
unknown: float

Examples:

name of the compiler: ocamlc
language: ocaml
build systems:
    - dune
    - opam
 reproducible binaries on differen archs: 
    - amd64: yes
    - arm64: yes
bootstrap reproducibly: yes
compile reproducible binaries: true

name of the distribution: alpine
type of reproducibility (reprotest/rebuild/none): none
reproducible status: []
reproducible status of build essential dependencies: []