brightway-benchmarks

Only the newest release gets fixes (SECURITY.md, "Supported versions"). Upgrading is checking out the new tag and running uv sync, and the CHANGELOG lists what changed in each version, with upgrade steps first when there are any. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/SECURITY.md#supported-versions

I track bugs and requests in GitHub issues. URL: https://github.com/simonvanlierde/brightway-benchmarks/issues

No vulnerability reports came in during the last 12 months, on the tracker or elsewhere. SECURITY.md promises credit unless the reporter prefers not to be named. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/SECURITY.md#fixes-and-disclosure

SECURITY.md documents it: a private advisory or email, an acknowledgement within 14 days saying whether I can reproduce it and when I expect a fix, then the published advisory with a CVE where one applies, linked from the CHANGELOG. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/SECURITY.md

The standard is ruff with every rule selected (select = ["ALL"]) and seven named exceptions, plus ruff's formatter, ty for types, and shellcheck, yamllint, actionlint, zizmor and rumdl through the git hooks. CONTRIBUTING.md requires just check and the hooks to pass. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/pyproject.toml

The required lint, types, tests job runs ruff, the ruff format check and ty, and the required git hooks job runs every hook over the repository. The protect-main ruleset blocks a merge until both pass. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/workflows/python.yml

Pure Python with no compiled code: the uv_build backend makes the source archive and wheel, and there is no compiler or linker step to pass variables to.

Pure Python: there are no binaries to strip, and the installed package is the source.

No make or other recursive build. One uv invocation builds the package.

The release file is git archive of the tagged commit, which is the same bytes for the same commit. The benchmark box is built from a base image pinned by digest, and each benchmarked release runs on a stack pinned in asv/<package>.stack.txt. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/workflows/release.yml

Clone the repository or unpack a release, then uv sync, Python's usual way to install a locked project; removing the folder uninstalls it. The benchmarks themselves run in Docker through just, and nothing is installed outside the project folder. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#set-up

uv installs into the project's virtual environment and honours UV_PROJECT_ENVIRONMENT and the other standard uv and pip variables; the project adds no installer of its own. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#set-up

CONTRIBUTING.md "Set up": install uv, just and Docker, run uv sync and prek install, then just check. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#set-up

pyproject.toml lists the direct dependencies and uv.lock pins the full tree. The stack each benchmarked package runs on is pinned in asv/<package>.stack.txt, and each run records the versions it installed. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/uv.lock

uv audit checks every locked dependency on each pull request, dependency review blocks a pull request that adds a vulnerable one, and Dependabot alerts are on with none open (2026-10-28). Renovate proposes updates weekly. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/workflows/python.yml

Every reused component is a package in uv.lock, an action pinned by commit, or the base image pinned by digest, so one version bump updates it, and Renovate opens that bump. Nothing is copied in. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/renovate.json5

ruff's ALL selection includes the pyupgrade and NumPy deprecation rules, so a deprecated API fails the lint job. Renovate keeps the dependencies current. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/pyproject.toml

Every pull request and push to main runs the pytest suite (181 tests, covering the generator, the box scripts and the result tools), and main requires it. smoke runs every benchmark in the box weekly, compat runs today's Brightway releases together nightly on Linux, macOS and Windows, and codspeed runs the benchmark suites on each pull request that touches them. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/workflows/python.yml

30 of the 41 fix commits since the project started (2026-10-13) changed tests/, 73%, for example #74, #73 and #57. Most of the rest fix the docs or the site. CONTRIBUTING.md requires a test that reproduces each bug. URL: https://github.com/simonvanlierde/brightway-benchmarks/pull/74

88% coverage on main (statements and branches, src/ and scripts/, the scripts run as subprocesses included), and the required CI job fails below 80% (fail_under). Three scripts that need Brightway run only in the box, where just smoke exercises them, and are left out. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/pyproject.toml

CONTRIBUTING.md: "A change in behaviour comes with a test that fails without it, and a bug fix with a test that reproduces the bug." URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#tests

The same sentence, in the instructions for contributions. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#tests

ruff selects every rule with seven named exceptions, ty checks types, and the required CI job fails on any warning from either. CONTRIBUTING.md asks to fix a warning or silence it on that line with the rule's name and a reason. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/pyproject.toml

docs/assurance-case.md shows the principles at work: least privilege (Brightway's code only in a box with no capabilities and a read-only repository, read-only workflow tokens), fail-safe defaults (no timing without a passing reference check, a symlink in box output stops the script), complete mediation, separation of privilege between measuring and publishing, and open design. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/docs/assurance-case.md#secure-design-principles

The project implements no cryptography. TLS, signing and key generation come from GitHub, Sigstore and OpenSSH at their current defaults, with no SHA-1 or other weak algorithm chosen by the project.

The project implements no cryptography. The attestations and signed tags use Sigstore and OpenSSH, which can change algorithm without a change here.

The project stores and processes no credentials or private keys. The workflows use GitHub's short-lived token, and CodSpeed an OIDC token.

The project's own code opens no network connection. uv, pip and asv download packages over HTTPS at their defaults.

The project implements no TLS. Where it uses TLS (GitHub, PyPI and conda-forge downloads through uv, pip and asv), those tools negotiate TLS 1.2 or later at their defaults.

The project makes no TLS connections of its own. uv, pip and asv verify certificates by default, and nothing here turns that off.

The project sends no private information. The only credentials in use are GitHub's workflow token and an OIDC token, sent over TLS with verified certificates by GitHub's own tooling.

Each release's source archive has a Sigstore build provenance attestation and is listed in SHA256SUMS, and CONTRIBUTING.md says how to check it with gh attestation verify. The release tag itself is signed. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/.github/CONTRIBUTING.md#releasing

Each release tag is an annotated tag signed with my SSH key (git tag -s, CONTRIBUTING.md "Releasing"). v0.1.0 verifies with git tag -v, and GitHub shows it as verified. URL: https://github.com/simonvanlierde/brightway-benchmarks/tags

The untrusted input is what the benchmark box writes, since it runs released code. The host scripts that read it refuse symlinks and paths outside their folder, copy only result records, and escape every name before it goes into a page. Workflow inputs reach scripts through environment variables, never expanded into the shell. The assurance case lists each boundary. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/docs/assurance-case.md#trust-boundaries

The benchmark box drops every capability, sets no-new-privileges, mounts the repository read-only, limits processes, memory and CPUs, and gets a new home folder each run. Workflows have read-only tokens, actions are pinned by commit and the base image by digest, and zizmor checks the workflows. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/scripts/box.sh

docs/assurance-case.md: the requirements, the threat model, a table of trust boundaries with the controls at each, the secure design principles applied, and the common weaknesses countered with their CWE numbers, each tied to a script, workflow or test. URL: https://github.com/simonvanlierde/brightway-benchmarks/blob/main/docs/assurance-case.md

CodeQL (GitHub's default setup) analyses the Python code and the workflows, ruff runs with the bandit (S) security rules as part of ALL, zizmor checks the workflows for injection and over-broad tokens, and uv audit checks the dependencies. URL: https://github.com/simonvanlierde/brightway-benchmarks/security/code-scanning

All the project's code is Python, which is memory-safe.