For AI agents: this page as Markdown · llms.txt · JSON Schema

Environment matrix

The environment matrix installs a Rerail package and checks it as a user would meet it: Linux and Windows on throwaway Azure VMs, macOS on GitHub-hosted runners.

CI (.github/workflows/env-matrix.yml) runs it after a push to main that changes the matrix, or what core ships per scripts/release/core-paths.mjs (the Release pull request's rule). Pack, test, docs or other CI changes leave the package as the last run tested it. just rerail-env-matrix runs the Azure environments by hand.

Environments

scripts/host/azure/environments.json lists the environments and what each is expected to do.

Environment Image Expected
ubuntu-24.04-x64, ubuntu-24.04-arm64 Ubuntu Server 24.04 LTS pass
debian-13-x64, debian-13-arm64 Debian 13 pass
ubuntu-22.04-x64 Ubuntu Server 22.04 LTS pass
debian-12-x64 Debian 12 pass
rhel-9-x64 Red Hat Enterprise Linux 9 pass
windows-11-x64, windows-11-arm64 Windows 11 Pro 25H2 pass
macos-26-arm64, macos-15-arm64 GitHub-hosted runner macos-26, macos-15 (Apple silicon) pass
macos-15-x64 GitHub-hosted runner macos-15-intel pass
Glibc support and expected failures

Linux engines target glibc 2.17, below the 2.28 Node 24 itself needs, so any glibc distribution running Node 24 should pass.

A known set-up failure is marked "expect": "setup-fails", with the printed text (failsWith) and reason (because). It passes while set-up fails that way. Once set-up succeeds it fails: support has changed, so switch the entry to "expect": "pass".

macOS on GitHub-hosted runners

Azure has no Macs, so macOS environments run the Linux guest, guest.sh, on a GitHub-hosted runner via matrix.mjs local.

How the runner becomes a clean Mac

A macOS environment has "os": "macos" plus a "runner" label to run on; the guest runs on it with sudo.

A runner is not a clean Mac: it carries Xcode, Homebrew and other Nodes. The guest makes zsh the account's shell and installs Node from nodejs.org's installer into /usr/local/bin, as a developer would. The facts record which Node the user's shell runs.

local refuses to run anywhere but a GitHub-hosted runner, which is discarded after the job. Without --env, the runner it is on is the environment, expected to pass. The release workflow installs every release this way on ubuntu-24.04 (Linux x64) and macos-15 (macOS arm64).

Node and Git

Each environment installs Node and Git the way a user would.

What each environment checks

Eight checks run as the ordinary user, never root or SYSTEM, so set-up writes PATH as on a person's computer.

  1. registry: a registry on the VM serves the package and every published rule pack in packs/; all else comes from npmjs.
  2. project: a Git repository with a package.json and one commit, plus empty agent folders.
  3. setup: npx @neralabs/rerail setup --json, answered until Rerail is on.
  4. status: rerail status shows the daemon running with Engine: ok.
  5. PATH: a new shell finds rerail.
  6. hooks: Claude Code's and Codex's SessionStart commands record their sessions with the daemon.
  7. daemon-restart: rerail daemon stop and rerail daemon start work.
  8. uninstall: rerail install --remove unhooks every agent and takes Rerail off PATH.
Exact pass conditions
Sessions, shells and recorded facts

Set-up writes PATH into the user's own startup files or registry hive. Linux and macOS use a login shell; Windows a signed-in desktop session (automatic sign-in, then a restart).

The PATH check uses login and interactive bash on Linux; login interactive zsh (as Terminal.app opens) and interactive zsh on macOS. On Windows, PATH is built from machine and user registry values, as a new terminal builds it.

The release workflow runs the same checks before every publish: guest.sh on Linux and macOS, and on Windows the registry, set-up, managed install, hook and uninstall steps of scripts/host/windows-host.ps1 via scripts/host/windows-install-smoke.ps1.

Facts are recorded, not judged: OS, glibc or Windows build, Node and its location, npm, Git; on Windows, also the execution policy and what npx does in a new Windows PowerShell.

Results

Each run writes <id>.json and <id>.log per environment, plus a summary.md table.

CI adds the table to the job summary (one per macOS job) and uploads the files as the env-matrix-<id> artifact. By hand, they land in .rerail/env-matrix/<run id>/.

Running it by hand

Run the Azure environments locally with just rerail-env-matrix, after az login to the subscription that holds the resource group:

sh
just rerail-env-matrix run --tarball neralabs-rerail-<version>.tgz                  # every environment
just rerail-env-matrix run --tarball <file> --env windows-11-x64,debian-12-x64      # some
just rerail-env-matrix run --npm next                                               # a published version from npmjs
just rerail-env-matrix run --tarball <file> --env ubuntu-24.04-x64 --keep           # leave the VM for a look
just rerail-env-matrix sweep --run-id <id>                                          # delete what a --keep run left

Network and clean-up

VMs accept no inbound connections and are deleted when done.

How VMs are reached and removed

Scripts reach a VM through Azure Run Command; the package and results move through blob URLs signed for three hours. Each VM's public IP carries only outbound traffic.

Each environment deletes its VM, disk, network interface and public IP when it finishes. sweep, run by CI after every run, deletes what a cancelled run left and any run container unwritten for three hours.

Staging in CI

CI stages the package once per run; each environment's job runs run --staged against it.

Re-runs and expiry

stage bundles the package and rule packs into the run's container, and re-runs of failed environments reuse it. Every attempt writes its results there, so the bundle lasts three hours past the last attempt.

Later re-runs fail with Nothing is staged in run-gh<id>: re-run all jobs instead, which stages the package again.

Windows passwords

Each Windows VM's administrator password is generated per run and readable in clear while the VM exists.

Where it is readable and why that is acceptable

With no inbound access and the VM deleted after its run, the password opens nothing. Still, limit resource group access to the people who run the matrix.

Azure setup

scripts/host/azure/bootstrap.sh creates the Azure resources once, idempotently.

Flags, resources and licensing

The script creates a resource group (rerail-envtest-rg in Canada Central by default) with a storage account whose blobs expire after seven days and a virtual network whose security group admits nothing from outside.

For CI, it creates the app registration rerail-env-matrix-github, trusted through OIDC for pushes to this repository's main, with Contributor on that resource group only.

Windows 11 images run under --license-type Windows_Client, which Azure allows when Microsoft 365 or Windows subscriptions license Windows 11 for the organisation's users.