# Rebuilding the three raw-run archives

*"Cancer Research: The Whole Arc", public edition 1. Guide version 1, written October 16, 2026.*

The records shared with this edition don't include three raw-run archives (NumPy `.npz` files). This guide says how
to rebuild each of them from public files only, with the scripts that made it, and how to check the result. Its tests
were partial. "What the tests showed" sets them out array by array: what reproduced exactly, what differed and why,
and which full rebuilds were not run.

| Archive | Holds | What the tests showed (October 16, 2026) |
|---|---|---|
| `coherence-growth-2026-10-11/sim_replication4000.npz` | Test 17's replication: starts 13 to 18 | Rebuilt in full: all 12 compared arrays exactly the same |
| `coherence-growth-2026-10-11/sim_fresh4000.npz` | Test 17's test: starts 7 to 12 | Rebuilt in full: 10 of the 12 compared arrays exactly the same; `z` and `logs` differ |
| `cause-test-2026-10-12/sim_cause4000.npz` | The Cause Test: starts 19 to 60, 42 base runs and 420 moments | Not rebuilt in full. In the parts rebuilt, 7 of the 42 base runs and 1 of the 420 moments, every array compared was exactly the same |

No rebuild matches its original byte for byte, because each archive also stores how many seconds every run took.
The check (step 6) therefore compares the arrays one by one, leaving those timings out.

## What the tests showed

A compared array either has the original's type, shape and SHA-256 fingerprint (step 6) or it doesn't. All the tests
ran on one machine, described under "What you need".

**Reproduced exactly**

- The replication archive, rebuilt in full: all 12 compared arrays, `starts`, `n`, `k`, `dt`, `settle`, `run`,
  `every`, `gap`, `seed`, `z`, `logs` and `trace`.
- The test archive, rebuilt in full: 10 of its 12 compared arrays, `starts`, `n`, `k`, `dt`, `settle`, `run`,
  `every`, `gap`, `seed` and `trace`.
- The Cause Test archive, in part only. For starts 19 to 25 (its first piece, 7 of the 42 base runs): their rows of
  `z`, `logs` and `trace`. For one moment (start 19, moment 1; 1 of the 420): its entries in all 33 of the archive's
  moment arrays, `R0`, `psi0`, `z2_0`, `z3_0`, `z4_0`, `C0`, `ab0`, `gap0`, `R1`, `psi1`, `z2_1`, `z3_1`, `z4_1`,
  `ratio`, `inband`, `C1`, `ab1`, `sumsq`, `moved`, `largest`, `rms`, `gap1`, `gaprel`, `amp`, `status`, `built`,
  `same`, `match`, `carried`, `code_error`, `reason`, `branch_z` and `branch_logs`.

**Different, because the original run's interruption and restart points weren't recorded**

- The test archive's `z` (coherence), which first differs at t = 1,527 to 2,812, depending on the start, and its
  `logs` (gap growth), which first differs at t = 320 to 1,985.

The original run of starts 7 to 12 was made in pieces of at most 230 seconds each (the setting in `nudge_test.py`),
each piece resuming where the last one stopped, and the points where they stopped weren't recorded. A rebuild runs
each start without stopping, as the replication originally ran. Its numbers part from the original's at different
times for different starts, consistent with each resumption having changed the last binary digits; in this chaotic
model such a difference grows. A rebuild is an equally valid run of the same model, settings and seeds, but its
later numbers differ from the original's, so results computed from it differ somewhat too.

**Not run**

- The full rebuild of the Cause Test archive: its other 35 pieces (the base runs of starts 26 to 60, and the 30
  pieces that run the 420 moments; of the moments, only start 19's first was run, on its own), the assembly of
  `sim_cause4000.npz`, and the check of it. Of the archive's 48 compared arrays, 12 haven't been compared at all:
  `starts`, `T`, and the settings `n`, `k`, `dt`, `settle`, `run`, `every`, `gap`, `seed`, `draws` and `after`. The
  other 36 have been compared only in the parts listed above.
- Any rebuild on another machine, operating system, or version of Python or NumPy.

**Never compared:** the timing arrays, `seconds` in the two Test 17 archives and `base_seconds` and `moment_seconds`
in the Cause Test's. They hold the seconds each run took, which differ from run to run.

This is partial verification. Exact reproduction is shown only where it is listed above. It is not shown for the
Cause Test archive as a whole, and the test archive's `z` and `logs` did not reproduce.

## What you need

- **Linux.** The scripts run their workers in Python process pools, which Linux starts by forking. Other systems are
  untested.
- **Python 3.11** (tested: 3.11.14) and **NumPy** (tested: 2.4.3 from PyPI, for example
  `python3 -m pip install numpy==2.4.3`; that command wasn't run for this guide). Everything else is in Python's
  standard library. Its `zoneinfo` needs the time-zone database, which most Linux systems have; otherwise
  `python3 -m pip install tzdata`. Matplotlib and SciPy aren't needed: the scripts use them only for reports and
  figures, which a rebuild never runs.
- **For exact matches like those above:** the same NumPy version, on a processor with the same vector instructions.
  The tests ran on an Intel Xeon Platinum 8581C (x86-64, 8 logical processors, AVX-512), where NumPy's bundled
  OpenBLAS 0.3.31.dev chose its SkylakeX code. Another NumPy version or processor can change the last binary digits
  of some results; the difference then grows, and the arrays no longer match the fingerprints, although the run is
  still a valid one.
- **No credentials, no private files and no network access** once the files are downloaded.

| Archive | Processes | Time, 8 logical processors | Memory | Disk |
|---|---|---|---|---|
| Replication | 6 at once | 139 s (tested) | about 45 MB per process (tested) | 0.6 MB |
| Test | 6 at once | 156 s (tested) | about 45 MB per process (tested) | 0.6 MB |
| Cause Test | 7 workers | not run in full; estimated from the original's run log (base runs 144 to 267 s each, moments 62 to 96 s each) at about 10.2 processor-hours in all, so roughly 1.5 to 2 hours | under 100 MB per worker (tested: 46 MB in a base run, 71 MB in a moment) | about 140 MB of temporary parts, removed at the end; the archive itself is 25 MB |

The processes start whatever the number of processors, so with fewer than 8 the times grow in proportion.

## 1. Get the files

Make one folder, for example `rebuild/`, laid out like this:

```
rebuild/
  rebuild_archives.py                       this guide's helper
  inputs.sha256                             the list below, saved as a file
  nudge-test-2026-10-10/nudge_test.py       from the Spiral Staircase page
  coherence-growth-2026-10-11/              10 records
  cause-test-2026-10-12/                    10 records
```

- **`nudge_test.py`:** on the Spiral Staircase page, under Memory in Motion, each of Tests 11 to 16 offers it as
  "Download the script" (125,301 bytes).
- **The 20 records:** from the Records section of the Cancer Research page (`the-spiral-staircase/cancer-research` on
  the site), folders `coherence-growth-2026-10-11` and `cause-test-2026-10-12`. Each record's address ends in
  `research-packets/cancer-research-arc/`, then its folder and file name. The folders may hold the other records too
  (run logs, reports and so on); the helper never changes them.
- **`rebuild_archives.py`:** published with this guide (20,265 bytes, SHA-256 `cd61f319a03e3dd9048791e38230accff962e7c623c8a1a6a0afe604f68b1e61`).

`inputs.sha256`, the 21 input files in the format of `sha256sum`:

```
b06730e7772be1f719cadae7924cdca3c2e0ebb89efb17104969363431f782e7  nudge-test-2026-10-10/nudge_test.py
7738b55b8a3757f7fb186e11611251de0f5eddc8a6a746f23c9d51784e3b5b3a  coherence-growth-2026-10-11/coherence_growth.py
a5b5eab338d2e473097b707c562998c3b6e9eb7ceb450e5e31e4c5d11db032c8  coherence-growth-2026-10-11/earlier_growth_check.py
99381ca8c95a4e17b3c17cf01acc580c6203f15242b4e1c9443b50bf78253c0e  coherence-growth-2026-10-11/replication.py
b36bd2c03ddecd9e8dc2237f06041311b6704d93a48257e2d75630cd665384c2  coherence-growth-2026-10-11/replication_lock.json
d261cd665a70bb7fb8323abd9249f40c774653fafc5d1addc9ce245de34d1a70  coherence-growth-2026-10-11/replication_prereg.md
27db2ce0a1346b5f1de198145a2793f7f531ffc6849fa53cc69d74462d05e858  coherence-growth-2026-10-11/coherence_growth.json
f3acff1e85e580d8fb20dfba210efaf8ded8011ec87df556c18653ca27992231  coherence-growth-2026-10-11/coherence_growth.txt
95e76112e2d32769e6fe5430de3d0b7d2e3a609d2ec8bd6161d74c95f21558a7  coherence-growth-2026-10-11/check.json
506ef2b72e01afcb837d88717e91d315ea9e9aa5e06091ad2372f47f3f4a7a06  coherence-growth-2026-10-11/earlier_growth_check.json
92fd74db1a8e3f655ab21ae018952aa2d6a4bba2f2c5c99ca644b0a4f3cf1e29  coherence-growth-2026-10-11/earlier_growth_check.txt
e182e564ca92c7401fc958da9698a93fdece8663eacba485e1c2f41cf69e8f54  cause-test-2026-10-12/cause_test.py
5fdd1c90df371675d1113a5eca799abdc2c639c00adbeaeb0339c05b02f487ca  cause-test-2026-10-12/cause_rule.py
103241b2fb361649f3e53cea900fedb1ee475c11200e8e8e8feeb2c2af4fecd3  cause-test-2026-10-12/design_numbers.py
39bc39762c551631327d5120b69a6b5daf27f7a6c5cdb5f06fe59a22b281af28  cause-test-2026-10-12/design_numbers.json
76d68062720f21e7c5f62ae14092e6c135ae207b036564ccb30e1a9770edca39  cause-test-2026-10-12/design_numbers.txt
1a1d72113ab035b14b9e80e413e7030291915d8d1a365ad1f658f95c277709a8  cause-test-2026-10-12/cause_moments.csv
225058b7c11159c412ced9b2f1d8c7ea48d9e48d2805b913be4bdc2278acbe19  cause-test-2026-10-12/cause_targets.json
5e3f8a8dba33a11bad8a0693e65803cfbfdb3bd83bfbaa7e868d41b02882d30c  cause-test-2026-10-12/cause_targets.txt
1c44034e64f4ec74d5afba7080cf1df1e96262680583f785447b153f3ce61112  cause-test-2026-10-12/cause_test_prereg.md
32d72d1e2a4b67a3642e3cdce0207b43e0af670683609c8d84fb1b00366da527  cause-test-2026-10-12/cause_lock.json
```

## 2. Check the files

```
cd rebuild
sha256sum -c inputs.sha256                  # tested: 21 OK
python3 -B rebuild_archives.py inputs       # tested: the same check, built into the helper
```

## 3. Rebuild the replication archive

```
python3 -B rebuild_archives.py replication  # tested: 139 s
```

Writes `coherence-growth-2026-10-11/sim_replication4000.npz` and its own run log,
`coherence-growth-2026-10-11/rebuild_replication_runlog.json`.

## 4. Rebuild the test archive

```
python3 -B rebuild_archives.py fresh        # tested: 156 s
```

Writes `coherence-growth-2026-10-11/sim_fresh4000.npz`. Its `z` and `logs` won't match the original's (see "What
the tests showed").

## 5. Rebuild the Cause Test archive

```
python3 -B rebuild_archives.py cause        # tested in part only, as "cause 1" (below)
```

Runs the Cause Test's 462 tasks in the script's own pieces (7 base runs or 14 moments each; 36 pieces) until all are
done, then assembles `cause-test-2026-10-12/sim_cause4000.npz` and removes the temporary parts. Its run log is
`cause-test-2026-10-12/rebuild_cause_runlog.json`. If it is stopped, the same command carries on from the last task
finished. `cause 5` runs at most 5 pieces, then stops.

Tested: the first piece (`cause 1`: 7 base runs, starts 19 to 25, 288 s), whose rows of `z`, `logs` and `trace` equal
the original's; and one moment (start 19, moment 1), run on its own through the same task function, whose entries in
all 33 moment arrays equal the original's. Not run: the other 35 pieces, the assembly and the check of the archive;
their code was only read.

## 6. Check the rebuilt archives

```
python3 -B rebuild_archives.py verify coherence-growth-2026-10-11/sim_replication4000.npz   # tested: 12 of 12
python3 -B rebuild_archives.py verify coherence-growth-2026-10-11/sim_fresh4000.npz         # tested: 10 of 12
python3 -B rebuild_archives.py verify cause-test-2026-10-12/sim_cause4000.npz               # not run
```

For every array, `verify` compares the type, the shape and the SHA-256 of the array's bytes in C order with the
original's, which the helper carries, and marks it SAME or DIFFERENT. The timing arrays (`seconds`, `base_seconds`,
`moment_seconds`) are listed but not compared. It exits with status 0 only if every compared array is the same.

The same fingerprint for one array, in Python (this prints the replication's `z` and its SHA-256, `51a8e4f6…`):

```python
import hashlib
import numpy as np
with np.load("coherence-growth-2026-10-11/sim_replication4000.npz") as data:
    a = data["z"]
    print(a.dtype.str, a.shape, hashlib.sha256(np.ascontiguousarray(a).tobytes()).hexdigest())
```

The original archives, whole:

```
2ebd7e66313d8ff2710f5aebaa2c8cb5fdc1ddc16cb3d767dd0fe75d27c61e0c  sim_replication4000.npz  (592,435 bytes)
541a2f8009d6c97c805faf12a1ce8aef4a34dfebb692a8ad1ca0cbd650491218  sim_cause4000.npz  (25,074,738 bytes)
a008e1c61a9ef4215a6b40b867d1b577e947bd0dfa296ab2a6676c204bb11ca0  sim_fresh4000.npz  (592,016 bytes)
```

## What the helper changes

Each command imports the original script, unchanged, and calls the script's own run function. The helper changes
three things, in memory only, never in a file:

1. **The lock check.** The replication and the Cause Test refuse to run unless every file their lock covers is
   unchanged, and some of those files aren't public (raw runs from Tests 16 and 17, Test 16's results file and two
   figures). The helper checks the settings and every covered file that is public, 10 of 15 for the replication and
   12 of 16 for the Cause Test, against the script's own lock file, and stops if any differs. It then hands the script
   that checked lock.
2. **The run logs** go to new files (`rebuild_replication_runlog.json`, `rebuild_cause_runlog.json`), so the records'
   own run logs stay as they are.
3. **No stopping** for the test and the replication: every start runs to the end in one go (the 230-second limit is
   lifted). The helper won't start if a progress folder (`parts/` or `replication_parts/`) is left from an earlier
   run; remove the folder and run the command again.

The Cause Test keeps its own pieces. Each of its tasks runs whole, in one go (a moment starts from the state its base
run saved), so the pieces aren't expected to change any numbers; that was checked only for the parts tested.

## Seeds

All are fixed in the scripts; there is nothing to set. For start number s:

- Starting phases: `numpy.random.default_rng(s - 1)`.
- The tiny gap between the two copies: `default_rng((20261011, s - 1))`.
- The Cause Test's ten moments: one offset per start from `default_rng((20261012, s - 1))`. The moments are also
  listed in `cause_moments.csv`.
- The Cause Test's changes at moment m: `default_rng((20261012, s - 1, m - 1, use))`, for use 0, 1 and 2.
- 20261015 seeds the resampling for the analyses' error bars; a rebuild doesn't use it.

## Every step, tested or not

| Step | Status |
|---|---|
| Installing NumPy 2.4.3 | Not run: the test machine already had it |
| `sha256sum -c inputs.sha256`, `inputs` | Tested |
| `replication`, then `verify` | Tested: 139 s, 12 of 12 arrays the same |
| `fresh`, then `verify` | Tested: 156 s, 10 of 12 the same; `z` and `logs` differ |
| `cause`, first piece | Tested as `cause 1`: 288 s; the 7 base runs' `z`, `logs` and `trace` equal the original's |
| One moment task | Tested through the script's task function: its entries in all 33 moment arrays equal the original's |
| `cause`, the other 35 pieces and the assembly | Not run; the code was only read (about 10 processor-hours) |
| `verify` on the Cause Test archive | Not run |
| The progress-folder refusal | Tested |
| Other systems, Python or NumPy versions, processors | Not run |