Can an AI Stop Before It Guesses—and Keep Working?

What is included

• Original studies, review v2. Frozen prompts, synthetic fixtures, versioned code, exact API requests and responses, visible tool events, scoring and review notes. Includes the checker studies, seven pilot designs, fresh test, failures and interrupted attempts.
• Original review v1. The unchanged first archive, also preserved inside v2, so the history can be checked.
• Fresh-context AI review. Both review passes for 48 saved episodes, reviewer prompts, sealed judgments, disclosed-key comparison and disagreements. This is AI review, not independent human peer review.
• Refund clarification follow-up. A separate 18-episode study with explicit synthetic financial terms, frozen cases, raw traffic, author judgments and historical comparison.

The package also contains the closed-round manuscript, claim map, methods supplement, public-domain dedication and a file-by-file provenance record. The four research ZIPs and three reading snapshots are unchanged copies.

Read the package guide or download the historical methods and materials supplement before running anything.

Reproduce the saved analysis

Extract the ZIP and open a terminal in its top-level directory. Use Python 3.10 or later; this edition was tested with Python 3.14.3. No API key, account or dependency installation is needed. Allow roughly 200 MB of working space.

python3 verify.py
python3 reproduce.py --out ../assumption-lab-reproduced-v1

Choose an output directory that does not already exist. The command verifies hashes, reproduces the original analysis, applies the saved AI review, and recomputes the refund follow-up and comparisons. It keeps step logs, rejects Python network connections and preserves the original inputs.

See the complete reproduction instructions for individual commands, expected checks, new paid replication and design extensions. New model calls are optional and separate from the offline procedure.

What the checks establish

Passing checks establish that the saved records reconcile and their calculations can be repeated. They do not independently validate the answer keys or reviewer judgments, prove general safety, or guarantee identical outputs from a future model run.

The AI worksheet reproduces 45 operational successes in 48 episodes, with failures 19, 29 and 47. Its stricter blind-review measure gives 21 of 48; these are different criteria. The clarified follow-up reproduces 17 operational successes in 18 episodes, with an unnecessary stop still present. Explanation errors, disagreements and unsuccessful attempts remain in the record.

The cases are synthetic. Opaque encrypted response state is retained with the visible traffic; it is not readable reasoning. No model weights or credentials are supplied. Independent human review remains open.

Reuse, provenance and corrections

Renaissance Solutions dedicates its rights in this evidence edition to the public domain under CC0 1.0, the same terms used for our lab notebooks and articles. See the rights statement and full legal text. Citation is appreciated, not required.

Older files say that public access and reuse terms had not yet been settled. This edition supplies those terms while preserving the historical files. Old private Page links and local paths are provenance; the documented offline procedure uses the included files.

The source inventory records every inclusion and exclusion. Administrative closeout receipts and editable Page mirrors are omitted from this reader edition. The original master is retained unchanged, and no experimental evidence was redacted or rewritten.

After release, corrections receive a new version and change note; previous versions and checksums remain available. To challenge a finding, identify the version, case, episode and event, explain the proposed interpretation, and state how it changes the result. A feedback template is included in the original review package.

Codex — GPT-6 Astra and Mike Besso. Can an AI Stop Before It Guesses—and Keep Working? Public evidence edition v1. Renaissance Solutions, October 17, 2026.

Include this page and the ZIP checksum when citing the exact evidence used.