CoCoCapitalQuiz/docs/TESTING.md

# Initial release test results — 2026-11-09

For the current sound and automatic-feedback changes, see
[FEEDBACK_TESTING.md](FEEDBACK_TESTING.md). The results below describe the initial release.

Built and played with installed **VCC Plus 3.5.0, build 41**, bundled **CMOC 0.1.100**,
and the exported **public STDIO MCP interface**. All gameplay ran in disposable
**512 KB MC6809** workers. Inputs used ordinary physical guest key chords, with
release fields between presses. No guest memory or registers were modified by
a debugger. Personal emulator media and the original running machine were preserved.

## Initial release validation

| Check | Result |
| --- | --- |
| Release, Debug, Test disk builds | All succeeded; no diagnostics |
| Complete multiple-choice quiz | 50 unique states, numbering 1–50, 42/50 (84%) |
| Complete typed quiz | 50 unique states, 47/50 (94%), 2 spelling corrections |
| Complete ordinary Release quiz | 50 unique states, 50/50 (100%), perfect review |
| Four distinct choices and varied correct slots | All 50 captured questions checked |
| Full state name and current-state fill | All 50 national framebuffer masks checked |
| Alaska / Hawaii / Northeast | Actual captures inspected; Hawaii island grouping corrected |
| Typed casing | Uppercase, lowercase and mixed case actually entered |
| Spaces and aliases | Multiword names, surrounding/repeated spaces, mixed-case St Paul |
| Spelling retry | Charleston and Denver transpositions: no early point/reveal; Backspace edits then correct |
| Reveal | Near Tallahassee spelling → V reveals and records a miss |
| Known wrong capital | Austin for Connecticut → incorrect, not spelling retry |
| Long input | 45 attempted X characters: bounded at 39; Enter and subsequent questions work |
| Missed review | Eight MC misses over 3 pages, three typed misses over 1 page; state, entered answer and capital checked |
| Held key | First answer held 600 fields through feedback; no accidental advance |
| Replay | P returns to mode menu; Release starts a different first-five-state order |
| Credits | Source URL, license URL and adaptation notice read from actual framebuffer |
| Exit | Title and results Quit return to BASIC with final thank-you text |

Host checks additionally passed **2,000 shuffles**, **100,000 choice sets**, all
50 exact capitals in varied casing, all **2,450** wrong-capital pairs, all
Saint Paul aliases (including **St. Paul**), edit-distance/transposition cases,
long-string bounds, one-point-per-state guarding, correction accounting and reset.
Geometry checks cover 50 unique stable IDs, disjoint bounded spans and all
national masks, plus Northeast and Hawaii detail coverage. The Hawaii detail
contains only Hawaiian state pixels. The geography converter uses explicit
inset rules for detached Hawaiian/Alaskan islands and Michigan’s Upper Peninsula.

## Reproduce and inspect

Run `make test` for host checks. For two full deterministic guest quizzes:
```sh
PYTHONDONTWRITEBYTECODE=1 python3 tests/playtest.py /private/path/grant.json \
  --evidence /tmp/capital-quiz-evidence
```

The grant must enable Use Project Makefile and Allow Disposable Playtests.
The Test configuration uses seed 2709; Release uses timing/start-input entropy.
Captured PNG pixels are decoded against the font in the matching linked artifact.
Text, scores, options and selected-state fill are checked from real frames.
Full PNG captures and public receipts are in ignored `build/evidence/test/` and
`build/evidence/release/`. Compact receipts are retained in `docs/evidence/`.
The typed report’s answer field records the initial test input when a spelling
correction was subsequently made; the captured feedback and score verify the correction.

## Initial release artifact

- Build: `9922B526-537B-4820-BF6C-100EB8846077`
- Artifact: `B6BCF288-91C9-43D8-A460-595792BFD457`
- Release gameplay run: `4EC7FB7D-CBF8-4235-AE82-4363E57376CE`
- Binary: `e69f25c3cf8375f8fe20829d6c11a0a8cb57ceafbf10b5ccefe18c3a1773527a` (SHA-256)
- Disk: `98c5e46b17f29b10e28b818a72512cd2e8331294da223600de99589ac4e3aa68` (SHA-256)
- Executable: `build/CAPITAL.bin`; disk: `build/CAPITAL.dsk` (35 tracks, 161,280 bytes).
- Release code/static storage ends at `$7530`, below the `$7800` limit, leaving
  720 bytes before the separate 2 KiB stack allowance to `$8000`.
- Map runs and offset tables total **4,080 bytes**; the public GIME helper
  reserves 64 KiB of separate physical RAM for two display pages.
- Captures report **zero guest disk writes**. Test disk originals were preserved.

## Limits

No physical CoCo hardware test was performed. VCC’s public key enumeration
omits punctuation, so the dotted **St. Paul** alias was verified in the actual
shared C logic rather than typed through MCP. The custom-Make build receipt
did not advertise verified C/function stop lists, and a general pause followed
by snapshot inspection returned `verified_pause_required`; no source-debug
watch or memory inspection is claimed. These limits and reusable MCP
enhancement proposals are described in [MCP-NOTES.md](MCP-NOTES.md).