Reject incomplete install packages and exclude inherited guest disk

This commit is contained in:
bholman committed 2026-11-18 16:33:51 -07:00
1 parent 8088c1fbf6
commit 9b8e25d977
9 files changed
+277 -25

No files matched your search

+4
View File
@@ -31,6 +31,10 @@ docs/MENUS.md for the complete tree and docs/VALIDATION.md for acceptance scope.
Limit downloadable source selections and provide a pinned full-clone build route.
- Require a current inventory identity and complete qualified build-input set
before packaging, rejecting legacy/stale receipts without rewriting evidence.
- Reject incomplete local manifests before SSH: require the referenced RBF, boot pair
and declared SSC firmware/path records; distinguish load-command delivery from startup.
- Exclude the inherited guest Boot.vhd disk from source export and pass explicit
qualified build directories through the Windows packaging wrapper.
- Stage and verify all supporting files before atomic launcher activation;
preserve planned/failed installation receipts and rollback locations.
- Separate measured timing paths from empty asynchronous checks and identify
+10
View File
@@ -160,6 +160,16 @@ timing evidence. Do not add an inventory hash to an old receipt and present it a
newly qualified. Packaging-only changes need not alter RTL, but do not waive this
receipt requirement.
Select that new fit explicitly when packaging; `-Build` defaults to `build/fpga`
only when omitted. Use fresh output names to preserve previous packages:
```powershell
# Complete local install package using your separately supplied verified ROMs.
.\windows\Package-Release.ps1 -Build build/fpga-qualified -Output dist/coco3plus-qualified
# Public core/source package; all separately supplied/assembled ROMs omitted.
.\windows\Package-Release.ps1 -Build build/fpga-qualified -Output dist/coco3plus-qualified-public -PublicPackage
```
The real local-package guards can target a new qualified build and package
without moving or editing historical evidence. From the checkout in PowerShell:
+11 -1
View File
@@ -107,6 +107,13 @@ repository. Build scripts need Git metadata and the full project inventory.
Follow `SOURCE-CHECKOUT.txt` beside the archive to clone the repository and check
out the exact release revision before following `docs/BUILD_WINDOWS.md`.
Before any SSH or target access, the installer validates format and local-ROM
state, SHA-256 syntax, the canonical digest-addressed RBF, the launcher's RBF
reference and `coco3plus` namespace, exact boot-ROM identities and default CFG.
An SSC-enabled local package also needs both exact chip ROM identities and the
matching 1024-byte `f4`/`f5` path records. Incomplete or contradictory packages are
rejected locally; `--without-ssc` remains supported with its matching CFG.
## First launch and recovery
Choose Computer → Coco 3 Plus. It should reach Disk Extended Color BASIC.
@@ -127,7 +134,10 @@ but some supporting files may already have been replaced: restore their backed-u
versions if needed before loading the previous core. If activation was attempted,
launcher_activated: null means a transport failure left its outcome uncertain;
inspect the installed launcher rather than assuming it was rolled back. A null
loaded value likewise means the load command was attempted but not confirmed.
`load_command_sent` value means the command write was attempted but not confirmed.
A true value confirms only that the MiSTer command write succeeded; it does not
confirm guest startup. `startup_observed` remains false until separate hardware
observation, which this installer does not perform.
When the old launcher needs restoration, restore _Computer/Coco 3 Plus.mgl from
the remote backup directory. Its previously addressed RBF remains in place.
+5
View File
@@ -42,6 +42,11 @@ Before public source or binary release, complete the component review for:
objects or resolve their publication status.
- GPL-covered framework modules: retain notices, applicable GPL texts and build
materials/corresponding source obligations for combined distributions.
- `docs/Boot.vhd` is an inherited 224,016-byte guest virtual-disk image, not VHDL
source. `changes.txt` describes `boot.vhd` as an autorun/mounted disk containing
demonstration software. Its software/content provenance and redistribution
terms require separate review. It is excluded from the downloadable source
payload; the inherited Git object remains in the preserved full history.
This list identifies release-review work. It does not revoke or replace original
component terms, establish a blanket prohibition, or claim that review is complete.
+130 -7
View File
@@ -3,6 +3,7 @@
Copyright (c) 2026 Brian K. Holman. BSD-3-Clause; see LICENSES/.
The target is a disposable local filesystem, never an SSH/device connection.
"""
import hashlib
import json
import os
from pathlib import Path, PurePosixPath
@@ -103,14 +104,17 @@ class InstallerTransactionTests(unittest.TestCase):
self.boot0 = 'games/coco3plus/boot0.rom'
self.boot1 = 'games/coco3plus/boot1.rom'
self.config = 'config/coco3plus_v51.CFG'
self.rbf = '_Computer/.coco3plus/fixture/core.rbf'
rbf_data = b'new bitstream fixture\n'
digest = hashlib.sha256(rbf_data).hexdigest()
self.rbf = '_Computer/.coco3plus/' + digest[:12] + '/coco3plus.rbf'
contents = {
# Deliberately put the launcher first: manifest ordering cannot activate it early.
self.launcher: b'new launcher fixture\n',
self.launcher: ('<mistergamedescription><rbf>' + self.rbf[:-4] +
'</rbf><setname>coco3plus</setname></mistergamedescription>').encode(),
self.boot0: b'new boot0 fixture\n',
self.config: b'new default configuration\n',
self.config: installer.default_config(False),
self.boot1: b'new boot1 fixture\n',
self.rbf: b'new bitstream fixture\n',
self.rbf: rbf_data,
}
entries = {}
for name, data in contents.items():
@@ -120,7 +124,15 @@ class InstallerTransactionTests(unittest.TestCase):
entries[name] = dict(
bytes=len(data), sha256=installer.sha(path),
policy='create-if-missing' if name == self.config else 'backup-and-replace')
self.manifest = dict(files=entries, rbf_sha256=entries[self.rbf]['sha256'])
# Synthetic bytes are accepted ONLY through per-test patched identities.
# These fixtures are not original ROMs and never leave disposable storage.
boot = {Path(name).name: (len(contents[name]), entries[name]['sha256'])
for name in (self.boot0, self.boot1)}
boot_patch = patch.object(installer, 'BOOT', boot)
boot_patch.start()
self.addCleanup(boot_patch.stop)
self.manifest = dict(format=1, public_package=False, roms_required_separately=False,
ssc_included=False, files=entries, rbf_sha256=digest)
(self.package / 'manifest.json').write_text(json.dumps(self.manifest))
for name, data in ((self.launcher, b'old launcher remains selectable\n'),
(self.boot0, b'old boot0 recoverable\n'),
@@ -133,6 +145,103 @@ class InstallerTransactionTests(unittest.TestCase):
return installer.install_release(
self.package, 'disposable.invalid', load=load, target=target, stamp=stamp)
def write_manifest(self):
(self.package / 'manifest.json').write_text(json.dumps(self.manifest))
def replace_member(self, name, content):
path = self.package / 'fat' / name
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(content)
self.manifest['files'][name] = dict(bytes=len(content), sha256=installer.sha(path),
policy='create-if-missing' if name == self.config else 'backup-and-replace')
def add_synthetic_ssc(self):
identities = {}
for name in installer.SSC:
content = ('synthetic SSC fixture ' + name).encode()
path = 'games/coco3plus/' + name
self.replace_member(path, content)
identities[name] = (len(content), self.manifest['files'][path]['sha256'])
ssc_patch = patch.object(installer, 'SSC', identities)
ssc_patch.start()
self.addCleanup(ssc_patch.stop)
self.manifest['ssc_included'] = True
self.replace_member(self.config, installer.default_config(True))
for index, name in ((4, 'pic-7040-510.bin'), (5, 'sp0256-al2.bin')):
self.replace_member('config/coco3plus.f' + str(index),
('/media/fat/games/coco3plus/' + name).encode().ljust(1024, b'\0'))
def assert_rejected_before_target(self):
self.write_manifest()
target = DisposableTarget(self.device, self.package)
with self.assertRaises(RuntimeError):
self.install(target)
self.assertEqual(target.events, [])
self.assertFalse((self.package / 'install-receipt.json').exists())
def test_missing_rbf_boot_or_config_rejected_before_target_access(self):
for name in (self.rbf, self.boot0, self.boot1, self.config):
with self.subTest(name=name):
entry = self.manifest['files'].pop(name)
self.assert_rejected_before_target()
self.manifest['files'][name] = entry
def test_missing_ssc_rom_or_path_record_rejected_before_target_access(self):
self.add_synthetic_ssc()
for name in ('games/coco3plus/pic-7040-510.bin',
'games/coco3plus/sp0256-al2.bin',
'config/coco3plus.f4', 'config/coco3plus.f5'):
with self.subTest(name=name):
entry = self.manifest['files'].pop(name)
self.assert_rejected_before_target()
self.manifest['files'][name] = entry
def test_complete_synthetic_ssc_package_validates(self):
self.add_synthetic_ssc()
self.write_manifest()
checked = installer.validated_manifest(self.package)
self.assertTrue(checked['ssc_included'])
def test_contradictory_launcher_or_rbf_digest_rejected_before_target(self):
original = (self.package / 'fat' / self.launcher).read_bytes()
for content in (original.replace(b'coco3plus</setname>', b'other</setname>'),
original.replace(self.rbf[:-4].encode(), b'_Computer/other'),
original.replace(b'</rbf>', b'</rbf><rbf>other</rbf>')):
with self.subTest(content=content):
self.replace_member(self.launcher, content)
self.assert_rejected_before_target()
self.replace_member(self.launcher, original)
# Keep the addressed prefix so this specifically checks full digest agreement.
digest = self.manifest['rbf_sha256']
self.manifest['rbf_sha256'] = digest[:12] + ('0' if digest[12] != '0' else '1') + digest[13:]
self.assert_rejected_before_target()
def test_bad_flags_digests_and_rom_identity_rejected_before_target(self):
for field, value in (('format', 2), ('format', True), ('public_package', True),
('public_package', 0), ('roms_required_separately', True),
('ssc_included', None), ('rbf_sha256', 'not-a-digest')):
with self.subTest(field=field, value=value):
original = self.manifest[field]
self.manifest[field] = value
self.assert_rejected_before_target()
self.manifest[field] = original
original_digest = self.manifest['files'][self.boot0]['sha256']
self.manifest['files'][self.boot0]['sha256'] = 'bad-file-digest'
self.assert_rejected_before_target()
self.manifest['files'][self.boot0]['sha256'] = original_digest
self.replace_member(self.boot0, b'wrong synthetic boot identity')
self.assert_rejected_before_target()
def test_ssc_wrong_path_or_rom_identity_rejected_before_target(self):
self.add_synthetic_ssc()
self.replace_member('config/coco3plus.f4',
b'/media/fat/games/wrong/pic.bin'.ljust(1024, b'\0'))
self.assert_rejected_before_target()
self.replace_member('config/coco3plus.f4',
b'/media/fat/games/coco3plus/pic-7040-510.bin'.ljust(1024, b'\0'))
self.replace_member('games/coco3plus/sp0256-al2.bin', b'wrong synthetic speech identity')
self.assert_rejected_before_target()
def test_support_interruption_keeps_old_launcher_and_recovery_evidence(self):
target = DisposableTarget(self.device, self.package, interrupt_on=self.boot1)
original_launcher = (self.device / self.launcher).read_bytes()
@@ -148,7 +257,7 @@ class InstallerTransactionTests(unittest.TestCase):
self.assertIn(self.launcher, receipt['planned_files'])
self.assertFalse(receipt['activation_attempted'])
self.assertIs(receipt['launcher_activated'], False)
self.assertFalse(receipt['loaded'])
self.assertFalse(receipt['load_command_sent'])
self.assertEqual(receipt['error']['type'], 'KeyboardInterrupt')
self.assertEqual((self.device / self.launcher).read_bytes(), original_launcher)
self.assertNotIn('activate', target.events)
@@ -171,7 +280,9 @@ class InstallerTransactionTests(unittest.TestCase):
self.assertEqual(receipt['status'], 'complete')
self.assertTrue(receipt['launcher_activated'])
self.assertTrue(receipt['loaded'])
self.assertTrue(receipt['load_command_sent'])
self.assertIs(receipt['startup_observed'], False)
self.assertNotIn('loaded', receipt)
self.assertEqual((self.device / self.config).read_bytes(), original_config)
self.assertEqual((self.device / self.launcher).read_bytes(),
(self.package / 'fat' / self.launcher).read_bytes())
@@ -186,6 +297,18 @@ class InstallerTransactionTests(unittest.TestCase):
self.assertFalse(target.local(receipt['launcher_temporary']).exists())
self.assertTrue(target.local(receipt['stage'] + '/fat/' + self.launcher).exists())
def test_load_command_failure_is_uncertain_and_does_not_claim_startup(self):
target = DisposableTarget(self.device, self.package)
with patch.object(target, 'load', side_effect=OSError('Unconfirmed command write')):
with self.assertRaisesRegex(OSError, 'Unconfirmed command write'):
self.install(target, load=True)
receipt = json.loads((self.package / 'install-receipt.json').read_text())
self.assertEqual(receipt['status'], 'failed')
self.assertTrue(receipt['launcher_activated'])
self.assertIsNone(receipt['load_command_sent'])
self.assertIs(receipt['startup_observed'], False)
self.assertNotIn('loaded', receipt)
def test_preflight_failure_keeps_plan_and_does_not_mutate_target(self):
target = DisposableTarget(self.device, self.package, fail_preflight=True)
before = {p.relative_to(self.device): p.read_bytes()
+44
View File
@@ -6,7 +6,9 @@ Original test contributions: BSD-3-Clause; see LICENSES/.
import hashlib
import io
import json
import os
from pathlib import Path
import shutil
import subprocess
import sys
import tarfile
@@ -25,6 +27,48 @@ def digest(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
class ActualRepositoryExportPolicy(unittest.TestCase):
def test_actual_inherited_guest_disk_is_not_source(self):
revision = subprocess.check_output(
['git', 'rev-parse', 'HEAD'], cwd=REPOSITORY).decode().strip()
tree = release.committed_tree(revision)
# Inspect the real inherited Git object, not a fabricated .vhd filename.
# changes.txt identifies boot.vhd as mounted/autorun guest software.
self.assertIn('docs/Boot.vhd', tree)
self.assertEqual(len(release.committed_blob(tree['docs/Boot.vhd'])), 224016)
exported = {name for name in tree if release.source_member(name)}
self.assertNotIn('docs/Boot.vhd', exported)
self.assertEqual(release.SOURCE_SUFFIXES['docs'], {'.md'})
self.assertIn('rtl/CPU09/cpu09_119.vhd', exported)
def test_powershell_wrapper_passes_selected_and_default_build(self):
shell = shutil.which('pwsh') or shutil.which('powershell')
if not shell:
self.skipTest('PowerShell unavailable for native wrapper contract')
with tempfile.TemporaryDirectory(dir=REPOSITORY / 'build') as folder:
base = Path(folder)
capture = base / 'capture.json'
fake = base / 'capture-python.ps1'
fake.write_text('$args | ConvertTo-Json -Compress | Set-Content -LiteralPath $env:COCO3PLUS_WRAPPER_CAPTURE\n$global:LASTEXITCODE=0\n')
environment = os.environ.copy()
environment['COCO3PLUS_WRAPPER_CAPTURE'] = str(capture)
for selection in ([], ['-Build', 'build/fpga-qualified']):
with self.subTest(selection=selection):
subprocess.run(
[shell, '-NoProfile', '-ExecutionPolicy', 'Bypass', '-File',
str(REPOSITORY / 'windows/Package-Release.ps1'),
'-Python', str(fake), '-Output', 'dist/qualified',
'-PublicPackage', *selection], cwd=REPOSITORY,
env=environment, check=True, capture_output=True)
arguments = json.loads(capture.read_text(encoding='utf-8-sig'))
index = arguments.index('--build')
self.assertEqual(arguments[index + 1],
selection[-1] if selection else 'build/fpga')
self.assertIn('--public', arguments)
self.assertEqual(arguments[arguments.index('--output') + 1],
'dist/qualified')
class SimulatorInstallGuards(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory(dir=REPOSITORY / 'build')
+70 -15
View File
@@ -14,8 +14,9 @@ import shlex
import subprocess
import sys
import uuid
import xml.etree.ElementTree as ET
from release import ROOT, make_release, sha
from release import BOOT, SSC, ROOT, default_config, make_release, sha
LAUNCHER = '_Computer/Coco 3 Plus.mgl'
@@ -110,24 +111,78 @@ class OpenSshTarget:
def validated_manifest(package):
"""Require a coherent, ROM-complete local package before any target call."""
manifest = json.loads((package / 'manifest.json').read_text())
if manifest.get('public_package'):
raise RuntimeError('Public core/source package omits required ROMs. '
'Build a complete local package with your verified '
'ROM inputs; see docs/INSTALL.md.')
for name, entry in manifest['files'].items():
if not isinstance(manifest, dict) or type(manifest.get('format')) is not int or manifest['format'] != 1:
raise RuntimeError('Unsupported release manifest format')
if manifest.get('public_package') is not False or manifest.get('roms_required_separately') is not False:
raise RuntimeError('A complete local package with verified ROM inputs is required; see docs/INSTALL.md')
if type(manifest.get('ssc_included')) is not bool:
raise RuntimeError('Explicit SSC inclusion state is required')
digest = manifest.get('rbf_sha256')
if not isinstance(digest, str) or not re.fullmatch(r'[0-9a-f]{64}', digest):
raise RuntimeError('Invalid RBF SHA-256')
files = manifest.get('files')
if not isinstance(files, dict):
raise RuntimeError('Manifest file inventory is required')
for name, entry in files.items():
if not isinstance(name, str):
raise RuntimeError('Invalid manifest path')
path = PurePosixPath(name)
if path.is_absolute() or '..' in path.parts or '\\' in name:
if (not name or path.is_absolute() or '..' in path.parts or
'\\' in name or ':' in name or path.as_posix() != name):
raise RuntimeError('Unsafe manifest path')
if entry['policy'] not in ('create-if-missing', 'backup-and-replace'):
if (not isinstance(entry, dict) or type(entry.get('bytes')) is not int or
entry['bytes'] < 1 or not isinstance(entry.get('sha256'), str) or
not re.fullmatch(r'[0-9a-f]{64}', entry['sha256'])):
raise RuntimeError('Invalid file identity: ' + name)
if entry.get('policy') not in ('create-if-missing', 'backup-and-replace'):
raise RuntimeError('Unsupported install policy: ' + name)
file = package / 'fat' / name
if (not file.is_file() or file.stat().st_size != entry['bytes'] or
if (not file.is_file() or file.is_symlink() or file.stat().st_size != entry['bytes'] or
sha(file) != entry['sha256']):
raise RuntimeError('Release changed: ' + name)
if (LAUNCHER not in manifest['files'] or
manifest['files'][LAUNCHER]['policy'] != 'backup-and-replace'):
raise RuntimeError('A replaceable Coco 3 Plus launcher is required')
def require(name, identity=None, policy='backup-and-replace', content=None):
entry = files.get(name)
if entry is None or entry['policy'] != policy:
raise RuntimeError('Required package member/policy missing: ' + name)
if identity is not None and (entry['bytes'], entry['sha256']) != identity:
raise RuntimeError('Incorrect required identity: ' + name)
if content is not None and (package / 'fat' / name).read_bytes() != content:
raise RuntimeError('Incorrect required content: ' + name)
core = '_Computer/.coco3plus/' + digest[:12] + '/coco3plus.rbf'
require(core)
if files[core]['sha256'] != digest:
raise RuntimeError('RBF member contradicts top-level SHA-256')
require(LAUNCHER)
try:
launcher = ET.fromstring((package / 'fat' / LAUNCHER).read_bytes())
except ET.ParseError as error:
raise RuntimeError('Invalid Coco 3 Plus launcher XML') from error
if (launcher.tag != 'mistergamedescription' or launcher.attrib or
len(launcher) != 2 or len(launcher.findall('rbf')) != 1 or
len(launcher.findall('setname')) != 1 or
any(child.attrib or len(child) for child in launcher) or
launcher.findtext('rbf') != core[:-4] or
launcher.findtext('setname') != 'coco3plus'):
raise RuntimeError('Launcher RBF reference/setname contradicts package')
for name, identity in BOOT.items():
require('games/coco3plus/' + name, identity)
ssc = manifest['ssc_included']
require('config/coco3plus_v51.CFG', policy='create-if-missing',
content=default_config(ssc))
ssc_members = {'games/coco3plus/' + name for name in SSC}
ssc_members.update(('config/coco3plus.f4', 'config/coco3plus.f5'))
if ssc:
for name, identity in SSC.items():
require('games/coco3plus/' + name, identity)
for index, name in ((4, 'pic-7040-510.bin'), (5, 'sp0256-al2.bin')):
content = ('/media/fat/games/coco3plus/' + name).encode().ljust(1024, b'\0')
require('config/coco3plus.f' + str(index), content=content)
elif ssc_members.intersection(files):
raise RuntimeError('SSC files contradict disabled SSC package state')
return manifest
@@ -167,7 +222,7 @@ def install_release(package, host, user='root', load=False, *, target=None, stam
launcher='/media/fat/' + LAUNCHER,
launcher_temporary='/media/fat/' + LAUNCHER + '.install-' + stamp + '.tmp',
activation_attempted=False, launcher_activated=False, load_requested=load,
loaded=False, planned_files=plans, active_file=None,
load_command_sent=False, startup_observed=False, planned_files=plans, active_file=None,
completed_files=[], history=['planned'],
recovery='Preserve stage and backup. Before activation the old launcher '
'remains, but supporting files may already have changed. Restore '
@@ -234,10 +289,10 @@ def install_release(package, host, user='root', load=False, *, target=None, stam
client.sync()
if load:
phase('loading')
receipt['loaded'] = None
receipt['load_command_sent'] = None
save()
client.load()
receipt['loaded'] = True
receipt['load_command_sent'] = True
receipt['status'] = 'complete'
receipt['phase'] = 'complete'
receipt['history'].append('complete')
+1 -1
View File
@@ -10,7 +10,7 @@ SOURCE_ROOT_FILES={'.gitattributes','.gitignore','AGENTS.md','CHANGELOG.md',
'CoCo3.sv','CoCo3FPGA.sdc','build_id_num.v','changes.txt','external-io.sdc',
'files.qip','pll.xml','reset-cpu.sdc','uart-reset.sdc'}
SOURCE_SUFFIXES={
'docs':{'.md','.vhd'},'roms':{'.md'},
'docs':{'.md'},'roms':{'.md'},
'rtl':{'.sv','.v','.vh','.vhd','.mif','.hex','.qip','.qsys','.ip','.tcl','.sdc',
'.ppf','.sip','.spd'},
'sys':{'.sv','.v','.vh','.vhd','.mif','.hex','.qip','.qsys','.ip','.tcl','.sdc'},
+2 -1
View File
@@ -1,7 +1,8 @@
[CmdletBinding()]
param([string]$Python='python',[string]$Output,[switch]$WithoutSSC,[switch]$PublicPackage)
param([string]$Python='python',[string]$Output,[string]$Build='build/fpga',[switch]$WithoutSSC,[switch]$PublicPackage)
$ErrorActionPreference='Stop'
$scriptArgs=@('-X','utf8','-B',"$PSScriptRoot/../tools/release.py")
$scriptArgs+=@('--build',$Build)
if ($Output) {$scriptArgs+=@('--output',$Output)}
if ($WithoutSSC) {$scriptArgs+='--without-ssc'}
if ($PublicPackage) {$scriptArgs+='--public'}