provevi.checks

read_events

source

read_events(folder: Path) -> list[EvidenceEvent]

Read the events of an evidence folder.

Raises EvidenceError if events.jsonl cannot be read, or at a line that is not an event.

verify

source

verify(folder: Path | RunRecord, run_id: str | None = None) -> list[str]

Check that a run’s evidence is consistent and whole.

It checks the chain, the sequence, the payloads and the manifest, and that the evidence is the whole record of one run: run_created first, one ending event last, as many events as the manifest says, the payloads that the manifest lists, and the run ID given, if one is. A malformed folder gives problems, never an exception.

The check proves that the folder is consistent. Someone who can write the folder can still write a consistent false record. To rule that out, compare the folder with the bundle committed in the store, which verify_against does.

verify_against

source

verify_against(folder: Path, events: bytes, manifest: bytes) -> list[str]

Compare the local evidence with the committed bundle.

A difference means that one of them was changed.

verify_outputs

source

verify_outputs(run_folder: Path) -> list[str]

Check that the files in output/ are exactly the recorded outputs.

Each must have its recorded bytes, be a regular file, and be recorded once. An output event names its file by its reference, <run URI>/output/<path>.

bundle_pin

source

bundle_pin(run_folder: Path, given: str | None = None) -> str | None

Find the pinned bundle to compare a run with.

It is the one given, if there is one, else the one that bundle.json names. bundle.json is in the run folder, so whoever can change the folder can remove it or point it elsewhere; a commit given from outside the folder does not have that weakness. Raises EvidenceError if bundle.json cannot be read.

verify_bundle

source

verify_bundle(run_folder: Path, pinned: str | None, catalog: Catalog | Callable[[], Catalog] | None, *, require: bool = False) -> list[str]

Compare a run’s evidence with its committed bundle, read through the catalog.

A bad URI, an unpinned URI, a missing catalog, a catalog that does not open and a bundle that cannot be read each give a problem.

class RunCheck

source

RunCheck(run_id: str, problems: list[str] = <factory>, pinned: str | None = None, experiment: str | None = None)

The result of verify_run.

RunCheck.to_dict() -> dict

verify_store_copy

source

verify_store_copy(run_folder: Path, store: EvidenceStore) -> list[str]

Check a run as the store holds it, against its run folder.

The store’s copy must verify, and the files in the run’s evidence folder, which are its bundle, must be the same as the store’s. For a store that records the bundle, the record must match bundle.json.

verify_run

source

verify_run(run_folder: Path, *, catalog: Catalog | Callable[[], Catalog] | None = None, bundle: str | None = None, require_bundle: bool = False, store: EvidenceStore | None = None) -> RunCheck

Check a run as prove verify does.

It checks the evidence, then the outputs, then the committed bundle, then the version of the experiment that the run names, read at its pin, whose hash must match the run’s. With a store that keeps its own record, such as a SqliteStore, the run as the store holds it is checked too, and must match the folder. A malformed run gives problems, never an exception.

verify_experiment

source

verify_experiment(run_folder: Path, catalog: Catalog | Callable[[], Catalog] | None) -> list[str]

Check the experiment that a run names in run_created.

Its pin must read through the catalog, and the file there must have the hash that the run recorded.