provevi.store

Stores: where the evidence of a run is kept.

A store opens a run and gives a RunWriter. provrun’s recorder hands the writer each event line, stores each large payload through it, and seals it with the manifest when the run ends. The file store keeps one folder for each run::

<evidence>/events.jsonl            one event on each line, in order
<evidence>/payloads/<sha256>       large payloads, named by their SHA-256
<evidence>/manifest.json           written when the run is sealed

class Sink

source

A place that evidence lines go to.

Sink.append

source

Sink.append(line: str) -> None

Take one event line.

Parameter	Type	Default	Description
`line`	str	required	One event line, with no newline

Sink.close

source

Sink.close() -> None

Finish: no line comes after. The base sink does nothing.

class JsonlSink

source

JsonlSink(path: Path)

Appends each line to a local file, with no buffer in between.

A line that fails to write is never written later, and a writer that takes a line back can cut it from the file. The file is made with mode 0o666 less the umask, as open() makes it.

Parameter	Type	Default	Description
`path`	Path	required	The file to append to; it and its folder are made if missing

JsonlSink.append

source

JsonlSink.append(line: str) -> None

Write the line whole, or not at all.

A short write followed by an error is cut back, so a lost line never joins the next one.

Parameter	Type	Default	Description
`line`	str	required	One event line; the sink adds the newline

JsonlSink.close

source

JsonlSink.close() -> None

Flush the file to disk and close it. A second close does nothing.

class PayloadStore

source

PayloadStore(root: Path)

Large payloads, each in a file named by its SHA-256, written once.

Parameter	Type	Default	Description
`root`	Path	required	The folder of payloads, made at the first put

PayloadStore.put

source

PayloadStore.put(data: bytes | str) -> str

Store a payload once: a payload already stored is not written again.

Parameter	Type	Default	Description
`data`	bytes \| str	required	The payload; a text is stored as UTF-8
returns	str		The payload’s SHA-256, which is its file’s name

PayloadStore.get

source

PayloadStore.get(digest: str) -> bytes

Read a stored payload.

Parameter	Type	Default	Description
`digest`	str	required	The payload’s SHA-256
returns	bytes		The payload; OSError when it is not stored

PayloadStore.names

source

PayloadStore.names() -> list[str]

List the payloads stored.

Parameter	Type	Default	Description
returns	list[str]		The SHA-256 of each payload, sorted; hidden temporary files are left out

class RunWriter

source

The evidence of one open run.

It is the recorder’s first sink: an event counts as written only when the writer holds its line.

RunWriter.put_payload

source

RunWriter.put_payload(data: bytes | str) -> str

Store a payload of the run once.

Parameter	Type	Default	Description
`data`	bytes \| str	required	The payload; a text is stored as UTF-8
returns	str		The payload’s SHA-256, which an event’s `payload` holds

RunWriter.get_payload

source

RunWriter.get_payload(digest: str) -> bytes

Read back a payload of the run.

Parameter	Type	Default	Description
`digest`	str	required	The payload’s SHA-256
returns	bytes		The payload

RunWriter.payload_names

source

RunWriter.payload_names() -> list[str]

The payloads of the run, for the manifest.

Parameter	Type	Default	Description
returns	list[str]		The SHA-256 of each payload stored, sorted

RunWriter.seal

source

RunWriter.seal(manifest: dict) -> None

Record the manifest. Called once, after close.

Parameter	Type	Default	Description
`manifest`	dict	required	The manifest: `run_id`, `events` (the count), `last` (the hash of the last line), `payloads`, `complete` and `problems`

RunWriter.manifest

source

RunWriter.manifest() -> dict

The manifest recorded by seal.

Parameter	Type	Default	Description
returns	dict		The manifest as seal recorded it

RunWriter.record_bundle

source

RunWriter.record_bundle(uri: str | None, pinned: str | None) -> None

Note where the run’s bundle was committed, after the seal.

A store that keeps no record of runs ignores it.

Parameter	Type	Default	Description
`uri`	str \| None	required	The URI the bundle was committed at; `None` when it was not committed
`pinned`	str \| None	required	The URI pinned to the commit; `None` when it was not committed

class RunRecord

source

A sealed or open run as a store holds it, read back for the checks.

RunRecord.lines

source

RunRecord.lines() -> list[str]

The event lines, in order. Raises EvidenceError if they cannot be read.

Parameter	Type	Default	Description
returns	list[str]		Each line with no newline

RunRecord.manifest_bytes

source

RunRecord.manifest_bytes() -> bytes | None

Read the manifest as stored. May raise OSError.

Parameter	Type	Default	Description
returns	bytes \| None		The bytes of the manifest; `None` when the run has none

RunRecord.payload_digest

source

RunRecord.payload_digest(name: str) -> str | None

Hash the stored payload called name.

May raise OSError.

Parameter	Type	Default	Description
`name`	str	required	The payload’s name: the SHA-256 that an event names it by
returns	str \| None		The SHA-256 of its stored bytes; `None` when there is none

RunRecord.payload_files

source

RunRecord.payload_files() -> tuple[set[str], list[str]]

List the stored payloads, and the entries that are not payloads.

Each entry that is not a payload, such as a special file, gives a problem.

Parameter	Type	Default	Description
returns	tuple[set[str], list[str]]		The names, and the problems

RunRecord.payload_bytes

source

RunRecord.payload_bytes(name: str) -> bytes | None

Read the stored payload called name.

Parameter	Type	Default	Description
`name`	str	required	The payload’s name: its SHA-256
returns	bytes \| None		Its bytes; `None` when there is none, and always in the base class

RunRecord.problems

source

RunRecord.problems() -> list[str]

Problems of the store itself that the chain does not show.

Parameter	Type	Default	Description
returns	list[str]		One line each; empty in the base class

class FolderRecord

source

FolderRecord(folder: Path)

A run’s evidence folder, as the file store writes it.

Parameter	Type	Default	Description
`folder`	Path	required	The run’s evidence folder

FolderRecord.lines

source

FolderRecord.lines() -> list[str]

The event lines, in order. Raises EvidenceError if events.jsonl cannot be read.

Parameter	Type	Default	Description
returns	list[str]		Each line of `events.jsonl`, with no newline

FolderRecord.manifest_bytes

source

FolderRecord.manifest_bytes() -> bytes | None

The manifest as the folder holds it. May raise OSError.

Parameter	Type	Default	Description
returns	bytes \| None		The bytes of `manifest.json`; `None` when it is missing

FolderRecord.payload_digest

source

FolderRecord.payload_digest(name: str) -> str | None

Hash the payload file called name. May raise OSError.

Parameter	Type	Default	Description
`name`	str	required	The payload’s name under `payloads/`: its SHA-256
returns	str \| None		The SHA-256 of the file’s bytes; `None` when it is missing

FolderRecord.payload_bytes

source

FolderRecord.payload_bytes(name: str) -> bytes | None

Read the payload file called name.

Parameter	Type	Default	Description
`name`	str	required	The payload’s name under `payloads/`: its SHA-256
returns	bytes \| None		The file’s bytes; `None` when it is not a regular file

FolderRecord.payload_files

source

FolderRecord.payload_files() -> tuple[set[str], list[str]]

List the files under payloads/, and the entries that are not files.

Hidden names are left out. Each entry that is not a regular file or a link, and each folder that cannot be read, gives a problem.

Parameter	Type	Default	Description
returns	tuple[set[str], list[str]]		The paths under `payloads/`, and the problems

class EvidenceStore

source

Where runs keep their evidence. One store can hold many runs.

EvidenceStore.open_run

source

EvidenceStore.open_run(run_id: str, folder: Path) -> RunWriter

Open the evidence of a new run.

Parameter	Type	Default	Description
`run_id`	str	required	The new run’s ID
`folder`	Path	required	The run’s evidence folder, which holds the bundle files that provrun commits through provcat
returns	RunWriter		The writer of the run’s evidence

EvidenceStore.read_run

source

EvidenceStore.read_run(run_id: str, folder: Path | None = None) -> RunRecord

Read the evidence of a run back.

Raises RunNotFound, or CannotReadBack for a store that only writes, as the base class does.

Parameter	Type	Default	Description
`run_id`	str	required	The run’s ID
`folder`	Path \| None	`None`	The run’s evidence folder, for a store that keeps payloads there
returns	RunRecord		The run as the store holds it

EvidenceStore.list_runs

source

EvidenceStore.list_runs() -> list[str]

List the run IDs in the store, oldest first.

The base class raises CannotReadBack.

Parameter	Type	Default	Description
returns	list[str]		The run IDs

EvidenceStore.run_folder

source

EvidenceStore.run_folder(run_id: str) -> Path | None

Find the run folder, which holds the run’s outputs.

Parameter	Type	Default	Description
`run_id`	str	required	The run’s ID
returns	Path \| None		The run folder; `None` when the store does not know it, and always in the base class

class FileRunWriter

source

FileRunWriter(folder: Path)

The file store’s writer: events.jsonl, payloads/ and manifest.json in the run’s evidence folder.

Parameter	Type	Default	Description
`folder`	Path	required	The run’s evidence folder; made if missing

FileRunWriter.append

source

FileRunWriter.append(line: str) -> None

Append the line to events.jsonl, whole or not at all.

Parameter	Type	Default	Description
`line`	str	required	One event line, with no newline

FileRunWriter.close

source

FileRunWriter.close() -> None

Flush events.jsonl to disk and close it.

FileRunWriter.put_payload

source

FileRunWriter.put_payload(data: bytes | str) -> str

Store a payload once under payloads/.

Parameter	Type	Default	Description
`data`	bytes \| str	required	The payload; a text is stored as UTF-8
returns	str		The payload’s SHA-256, its file’s name under `payloads/`

FileRunWriter.get_payload

source

FileRunWriter.get_payload(digest: str) -> bytes

Read a payload from payloads/.

Parameter	Type	Default	Description
`digest`	str	required	The payload’s SHA-256
returns	bytes		The payload; OSError when it is not stored

FileRunWriter.payload_names

source

FileRunWriter.payload_names() -> list[str]

The payloads under payloads/.

Parameter	Type	Default	Description
returns	list[str]		The SHA-256 of each payload, sorted

FileRunWriter.seal

source

FileRunWriter.seal(manifest: dict) -> None

Write the manifest to manifest.json.

Parameter	Type	Default	Description
`manifest`	dict	required	The manifest the recorder made

FileRunWriter.manifest

source

FileRunWriter.manifest() -> dict

Read manifest.json back.

Parameter	Type	Default	Description
returns	dict		The manifest that manifest.json holds

class FileStore

source

FileStore(root: str | Path | None = None)

The file store: each run’s evidence in files in its own folder.

Parameter	Type	Default	Description
`root`	str \| Path \| None	`None`	The runs root that read_run, list_runs and run_folder look in; `None` for the default runs root

FileStore.open_run

source

FileStore.open_run(run_id: str, folder: Path) -> RunWriter

Open the evidence of a new run in its folder.

Parameter	Type	Default	Description
`run_id`	str	required	The new run’s ID; the file store does not use it
`folder`	Path	required	The run’s evidence folder, where the files go
returns	RunWriter		A `FileRunWriter` on the folder

FileStore.read_run

source

FileStore.read_run(run_id: str, folder: Path | None = None) -> RunRecord

Read a run’s evidence folder back.

Without folder, it raises RunNotFound when there is no run with that ID under the runs root.

Parameter	Type	Default	Description
`run_id`	str	required	The run’s ID, a folder under the runs root; not used when folder is given
`folder`	Path \| None	`None`	The run’s evidence folder; `None` to find it under the runs root
returns	RunRecord		A `FolderRecord` on the evidence folder

FileStore.list_runs

source

FileStore.list_runs() -> list[str]

List the runs under the runs root, as list_runs does.

Parameter	Type	Default	Description
returns	list[str]		The run IDs under the runs root, oldest first

FileStore.run_folder

source

FileStore.run_folder(run_id: str) -> Path | None

Find the run’s folder under the runs root.

Parameter	Type	Default	Description
`run_id`	str	required	The run’s ID
returns	Path \| None		The run folder under the runs root; `None` when there is no such run

check_store_uri

source

check_store_uri(uri: str) -> None

Check that a URI names a store, and open nothing.

A store URI is file, or sqlite:///path with no query or fragment. Raises ProveviError for any other.

Parameter	Type	Default	Description
`uri`	str	required	A store URI, such as `file` or `sqlite:///evidence.db`

open_store

source

open_store(uri: str | None, root: str | Path | None = None, *, create: bool = True) -> EvidenceStore

Open the store a URI names.

sqlite:///evidence.db is relative to the current folder when the store opens, and sqlite:////var/evidence.db is absolute, as in SQLAlchemy and MLflow; ~ is the home folder. Raises ProveviError.

Parameter	Type	Default	Description
`uri`	str \| None	required	`file`, or `None`, for the file store, or `sqlite:///path` for a SQLite file
`root`	str \| Path \| None	`None`	The runs root of the file store; `None` for the default runs root
`create`	bool	`True`	Make the SQLite file if it is missing, and bring an older schema up to date; with False, a SQLite store only reads, and must exist
returns	EvidenceStore		A `FileStore` or a `SqliteStore`