provevi.mlflow_copy

The copy in MLflow: a sealed run, rebuilt from its evidence in MLflow’s tables.

The evidence is the record of a run, and MLflow holds a copy for its viewer, its search and its assessments. copy_run reads a sealed run from a store, checks its chain, and writes it to the place that the telemetry policy approves:

MLFLOW_TRACKING_URI names the place, and the policy’s mlflow destination that matches it gives the level, which is what the copy holds. With the variable unset, the copy goes to the store’s own file. A place that no destination matches gets nothing.

The copy makes an MLflow experiment named for the experiment that the run names, such as acme/risk/dev/credit/q, or else for the run’s agent class, such as acme/demo/dev/revenue-agent, and an MLflow run tagged with provrun’s run ID. MLflow picks its own run ID, so the tag is how provevi finds the copy again. Copying a run again deletes the earlier copy first, so the copy can be rebuilt at any time.

The run’s trace keeps provrun’s trace and span IDs. Each step with a span becomes a span from its start to its end, and the run is the root span. provevi writes whole spans through MLflow’s store, so MLflow adds nothing of its own, such as the user name or the git commit, and the copy does not depend on MLflow’s global tracking URI. MLflow is loaded only here, as the optional extra provevi[mlflow], with its usage telemetry off.

What the copy holds follows the destination’s level, as for any destination of the policy. At every level it holds the experiment, the run with its tags and its settings that are names, each input’s SHA-256, and the spans with their names, models and IDs. A name here is a short word with no URL, no user and password, no = and no leading /, as a secret or a path could be written so. usage adds the token counts and the counts of calls. bodies adds every other setting, each input’s URIs, each step’s data and URIs, the requests and responses of model calls, and the events that had no span, as events of the root span.

prove mlflow RUN makes the copy, and prove ui opens MLflow’s viewer on the store’s own file.

class MlflowCopy

source

MlflowCopy(run_id: str, destination: str, level: str, tracking_uri: str, experiment_id: str, mlflow_run_id: str, trace_id: str | None, spans: int, assessments: int = 0, replaced: list[str] = <factory>)

Where a run was copied in MLflow, and what the copy holds.

Parameter Type Default Description
run_id str required provrun’s run ID
destination str required The name of the policy’s mlflow destination that approved the copy
level str required The destination’s level, which says what the copy holds
tracking_uri str required Where the copy went, with no user or password
experiment_id str required MLflow’s ID of the experiment the copy is in
mlflow_run_id str required MLflow’s ID of the run it made
trace_id str | None required MLflow’s ID of the run’s trace; None when it wrote none
spans int required The number of spans in the trace
assessments int 0 The number of assessments copied onto the trace
replaced list[str] <factory> MLflow’s IDs of the earlier copies of the run, now deleted, or marked deleted on a tracking server

tracking_target

source

tracking_target(policy: TelemetryPolicy, store: EvidenceStore, environ: Mapping[str, str] | None = None) -> tuple[str, str, str]

Find where a copy of a run goes, and at what level, from MLFLOW_TRACKING_URI.

Raises ProveviError when the policy approves no mlflow destination for it, or when the variable is unset and the store has no file of its own.

Parameter Type Default Description
policy TelemetryPolicy required The telemetry policy, whose mlflow destination must match
store EvidenceStore required The store, whose own file takes the copy when MLFLOW_TRACKING_URI is unset
environ Mapping[str, str] | None None The environment to read MLFLOW_TRACKING_URI from; None for os.environ
returns tuple[str, str, str] The destination’s name, its level and the tracking URI

copy_run

source

copy_run(store: EvidenceStore, run_id: str, policy: TelemetryPolicy, *, environ: Mapping[str, str] | None = None, folder: Path | None = None) -> MlflowCopy

Copy one sealed run into MLflow.

The copy goes to the place, and at the level, that the policy approves for MLFLOW_TRACKING_URI. A copy made before is deleted first. Raises ProveviError if the run does not verify, if the policy approves no place, or if MLflow cannot write there.

Parameter Type Default Description
store EvidenceStore required The store that holds the run
run_id str required The run’s ID
policy TelemetryPolicy required The telemetry policy, which approves the place and gives the level
environ Mapping[str, str] | None None The environment to read MLFLOW_TRACKING_URI from; None for os.environ. The redirect variables are always read from os.environ
folder Path | None None The run’s evidence folder, which the file store needs
returns MlflowCopy Where the copy went and what it holds

refuse_redirects

source

refuse_redirects(url: str, environ: Mapping[str, str] | None = None) -> None

Refuse a copy that a variable would send somewhere else.

Raises ProveviError if a variable in environ, such as a proxy, would send a copy to url somewhere else, or change which servers it trusts. provrun calls it as a run starts, so that the run records a copy that would be refused.

Parameter Type Default Description
url str required The tracking URI the copy goes to
environ Mapping[str, str] | None None The environment to check; None for os.environ

ui_command

source

ui_command(path: Path, port: int) -> tuple[list[str], dict[str, str]]

Make the command that serves MLflow’s viewer on the store’s file, on 127.0.0.1 only.

It puts the file in WAL mode, where it stays.

Parameter Type Default Description
path Path required The store’s SQLite file
port int required The port on 127.0.0.1
returns tuple[list[str], dict[str, str]] The command, and its environment: this process’s, without its MLflow and OpenTelemetry variables, plus provevi’s MLflow settings, which turn off MLflow’s telemetry