Plan for provrun, the runtime beside provcat

Room for later backends

The later backends need no change to the classes above. They need new subclasses only:

• Docker: a BindMountStager and a DockerBackend. The container serves the Foundry hosted-agent protocol, POST /responses on port 8088, so the same image runs in Foundry.
• Foundry prompt agents: a SessionUploadStager or a BlobRefStager, and a FoundryPromptBackend. Function tools run in provrun through the function_call and function_call_output round trip.
• Foundry hosted agents: a FoundryHostedBackend and an importer for Application Insights spans.
• Remote tools: a remote MCP mode on Tool, with an importer for the calls that provrun cannot see.

Foundry has no read-only mounts, and its session files are writable. So the stager records the hash of each staged input, and integrity checks use those hashes.

Phase 0: make the workspace

1. Wait until no other work on provcat is uncommitted, so that the move does not collide with it.
2. Move src/provcat, tests and pyproject.toml into packages/provcat with git mv, so that the history follows the files.
3. Write the workspace root pyproject.toml, and run uv lock.
4. Run the provcat unit and integration tests. They must pass with no change to their code.
5. Fix the paths in README.md and in docs/.

Done when: uv run provc --help works from the root, and the provcat tests pass.

Phase 1: run context and evidence

1. Write RunContext, with the run ID, the environment, the actor and the evidence mode. A context variable holds the current run, so that code inside a tool can find it.
2. Write EvidenceEvent, EvidenceRecorder, JsonlSink and the payload store. The payload store writes each payload under its SHA-256.
3. Write the hash chain and the manifest.
4. Write recorder.step(type, ...), which opens a span and writes the event. When the step fails, it writes step_failed with the name of the step and the error.
5. Write the commit of the bundle through provcat.
6. Write the strict and lenient modes.

Tests:

• A run with three steps writes a chain that verifies.
• A changed or removed event fails the verification.
• A sink that fails stops the run in strict mode and does not stop it in lenient mode.
• The bundle commits to a versioned bucket and reads back at the same version. The tests use a moto S3 bucket. A file store has no commits, so there the bundle stays unpinned.

Phase 2: resolver, broker and snapshots

1. Write Grant and StubBroker. The stub grants a URI only if the URI is in the allow list of the run. The environment segment of the URI must also match the environment of the run. The stub grants only the mode mounted_snapshot. (Later, the stub also grants pinned_read, for the run’s experiment, which provrun reads itself. provrun’s page has the rules now.) A physical URI, such as file://, has no environment segment, so the stub grants one only in dev.
2. Write ProvcatResolver. It calls pin and then makes a snapshot of the pinned version.
3. Add snapshots to provcat as Catalog.snapshot(uri), which returns a Snapshot with the pinned URI, the path and the SHA-256. A snapshot lives at .provcat/snapshots/<kind>/<sha256>/<name> in the cache, where the kind is file or folder. The SHA-256 is of the content. For a folder, it is the SHA-256 of a list of its files and their hashes. So two versions with the same kind, the same name and the same content share one snapshot. A snapshot is read-only. provcat reuses it only if its hash still matches, and replaces it otherwise. It is made with a copy-on-write clone where the disk has one, and with a copy elsewhere.

When this plan was written, the provcat cache kept one path for each store and key, whatever the version. It now keeps fixed versions in their own folders, but the path without a version still moves. A later fetch of a moved branch rewrites that path in place. A mount of that path is not safe, so mounts use snapshots only.

Tests:

• The stub denies a URI that is not in the allow list.
• The stub denies a URI from another environment.
• A snapshot does not change after the branch moves and the cache fetches the new version.
• Every grant and every snapshot writes its evidence event.

Phase 3: specs, stager and backends

1. Write AgentSpec and RunSpec as dataclasses that convert to JSON and back.
2. Write InProcessStager. It copies each snapshot into the input folder of the run, with a copy-on-write clone where the disk has one. The PydanticAI backend mounts that folder read-only at /input, and the output folder writable at /output. A clone shares no inode with the snapshot, so a change in one run cannot reach the snapshot that other runs use.
3. Write the Backend interface with start, poll, stream, cancel and result, all asynchronous.
4. Write LiteLLMBackend for one text call or one multimodal call.
5. Write PydanticAIBackend, with the filesystem toolset and grep_file.
6. Write ToolRegistry, which loads tools from Python entry points and from a tools folder. Each module in a tools folder names its tools in a list called TOOLS.
7. Write Outputter. It hashes each file in /output. RunContext commits the folder through provcat to run://.../<run_id>/output.
8. Write the PydanticAI importer. It puts the whole message history into one trace_imported event with confidence parsed. The wrappers have already recorded each model call and tool call.

Tests:

• Use the PydanticAI TestModel and FunctionModel, and a LiteLLM mock, so that the tests use no network and no keys.
• An agent that reads /input/<name> and writes /output/answer.txt gives a bundle with every event from the list in section “Classes”.
• An agent cannot write to /input.

Phase 4: the provr command

1. Write provr run <spec.yaml>, which runs one agent spec and prints the run ID.
2. Write provr show <run_id>, which prints the events of a run.
3. Write provr verify <run_id>, which checks the hash chain and the hashes of the payloads and the outputs.
4. Write docs/provrun.md, with one example that reads a file URI and one that reads a lakeFS URI.

Done when: the two examples run from a clean checkout, and provr verify passes on both bundles.