provcat.stores

Stores read and write physical URIs. One driver class serves one kind of storage.

This follows the Perl URI module: the URI scheme picks the class. A store is a named instance of a driver with options, so two stores, such as s3:// and a configured s3-acme://, can use the same driver with different endpoints. Register a new driver with register_driver.

A driver gives four things:

stat(uri, cache): one Entry, a file or a folder, or NotFound.
list(uri, cache): the Entries directly inside a folder.
pin(uri, cache): the URI with any moving name (a branch) replaced by a fixed one.
fetch(uri, cache): a local path that holds the object.

Every driver but github and hf can be written, and also gives:

put(uri, src, cache): write the local file src at uri, and return its Entry.
delete(uri, cache): delete the object at uri.

The lakefs driver and the object stores (s3, gcs and azure) also give commit and branch. An object store needs versioning on its bucket for a commit. lakefs also gives merge. github and hf refuse writes with a StoreError.

Every method takes the cache, because pinning a moving name records what it pointed at, and offline mode reads only the cache.

class Entry

source

Entry(uri: Uri, kind: str, size: int | None = None, version: str | None = None)

One file or folder in a store, as stat, list and put describe it.

Parameter	Type	Default	Description
`uri`	Uri	required	The physical URI of the file or folder
`kind`	str	required	“file” or “folder”
`size`	int \| None	`None`	The size in bytes; None for a folder, or when the store does not say
`version`	str \| None	`None`	A value that changes when the content changes

Entry.as_dict

source

Entry.as_dict() -> dict

The entry as a plain dict, as provc prints it.

Parameter	Type	Default	Description
returns	dict		The four fields, with the URI as text, ready for JSON

class Store

source

Store(name: str, options: dict | None = None, *, offline: bool = False)

The interface of a store driver: read, and maybe write, the physical URIs of one kind of storage.

A subclass sets driver, the name that driver: in the configuration picks, and is registered with register_driver.

Parameter	Type	Default	Description
`name`	str	required	The store’s name in the configuration, which is also the scheme of its URIs
`options`	dict \| None	`None`	The store’s options from the configuration, without driver; each is checked against option_types
`offline`	bool	`False`	Work from the cache alone; a driver raises OfflineMiss for what the cache does not hold

Store.option_types

source

Store.option_types() -> dict[str, type | tuple]

The options this driver takes, and the type of each.

Parameter	Type	Default	Description
returns	dict[str, type \| tuple]		Each option’s name and its type, or a tuple of types

Store.pin

source

Store.pin(uri: Uri, cache: LocalCache) -> Uri

Replace a moving name, such as a branch, with a fixed version. A store without versions returns uri.

Parameter	Type	Default	Description
`uri`	Uri	required	A physical URI in this store
`cache`	LocalCache	required	The cache, which records what a moving name pointed at
returns	Uri		The URI with any moving name replaced by a fixed version

Store.stat

source

Store.stat(uri: Uri, cache: LocalCache) -> Entry

Describe one file or folder.

Parameter	Type	Default	Description
`uri`	Uri	required	A physical URI in this store
`cache`	LocalCache	required	The cache, which offline mode reads
returns	Entry		The file or the folder at uri; NotFound when there is none

Store.list

source

Store.list(uri: Uri, cache: LocalCache) -> list[Entry]

List a folder, one level deep.

Parameter	Type	Default	Description
`uri`	Uri	required	A physical URI of a folder in this store
`cache`	LocalCache	required	The cache, which offline mode reads
returns	list[Entry]		The files and folders directly inside it

Store.fetch

source

Store.fetch(uri: Uri, cache: LocalCache) -> Path

Return a local path that holds a file or folder: a copy in the cache, or the file itself for a local store.

Parameter	Type	Default	Description
`uri`	Uri	required	A physical URI in this store
`cache`	LocalCache	required	The cache, where a remote store copies the file or folder
returns	Path		A local path that holds the file or folder

Store.read

source

Store.read(uri: Uri, cache: LocalCache) -> bytes

Fetch a file and return its bytes.

Parameter	Type	Default	Description
`uri`	Uri	required	A physical URI of a file in this store
`cache`	LocalCache	required	The cache, as for fetch
returns	bytes		The bytes of the file

Store.put

source

Store.put(uri: Uri, src: Path, cache: LocalCache) -> Entry

Write a local file at uri. A store that cannot be written, as this base, raises StoreError.

Parameter	Type	Default	Description
`uri`	Uri	required	The physical URI to write
`src`	Path	required	The local file to write
`cache`	LocalCache	required	The cache
returns	Entry		The file written

Store.delete

source

Store.delete(uri: Uri, cache: LocalCache) -> None

Delete the object at uri. A store that cannot be written, as this base, raises StoreError.

Parameter	Type	Default	Description
`uri`	Uri	required	The physical URI of the file to delete
`cache`	LocalCache	required	The cache

register_driver

source

register_driver() -> type[Store]

Make a driver class available to the configuration under its driver name. A later class with the same name replaces it.

Parameter	Type	Default	Description
returns	type[Store]		The same class, so this works as a class decorator