provcat.files

File handles that Catalog.open_file returns.

A reader reads one pinned version that the catalog fetched into the cache. Each read, readline or readall runs the read_block spot once with its byte range, and close runs the close spot with every range read. A writer writes to a temporary file, and close puts it in the store through the put spot. A writer used in a with block that ends in an error puts nothing, and so does a writer that is dropped without a close. A writer opened with autocommit off puts at commit(), as fsspec transactions expect.

class CatalogReader

CatalogReader(uri: Uri, pinned: Uri, path: Path, block: Callable[..., Any], done: Callable[[CatalogReader], None])

A read-only handle on one pinned version of a file.

size is the size at open; a read never goes past it. ranges lists the byte ranges handed out so far, merged, as (start, end). A wrapper that reads ahead, such as io.TextIOWrapper, makes them larger than what its caller used.

Parameter	Type	Default	Description
`uri`	Uri	required	The URI as opened, logical or physical
`pinned`	Uri	required	The pinned physical URI that the handle reads
`path`	Path	required	The local copy of that version in the cache
`block`	Callable[…, Any]	required	Called as block(start, length, read) for each read; it runs the read_block spot around read
`done`	Callable[[CatalogReader], None]	required	Called with the handle at close; it runs the close spot

CatalogReader.readable

CatalogReader.readable() -> bool

A reader can be read.

Parameter	Type	Default	Description
returns	bool		Always True

CatalogReader.seekable

CatalogReader.seekable() -> bool

A reader can seek.

Parameter	Type	Default	Description
returns	bool		Always True

CatalogReader.tell

CatalogReader.tell() -> int

Return the position in the file.

Parameter	Type	Default	Description
returns	int		The position, in bytes from the start

CatalogReader.seek

CatalogReader.seek(offset: int, whence: int = 0) -> int

Move to a position in the file.

Parameter	Type	Default	Description
`offset`	int	required	The new position, counted from whence
`whence`	int	`0`	io.SEEK_SET, io.SEEK_CUR or io.SEEK_END
returns	int		The new position, in bytes from the start

CatalogReader.readinto

CatalogReader.readinto(buf: bytearray | memoryview) -> int

Read into buf, no further than the size at open. It runs the read_block spot.

Parameter	Type	Default	Description
`buf`	bytearray \| memoryview	required	A writable buffer to fill
returns	int		The number of bytes read; 0 at the end of the file

CatalogReader.readall

CatalogReader.readall() -> bytes

Read to the end of the file. It runs the read_block spot.

Parameter	Type	Default	Description
returns	bytes		The bytes up to the size at open

CatalogReader.read

CatalogReader.read(size: int | None = -1) -> bytes

Read up to size bytes. It runs the read_block spot.

Parameter	Type	Default	Description
`size`	int \| None	`-1`	The most bytes to read; -1 or None reads to the end
returns	bytes		The bytes read; empty at the end of the file

CatalogReader.readline

CatalogReader.readline(size: int | None = -1) -> bytes

Read one line. It runs the read_block spot.

Parameter	Type	Default	Description
`size`	int \| None	`-1`	The most bytes to read; -1 or None reads to the end of the line
returns	bytes		One line, with its newline when it has one

CatalogReader.close

CatalogReader.close() -> None

Close the handle, once; a second close does nothing.

CatalogReader.abandon

CatalogReader.abandon() -> None

Close without the close spot, for a handle that its caller never got.

class CatalogWriter

CatalogWriter(uri: Uri, done: Callable[[CatalogWriter, Path], None], autocommit: bool = True)

A handle that writes a new file into a temporary file, and then puts it in the store.

Close, or commit when autocommit is off, puts it at uri; entries then holds what the store wrote.

Parameter	Type	Default	Description
`uri`	Uri	required	The URI to write, as opened
`done`	Callable[[CatalogWriter, Path], None]	required	Called with the handle and the temporary file to put it; it runs the close spot and the put
`autocommit`	bool	`True`	Put at close; off, put at commit()

CatalogWriter.writable

CatalogWriter.writable() -> bool

A writer can be written.

Parameter	Type	Default	Description
returns	bool		Always True

CatalogWriter.write

CatalogWriter.write(b: bytes | bytearray | memoryview) -> int

Write bytes to the temporary file; nothing reaches the store before the put.

Parameter	Type	Default	Description
`b`	bytes \| bytearray \| memoryview	required	The bytes to add to the temporary file
returns	int		The number of bytes written

CatalogWriter.tell

CatalogWriter.tell() -> int

Return the position in the temporary file.

Parameter	Type	Default	Description
returns	int		The number of bytes written so far

CatalogWriter.close

CatalogWriter.close() -> None

Close the handle. With autocommit it puts the file through the close spot; without, commit() or discard() comes later.

CatalogWriter.commit

CatalogWriter.commit() -> None

Put the file, when autocommit is off. ValueError when it was already put or discarded.

CatalogWriter.discard

CatalogWriter.discard() -> None

Close without writing to the store.

CatalogWriter.abandon

CatalogWriter.abandon() -> None

Close without writing to the store, for a handle that its caller never got; the same as discard.