StorageServices API¶
gfslib.storage.client.StorageServices
¶
Client for the server storage API.
Initialize with the base storage service URL. Examples of base URL:
- https://.../api/ws/
base_url = base_url.rstrip('/')
instance-attribute
¶
timeout = timeout
instance-attribute
¶
__init__(base_url, timeout=30.0)
¶
compute_sha256(path)
staticmethod
¶
Compute SHA-256 hex digest of a file.
delete(remote_path)
¶
Remove one or more remote files or directories recursively.
download(remote_path, dest=None, byte_range=None, ignore_sha=False, write_chunk_size=10 * 1024 * 1024, validate_paths=True, ignore_missing=False, preserve_paths=False, overwrite=True, verify_sha=False)
¶
Download a file or multiple files.
If the remote path is a single string, a single file will be downloaded
and returned as bytes (or written to dest if provided).
For multiple files, pass a list of strings in remote_path and provide
dest as a destination folder. The server returns a stream in the form
dest. Set ignore_missing to skip requested files that do not
exist instead of failing the entire multi-file download.
Batch downloads flatten paths by default for compatibility. Set preserve_paths=True to retain workspace-relative paths. Colliding names in a batch raise ValueError. overwrite=False protects existing files. verify_sha=True checks server SHA-256 metadata; it cannot be combined with ignore_sha or a byte range. Files are staged until complete.
download_directory(remote_path, dest, overwrite=False, verify_sha=False, write_chunk_size=10 * 1024 * 1024)
¶
Download directory contents, preserving paths relative to remote_path.
Existing files are rejected unless overwrite=True. Completed files remain if a later file fails; each individual file is published atomically. Empty directories are not represented in the GF multifile stream.
exists(path)
¶
Return whether a remote file or directory exists.
is_dir(path)
¶
Return whether a remote path exists and is a directory.
is_file(path)
¶
Return whether a remote path exists and is a file.
ls(path=None, recursive=False, include_directories=False)
¶
List remote files (short). Returns the requests.Response object.
ls_long(path=None, recursive=False, include_directories=False)
¶
List remote files (long). Returns the requests.Response object.
metadata(filepaths, ignore_sha=False)
¶
Get metadata for the provided filepaths.
mkdir(paths)
¶
Create directories and return per-path status/error records.
path_info(path)
¶
Return whether a path exists and its server-provided entry details.
plan_sync(local_dir, remote_prefix='', direction='upload', ignore_sha=False, include=None, exclude=())
¶
Plan a one-way sync without modifying files.
Patterns are case-sensitive shell wildcards against relative paths with forward slashes. Exclusions win; an empty include list selects nothing. No files are deleted. ignore_sha=True plans every selected file for transfer, including files already present at the destination.
rename(source, destination, overwrite=False)
¶
Rename or move a remote path. Overwriting must be explicitly enabled.
rmdir(paths, recursive=False)
¶
Remove directories; only empty directories are removed by default.
Inspect returned records for individual failures, including nonempty directories. HTTP failures raise requests.HTTPError.
set_api_key(key)
¶
Set the X-API-Key to use for requests.
sync_local_to_remote(local_dir, remote_prefix='', ignore_sha=False, dry_run=False, include=None, exclude=())
¶
Upload missing/changed selected files, without deleting remote files.
Returns relative path -> uploaded/skipped (or uploaded (dry-run)). Use plan_sync() for structured actions and comparison reasons.
sync_remote_to_local(local_dir, remote_prefix='', ignore_sha=False, dry_run=False, include=None, exclude=(), verify_sha=False, overwrite=True)
¶
Download missing/changed selected files, without deleting local files.
Dry runs do not create the destination directory. Changed existing files are replaced unless overwrite=False. verify_sha checks transferred data independently of the ignore_sha option for planning comparisons.
upload(remote_path, data)
¶
Upload data to remote_path using PUT.
data can be raw bytes, a string (will be encoded as utf-8), or a path
to a local file to stream.
upload_file(remote_path, local_path)
¶
Stream a local file. Missing files raise FileNotFoundError.
upload_many(files, file_comparison='TimeModified', chunk_size=1024 * 1024)
¶
Stream remote-path -> local-file mappings in one GF upload request.
Each file is hashed and then streamed in bounded chunks. The server's fileComparison policy controls replacement (default TimeModified). A failed request can leave earlier files uploaded; this is not atomic across files and is never retried automatically. HTTP errors raise.
upload_text(remote_path, text)
¶
Upload UTF-8 text, even when the text names an existing local file.