Helper functions for file operations.
| Method | Description |
|---|---|
| upload_file | Upload a file to a bucket |
| upload_files | Upload multiple files to a bucket, recursive directory upload is also supported |
| download_file | Download a file from a bucket |
| download_files | Download multiple files from a bucket to local directory preserving tree structure |
upload_file(client, project_name, file_path, bucket_name='default', file_name=None)
Upload a file to a bucket
Helper function to upload a file to a bucket. A signed url will be generated and used to upload the file.
importubiopsconfiguration=ubiops.Configuration()
# Configure API token authorizationconfiguration.api_key['Authorization'] ='Token <YOUR_API_TOKEN>'# Defining host is optional and default to https://api.ubiops.com/v2.1configuration.host="https://api.ubiops.com/v2.1"# Enter a context with an instance of the API clientapi_client=ubiops.ApiClient(configuration)
project_name='project_name_example'# str file_path='path/to/local_file_example'# strbucket_name='default'# str (optional)file_name='remote_file_example'# str (optional) - may contain prefixes# Upload a fileubiops.utils.upload_file(api_client, project_name, file_path, bucket_name=bucket_name, file_name=file_name)
# Close the connectionapi_client.close()| Name | Type | Notes |
|---|---|---|
| client | ubiops.ApiClient | |
| project_name | str | |
| file_path | str | |
| bucket_name | str | [optional] [default to 'default'] |
| file_name | str | [optional] [default to None (use local file name)] |
upload_files(client, project_name, file_paths, bucket_name='default', file_prefix=None, parallel_uploads=5)
Upload multiple files and directories with files to a bucket
Helper function to upload a list of files and directories to a bucket. When a directory path is specified, the directory is recursively uploaded to a bucket preserving tree structure.
Signed urls will be generated and used to upload the files.
If a 'file_prefix' is provided, it will be added to all file names. This allows putting all files and uploaded directories in a 'sub-directory' in the bucket.
By specifying the number of parallel uploads, the uploads can be made faster.
importubiopsconfiguration=ubiops.Configuration()
# Configure API token authorizationconfiguration.api_key['Authorization'] ='Token <YOUR_API_TOKEN>'# Defining host is optional and default to https://api.ubiops.com/v2.1configuration.host="https://api.ubiops.com/v2.1"# Enter a context with an instance of the API clientapi_client=ubiops.ApiClient(configuration)
project_name='project_name_example'# str file_paths= [
'path/to/local_file_1',
'path/to/other/local_file_2',
'path/to/local_directory',
] # list[str]bucket_name='default'# str (optional)file_prefix='remote/directory'# str (optional) - may contain slashesparallel_uploads=10# int (optional) - number of parallel uploads, default - 5# Upload list of filesubiops.utils.upload_files(
api_client,
project_name,
file_paths,
bucket_name=bucket_name,
file_prefix=file_prefix, parallel_uploads=parallel_uploads
)
# Close the connectionapi_client.close()| Name | Type | Notes |
|---|---|---|
| client | ubiops.ApiClient | |
| project_name | str | |
| file_paths | list[str] | |
| bucket_name | str | [optional] [default to 'default'] |
| file_prefix | str | [optional] [default to None (files are uploaded to bucket root)] |
| parallel_uploads | int | [optional] [default to 5] |
download_file(client, project_name, bucket_name='default', file_name=None, file_uri=None, output_path='.', stream=True, chunk_size=8192)
Download a file from a bucket
Helper function to download a file from a bucket. A signed url will be generated and used to download the file.
importubiopsconfiguration=ubiops.Configuration()
# Configure API token authorizationconfiguration.api_key['Authorization'] ='Token <YOUR_API_TOKEN>'# Defining host is optional and default to https://api.ubiops.com/v2.1configuration.host="https://api.ubiops.com/v2.1"# Enter a context with an instance of the API clientapi_client=ubiops.ApiClient(configuration)
project_name='project_name_example'# stroutput_path='.'# str (optional) - path to file or directory# Download a file using a file_urifile_uri='ubiops-file://default/remote_file_example'# str (optional)ubiops.utils.download_file(
api_client, project_name, file_uri=file_uri, output_path=output_path, stream=True, chunk_size=8192
)
# Or download a file using a bucket_name and file_namebucket_name='default'# str (optional)file_name='remote_file_example'# str (optional) - may contain prefixesubiops.utils.download_file(
api_client,
project_name,
bucket_name=bucket_name,
file_name=file_name,
output_path=output_path,
stream=True,
chunk_size=8192
)
# Close the connectionapi_client.close()| Name | Type | Notes |
|---|---|---|
| client | ubiops.ApiClient | |
| project_name | str | |
| bucket_name | str | [optional] [default to 'default'] |
| file_name | str | [optional] |
| file_uri | str | [optional] |
| output_path | str | [optional] [default to '.' (current working directory)] |
| stream | bool | [optional] [default to True] |
| chunk_size | int | [optional] [default to 8192] |
download_files(client, project_name, bucket_name='default', prefix=None, output_path='.', stream=True, chunk_size=8192, parellel_download=5)
Download multiple files from a bucket
Helper function to download multiple files or all files from a bucket. A signed url will be generated and used to download the file.
If a 'prefix' is provided, only files with this prefix will be downloaded. This is useful to download files from a 'sub-directory' in the bucket.
By specifying the number of parallel downloads, the downloads can be made faster.
importubiopsconfiguration=ubiops.Configuration()
# Configure API token authorizationconfiguration.api_key['Authorization'] ='Token <YOUR_API_TOKEN>'# Defining host is optional and default to https://api.ubiops.com/v2.1configuration.host="https://api.ubiops.com/v2.1"# Enter a context with an instance of the API clientapi_client=ubiops.ApiClient(configuration)
project_name='project_name_example'# stroutput_path='.'# str (optional) - path to file or directory# Download all files from "default" bucketubiops.utils.download_files(
api_client, project_name, output_path=output_path, stream=True, chunk_size=8192
)
# Or download files from a certain location in the bucket with 10 parallel downloadsbucket_name='bucket_name'# str (optional)prefix='remote/directory'# str (optional) - may contain slashesubiops.utils.download_files(
api_client,
project_name,
bucket_name=bucket_name,
prefix=prefix,
output_path=output_path,
stream=True,
chunk_size=8192,
parallel_downloads=10
)
# Close the connectionapi_client.close()| Name | Type | Notes |
|---|---|---|
| client | ubiops.ApiClient | |
| project_name | str | |
| bucket_name | str | [optional] [default to 'default'] |
| prefix | str | [optional] |
| output_path | str | [optional] [default to '.' (current working directory)] |
| stream | bool | [optional] [default to True] |
| chunk_size | int | [optional] [default to 8192] |
| parallel_download | int | [optional] [default to 5] |