deriva.transfer.upload package

Submodules

deriva.transfer.upload.deriva_upload module

class deriva.transfer.upload.deriva_upload.DerivaUpload(config_file=None, credential_file=None, server=None, dcctx_cid=None)[source]

Bases: object

Base class for upload tasks. Encapsulates a catalog instance and a hatrac store instance and provides some common and reusable functions.

This class is not intended to be instantiated directly, but rather extended by a specific implementation.

DefaultConfigFileName = 'config.json'
DefaultServerListFileName = 'servers.json'
DefaultTransferStateBaseName = '.deriva-upload-state'
DefaultTransferStateFileName = '%s-%s.json'
acquire_dependent_locks(directory)[source]
static archive_preprocessing_enabled(asset_mapping)[source]
cancel()[source]
cleanup()[source]
cleanupTransferState()[source]
defaultFileCallback(**kwargs)[source]
delTransferState(file_path)[source]
delete_dependent_locks(directory)[source]
static find_file_in_dir_hierarchy(filename, path)[source]

Find all instances of a filename in the entire directory hierarchy specified by path.

getAssetMapping(file_path)[source]
Parameters:file_path
Returns:
static getCatalogTable(asset_mapping, metadata_dict=None)[source]
classmethod getConfigPath()[source]

This method must be implemented by subclasses.

getCurrentConfigFilePath()[source]
getDefaultConfigFilePath()[source]
classmethod getDefaultServer()[source]
getDeployedConfigFilePath()[source]
classmethod getDeployedConfigPath()[source]
classmethod getFileDisplayName(file_path, asset_mapping=None)[source]
static getFileHashes(file_path, hashes=frozenset({'md5'}))[source]
static getFileSize(file_path)[source]
getFileStatusAsArray()[source]
getRemoteConfig()[source]
classmethod getServers()[source]

This method must be implemented by subclasses.

getTransferState(file_path)[source]
getTransferStateFileName()[source]
getTransferStateStatus(file_path)[source]
getUpdatedConfig()[source]
classmethod getVersion()[source]

This method must be implemented by subclasses.

getVersionCompatibility()[source]
static guessContentType(file_path)[source]
initialize(cleanup=False)[source]
static interpolateDict(src, dst, allowNone=False)[source]
interrupt_handler(signum, frame)[source]
static isFileNewer(src, dst)[source]
isVersionCompatible()[source]
loadTransferState(directory, purge=False)[source]
static pruneDict(src, dst, stringify=True)[source]
reset()[source]
scanDirectory(root, abort_on_invalid_input=False, purge_state=False)[source]
Parameters:
  • root
  • abort_on_invalid_input
  • purge_state
Returns:

setConfig(config_file)[source]
setCredentials(credentials)[source]
setServer(server)[source]
setTransferState(file_path, transfer_state)[source]
set_dcctx_cid(cid)[source]
uploadFile(file_path, asset_mapping, match_groupdict, callback=None)[source]

Primary API subclass function. :param file_path: :param asset_mapping: :param match_groupdict: :param callback: :return:

uploadFiles(status_callback=None, file_callback=None)[source]
validateFile(root, path, name)[source]
writeTransferState()[source]
class deriva.transfer.upload.deriva_upload.Enum[source]

Bases: tuple

class deriva.transfer.upload.deriva_upload.FileUploadState(state=2, status='Pending', result=None)[source]

Bases: object

asdict()[source]
class deriva.transfer.upload.deriva_upload.GenericUploader(config_file=None, credential_file=None, server=None, dcctx_cid=None)[source]

Bases: deriva.transfer.upload.deriva_upload.DerivaUpload

classmethod getConfigPath()[source]

This method must be implemented by subclasses.

classmethod getServers()[source]

This method must be implemented by subclasses.

classmethod getVersion()[source]

This method must be implemented by subclasses.

classmethod setServers(servers)[source]
class deriva.transfer.upload.deriva_upload.UploadEntry(asset_group, asset_mapping, groupdict, path)[source]

Bases: object

deriva.transfer.upload.deriva_upload_cli module

class deriva.transfer.upload.deriva_upload_cli.DerivaUploadCLI(uploader, description, epilog)[source]

Bases: deriva.core.base_cli.BaseCLI

main()[source]
static upload(uploader, data_path, hostname, catalog=1, token=None, config_file=None, credential_file=None, no_update=False, purge=False, dry_run=False, output_file=None)[source]

Module contents

exception deriva.transfer.upload.DerivaUploadCatalogCreateError[source]

Bases: RuntimeError

exception deriva.transfer.upload.DerivaUploadCatalogUpdateError[source]

Bases: RuntimeError

exception deriva.transfer.upload.DerivaUploadConfigurationError[source]

Bases: ValueError

exception deriva.transfer.upload.DerivaUploadError[source]

Bases: RuntimeError