deriva.transfer.upload package

Submodules

deriva.transfer.upload.deriva_upload module

class deriva.transfer.upload.deriva_upload.DerivaUpload(config_file=None, credential_file=None, server=None, dcctx_cid=None)

Bases: object

Base class for upload tasks. Encapsulates a catalog instance and a hatrac store instance and provides some common and reusable functions.

This class is not intended to be instantiated directly, but rather extended by a specific implementation.

DefaultConfigFileName = 'config.json'

DefaultServerListFileName = 'servers.json'

DefaultTransferStateBaseName = '.deriva-upload-state'

DefaultTransferStateFileName = '%s-%s.json'

acquire_dependent_locks(directory)

static archive_preprocessing_enabled(asset_mapping)

cancel()

cleanup()

cleanupTransferState()

defaultFileCallback(**kwargs)

delTransferState(file_path)

delete_dependent_locks(directory)

static find_file_in_dir_hierarchy(filename, path)

Find all instances of a filename in the entire directory hierarchy specified by path.

getAssetMapping(file_path)

Parameters:file_path – Returns:

static getCatalogTable(asset_mapping, metadata_dict=None)

classmethod getConfigPath()

This method must be implemented by subclasses.

getCurrentConfigFilePath()

getDefaultConfigFilePath()

classmethod getDefaultServer()

getDeployedConfigFilePath()

classmethod getDeployedConfigPath()

classmethod getFileDisplayName(file_path, asset_mapping=None)

static getFileHashes(file_path, hashes=frozenset({'md5'}))

static getFileSize(file_path)

getFileStatusAsArray()

getRemoteConfig()

classmethod getServers()

This method must be implemented by subclasses.

getTransferState(file_path)

getTransferStateFileName()

getTransferStateStatus(file_path)

getUpdatedConfig()

classmethod getVersion()

This method must be implemented by subclasses.

getVersionCompatibility()

static guessContentType(file_path)

initialize(cleanup=False)

static interpolateDict(src, dst, allowNone=False)

interrupt_handler(signum, frame)

static isFileNewer(src, dst)

isVersionCompatible()

loadTransferState(directory, purge=False)

static pruneDict(src, dst, stringify=True)

reset()

scanDirectory(root, abort_on_invalid_input=False, purge_state=False)

Parameters:

• root –
• abort_on_invalid_input –
• purge_state –

Returns:

setConfig(config_file)

setCredentials(credentials)

setServer(server)

setTransferState(file_path, transfer_state)

set_dcctx_cid(cid)

uploadFile(file_path, asset_mapping, match_groupdict, callback=None)

Primary API subclass function. :param file_path: :param asset_mapping: :param match_groupdict: :param callback: :return:

uploadFiles(status_callback=None, file_callback=None)

validateFile(root, path, name)

writeTransferState()

class deriva.transfer.upload.deriva_upload.Enum

Bases: tuple

class deriva.transfer.upload.deriva_upload.FileUploadState(state=2, status='Pending', result=None)

Bases: object

asdict()

class deriva.transfer.upload.deriva_upload.GenericUploader(config_file=None, credential_file=None, server=None, dcctx_cid=None)

Bases: deriva.transfer.upload.deriva_upload.DerivaUpload

classmethod getConfigPath()

This method must be implemented by subclasses.

classmethod getServers()

This method must be implemented by subclasses.

classmethod getVersion()

This method must be implemented by subclasses.

classmethod setServers(servers)

class deriva.transfer.upload.deriva_upload.UploadEntry(asset_group, asset_mapping, groupdict, path)

Bases: object

deriva.transfer.upload.deriva_upload_cli module

class deriva.transfer.upload.deriva_upload_cli.DerivaUploadCLI(uploader, description, epilog)

Bases: deriva.core.base_cli.BaseCLI

main()

static upload(uploader, data_path, hostname, catalog=1, token=None, config_file=None, credential_file=None, no_update=False, purge=False, dry_run=False, output_file=None)

Module contents

exception deriva.transfer.upload.DerivaUploadCatalogCreateError

Bases: RuntimeError

exception deriva.transfer.upload.DerivaUploadCatalogUpdateError

Bases: RuntimeError

exception deriva.transfer.upload.DerivaUploadConfigurationError

Bases: ValueError

exception deriva.transfer.upload.DerivaUploadError

Bases: RuntimeError