deriva.transfer.restore package

Submodules

deriva.transfer.restore.deriva_restore module

class deriva.transfer.restore.deriva_restore.DerivaRestore(*args, **kwargs)[source]

Bases: object

Restore a DERIVA catalog from a bag archive or directory. Core restore logic re-purposed from ErmrestCatalog.clone_catalog().

BASE_ASSETS_INPUT_PATH = 'assets'
BASE_DATA_INPUT_PATH = 'records/{}/{}.json'
RESTORE_STATE_URL = 'tag:isrd.isi.edu,2019:restore-status'
static check_column_compatibility(src, dst)[source]

Check compatibility of source and destination column definitions.

cleanup_restored_catalog()[source]
copy_cdef(column)[source]

Copy column definition with conditional parts.

copy_kdef(key)[source]
copy_sdef(schema)[source]

Copy schema definition structure with conditional parts for cloning.

copy_tdef_core(table)[source]

Copy table definition structure with conditional parts excluding fkeys.

copy_tdef_fkeys(table)[source]

Copy table fkeys structure.

get_json_recordset(data, chunk_size, after=None, after_column='RID')[source]
get_table_path(sname, tname, is_bag)[source]
load_json_file(file_path)[source]
open_json_stream_file(table_path)[source]

Open a JSON-Stream file for reading, caller is responsible for closing.

prune_parts(dest, *extra_victims)[source]
restore(**kwargs)[source]

Perform the catalog restore operation. The restore process is broken up into six phases:

  1. Pre-process the input path.
    • If the input path is a file, it is assumed that it is a compressed archive file that can be extracted

    into an input directory via a supported codec: tar,`tgz`,`bz2`, or zip. - If the input directory is a valid _bag_ directory structure, the bag will be materialized.

  2. The catalog schema will be restored first. The schema is restored from a ERMRest JSON schema document file.

    The schema document file must be named catalog-schema.json and must appear at the root of the input directory. The restore process can be configured to exclude the restoration of an enumerated set both schema and tables.

  3. The catalog table data will be restored, if present. The table date restoration process is resilient to

    interruption and may be restarted. However, if the catalog schema or data is mutated outside of the scope of the restore function in-between such restarts, the restored catalog’s consistency cannot be guaranteed. The restore process can be configured to exclude the restoration of table data for a set of tables.

  4. The catalog foreign keys will be restored.

  5. The catalog assets will be restored, if present.

  6. On success, the restore state marker annotations will be deleted and the catalog history will be truncated.

Parameters:kwargs –
Returns:
set_config(config)[source]
set_credentials(credentials)[source]
upload_assets()[source]

deriva.transfer.restore.deriva_restore_cli module

class deriva.transfer.restore.deriva_restore_cli.DerivaRestoreCLI(description, epilog, **kwargs)[source]

Bases: deriva.core.base_cli.BaseCLI

main()[source]

Module contents

exception deriva.transfer.restore.DerivaRestoreAuthenticationError[source]

Bases: Exception

exception deriva.transfer.restore.DerivaRestoreAuthorizationError[source]

Bases: Exception

exception deriva.transfer.restore.DerivaRestoreConfigurationError[source]

Bases: ValueError

exception deriva.transfer.restore.DerivaRestoreError[source]

Bases: RuntimeError