Logo

Getting to know DERIVA

  • About DERIVA
    • Features
    • FAIRness
  • DERIVA Platform
    • Server Components
      • ERMrest
      • Hatrac
      • ERMresolve
      • deriva-web
    • Client Libraries
      • Python APIs (deriva-py)
      • Javascript Libraries (ERMrestJS)
    • Client Applications
      • Web Interface (Chaise)
      • Desktop clients (deriva-qt)
      • Commandline Clients (deriva-py)
  • Key Concepts
    • Old Way: Ad hoc files and directories
    • New Way: Model and manage the relationships between data
    • Assets
    • The Catalog Model
    • Representing entities in a catalog
    • Representing relationships between entities
      • Cardinality
        • One-to-one (1:1) relationships
        • One-to-many (1:m) relationships
        • Many-to-one (m:1) relationships
        • Many-to-many (m:n) relationships

Using DERIVA

  • Using the Chaise Data Browser
    • Example of a landing page
    • Filtering sidebar
    • Search results view
    • Record page
  • Managing Data in DERIVA with deriva-client
    • Installed Applications
      • Command-Line Interface (CLI) applications
      • Graphical User Interface (GUI) applications
    • Installer packages for Windows and MacOSX
    • Installing deriva-client from PyPi via pip
      • Installation Prerequisites
      • Installation Quickstart
        • Mac/Linux
        • Important Note: For MacOSX users running Python 3.5.x with pip version < 9.0.3
        • Windows
        • IMPORTANT NOTE: Python virtual environments versus user environments
      • Installation Procedure
        • Installation Sequence
    • Managing data with the datapath API (deriva-py)
    • Reference Documentation
    • Source Code
  • Starting a Deriva-Py Data Management Project
    • Python Client Bindings
    • Your Project-Specific Groups
      • Suggested Groups
      • Keeping Track of Group Identifiers
    • Your First Hatrac Namespace
    • Your First Catalog
      • Initial Catalog Content
      • Your First Catalog Policy
      • Static Policy Inheritance
      • Exposing the ERMrest Client Table
    • Extending Your Catalog
      • Adding a Table
      • Adding an Asset Table
      • Asset Configuration
      • Allowing Self-Service Editing by Writers
      • Preventing Attachments by Unrelated Writers
      • Pitfall of Foreign Key Policies
      • Adding a Column
      • Removing a Column
      • Removing a Table
      • Other Model Changes…?
    • Further Discussion
      • Curators versus Writers
      • Anonymous Access
      • Extending ERMrest Client Table Definition
      • Enumerable Catalogs support ERMresolve
  • Annotations Guide
    • Adapting the User Experience with Annotations
      • Base Model Annotation
      • Annotation support in Chaise/ERMRestJS
        • Contexts
        • Facet search
        • Formatting and presentation
        • Pseudo-columns logic and heuristics
        • Table Alternatives
        • Export
      • Annotation support in ioboxd
    • Base Model Annotation
      • Notation and Usage
        • Example to Set Annotation
      • Annotations
        • Tag: 2015 Display
        • Tag: 2015 Vocabulary (deprecated)
        • Tag: 2016 Ignore (Deprecated)
        • Tag: 2016 App Links
        • Tag: 2016 Immutable
        • Tag: 2016 Generated
        • Tag: 2016 Non Deletable
        • Tag: 2016 Visible Columns
        • Tag: 2017 Key Display
        • Tag: 2016 Foreign Key
        • Tag: 2016 Column Display
        • Tag: 2016 Table Display
        • Tag: 2016 Visible Foreign Keys
        • Tag: 2016 Table Alternatives
        • Tag: 2019 Export
        • Tag: 2016 Export (Deprecated)
        • Tag: 2017 Asset
        • Tag: 2018 Citation
        • Tag: 2018 Required
        • Tag: 2018 Indexing Preferences
        • Tag: 2019 Chaise Config
        • Tag: 2019 Source Definitions
      • Tag: 2021 Google Dataset
        • Context Names
      • Pattern Expansion
    • Annotation Contexts Used By Chaise
    • Facet JSON Structure
      • Structure
      • Logical operators
      • Data Source
      • Constraints
      • Extra Attributes
        • entity v.s. scalar facet
        • markdown_name
        • comment
        • hide_null_choice/hide_not_null_choice
        • bar_plot
        • order
      • hide_num_occurrences
      • Examples
        • Example 1
        • Example 2
        • More Examples
    • Facet Examples
      • Entity vs. Scalar
      • Choice Vs. Range
      • Facet Title
      • Open Vs. Close
      • Change order of scalar values
    • Template and Markdown Guide
      • Using annotations to create templates with Markdown
      • Mustache Templating
        • Variables
        • Examples
      • Handlebars Templating
        • Handlebars Paths
        • Helpers
        • Block Helpers
        • Using Arrays
        • Accessing keys with spaces and special characters
        • Subexpressions:
        • Boolean Helpers
      • Markdown Formatting
        • Table of Contents
        • Inline Vs. Block
        • Attributes
        • Examples
    • PreFormat Column Value
      • Syntax
        • Flags field:
        • Width field
        • Precision Field
        • Type Field
        • Named arguments:
      • Examples:
      • Syntax for Dates and Timestamps
        • Most common Year, Month, and Day tokens:
        • Most common Hour, minute, second, millisecond, and offset tokens:
    • Pseudo-Columns Logic And Heuristics
      • Table of contents
      • Where To Use
      • Syntax
        • Simple Syntax
        • General Syntax
        • Source Definition Attributes
        • Display Attributes
      • Logic And Heuristics
        • Displayname
        • Value
        • Sort
      • Virtual Column
      • Examples
        • Visible Column List
        • Visible ForeignKey List
        • Specific Pseudo Columns
    • Pseudo-Column Display
      • Accessing Pseudo-Column Values Using By Utilizing Source Definitions
      • Accessing Current Pseudo-Column Data
        • $self Data Structure
        • Examples
    • Pseudo Column in Templating Environments
      • 1. Defining Sources
      • 2. Defining Pseudo-Column Display
        • Accessing Columns and Fkeys
        • Accessing Sources (Wait For)
      • Examples
    • Table Alternatives
      • Definition and assumptions
      • Related Entities
      • Understanding Recordset
      • ERMrestJS Restrictions
      • Faceting
        • 1. Alternative Table For Main Table
        • 2. Alternative Table in Facet List
    • Export Annotation
      • Structure
      • How ERMrestJS Interperts It

Reference

  • Python Command-line Clients and API (deriva-py)
    • DataPath
      • Features
      • Limitations
      • Tutorials
    • ERMrest Model Management
      • Features
      • Limitations
      • Examples
        • Add Table to Schema
        • Add a Vocabulary Term Table
        • Add Column to Table
        • Remove a Column from a Table
        • Alter a Table
    • ErmrestCatalog
      • Catalog Binding
      • Client Credentials
      • High-Level API Factories
      • Low-Level HTTP Methods
      • HTTP Caching
    • Source Code and Module Documentation (deriva package)
      • Subpackages
        • deriva.config package
        • deriva.core package
        • deriva.transfer package
        • deriva.seo package
      • Module contents
    • deriva-acl-config
      • Using deriva-acl-config to configure ACLs
      • Config file format
        • The groups stanza
        • The group_list_table stanza
        • The acl_definitions stanza
        • The acl_bindings stanza
        • The catalog_acls stanza
        • The schema_acls stanza
        • The table_acls stanza
        • The column_acls stanza
        • The foreign_key_acls stanza
      • Security Considerations
    • deriva-annotation-config
      • Using deriva-annotation-config to configure annotations
      • Config file format
        • The known_attributes stanza
        • The schema_annotations stanza
        • The table_annotations stanza
        • The column_annotations stanza
        • The foreign_key_annotations stanza
    • deriva-hatrac-cli
      • Features
      • Common options
        • Hostname
        • Authentication
      • Namespace operation examples
        • List
        • Create Namespace
        • Delete Namespace
      • Object operation examples
        • Put an object
        • Get an object
        • Delete an object
      • ACL operation examples
        • Get ACLs
        • Set ACLs
        • Delete ACLs
    • deriva-download-cli
      • Features
      • Command-Line options
        • Positional arguments:
        • Optional arguments:
      • Configuration file format
        • Example configuration file:
        • Configuration file element: catalog
        • Configuration file element: bag
        • Configuration file element: env
      • Supported processors
      • Processor details
        • env
        • csv
        • json
        • json-stream
        • download
        • fetch
      • Supported transform_processors
      • Transform Processor details
        • strsub
        • interpolation
        • cat
    • deriva-sitemap-cli
      • Using deriva-sitemap-cli to create sitemaps
      • Examples
    • deriva-backup-cli
      • Features
      • Command-Line options
        • Positional arguments:
        • Optional arguments:
    • deriva-restore-cli
      • Features
      • Command-Line options
        • Positional arguments:
        • Optional arguments:
  • Desktop GUI Applications (deriva-qt)
    • Using deriva-auth to authenticate to DERIVA systems
      • TBD
    • Uploading files via DERIVA-Upload and DERIVA-Auth
      • From a desktop system
        • 1. Initial setup
        • 2. Uploading files
        • 3. Logging out
      • From a remote server
        • 1. Initial setup
        • 2. Getting an authentication token
        • 3. Uploading files
        • 4. Logging out
  • Catalog Management Command-line Clients and API (deriva-catalog-manage)
    • TBD
    • TBD
    • TBD
    • TBD
    • deriva.utils.catalog package
      • Subpackages
        • deriva.utils.catalog.components package
        • deriva.utils.catalog.manage package
  • Entity Relationship Store (ERMrest)
    • ermrest-registry-purge
      • Synopsis
      • Description
      • Options
      • Exit Status
      • Examples
    • Client table
      • Extensibility and Localization
      • Conservative default policy
      • Interface Guarantees
    • ERMrest API
      • URL Conventions
      • Resource and Service Model
        • Graph of Web Resources
        • Data Resource Naming Language
        • RESTful Operations Overview
        • HTTP Concurrency Control
        • Set-based Data Resources and Representations
    • Catalog Operations
      • Service Ad Retrieval
      • Catalog Creation
      • Catalog Retrieval
      • Catalog Deletion
      • Catalog Alias Creation
      • Catalog Alias Retrieval
      • Catalog Alias Update
      • Catalog Alias Deletion
    • Model Resource Naming
      • Catalog Names
      • Catalog Aliases
      • Generic Model Sub-Resources
        • Annotations
        • Comments
        • ACLs
        • ACL Bindings
      • Model Names
      • Schema Names
      • Table Names
        • Column Names
        • Key Names
        • Foreign Key Names
    • Model Operations
      • Schemata Retrieval
      • Bulk Schemata and Table Creation
      • Schema Creation
      • Schema Retrieval
      • Schema Alteration
      • Schema Deletion
      • Table List Retrieval
      • Table Creation
      • Table Retrieval
      • Table Alteration
      • Table Deletion
      • Column List Retrieval
      • Column Creation
      • Column Retrieval
      • Column Alteration
      • Column Deletion
      • Key List Retrieval
      • Key Creation
      • Key Retrieval
      • Key Alteration
      • Key Deletion
      • Foreign Key List Retrieval
      • Foreign Key Creation
      • Foreign Key Retrieval
      • Foreign Key Alteration
      • Foreign Key Deletion
      • Annotations
        • Annotation List Retrieval
        • Annotation Creation
        • Annotation Bulk Update
        • Annotation Retrieval
        • Annotation Deletion
      • Comments
        • Comment Creation
        • Comment Retrieval
        • Comment Deletion
      • Access Control Lists
        • Access Control Lists Retrieval
        • Bulk Access Control List Update
        • Access Control List Creation
        • Access Control List Retrieval
        • Access Control List Deletion
      • Access Control List Bindings
        • Access Control List Bindings Retrieval
        • Bulk Access Control List Binding Update
        • Access Control List Binding Creation
        • Access Control List Binding Retrieval
        • Access Control List Binding Deletion
    • Standard System Columns
      • Interface Guarantees
    • Access Control Core Concepts
      • Catalog Resource Hierarchy
        • Usage Scenarios
      • Access Modes
      • Policy Scoping and Resource Dependencies
      • Data-Dependent Policies
      • Policy Inheritance and Implication
      • Model-level ACLs
        • Catalog ACLs
        • Schema ACLs
        • Table ACLs
        • Column ACLs
        • Reference ACLs
      • Dynamic ACL Bindings
        • Dynamic Table ACLs
        • Dynamic Column ACLs
        • Dynamic Reference ACLs
    • Static ACL Technical Reference
      • Available Static ACL Names
      • ACL Representation
      • Restrictions on Wildcard Policies
    • Dynamic ACL Binding Technical Reference
      • Available Dynamic ACL Types
      • Dynamic ACL Binding Representation
        • Inheritence and False Binding
        • Projection Document
        • Effective ACL Projection Query
        • Projection Types
        • Scoping ACLs
    • Access Decision Introspection
      • Static Rights Summary
      • Row-Level Rights Summary
      • Predicting Access Decisions
    • Data Resource Naming
      • Entity Resolution Names
      • Entity Names
      • Attribute Names
      • Aggregate Names
      • Attribute Group Names
      • Attribute Binning
        • Supported Column Types for Binning
      • Rights Summary
      • Data Paths
        • Path Root
        • Path Filters
        • Entity Links
        • Table Instance Aliases
        • Path Context Reset
      • Filter Language
        • Unary Filter Predicate
        • Binary Filter Predicate
        • Negated Filter
        • Parenthetic Filter
        • Conjunctive Filter
        • Disjunctive Filter
        • Conjunctive Normal Form
        • Disjunctive Normal Form
      • Sort Modifier
      • Paging Modifiers
        • Before Modifier
        • After Modifier
      • Accept Query Parameter
      • Download Query Parameter
      • Defaults Query Parameter
      • Nondefaults Query Parameter
      • Onconflict Query Parameter
      • Limit Query Parameter
      • Data Paging
    • Data Operations
      • Entity Resolution
      • Entity Creation
        • Entity Creation with Defaults
        • Idempotent Entity Creation
      • Entity Update
      • Entity Retrieval
      • Entity Deletion
      • Attribute Retrieval
      • Attribute Deletion
      • Attribute Group Retrieval
      • Attribute Group Update
        • Attribute Group Update with Renaming
      • Aggregate Retrieval
    • History Resource Naming
      • History Range
      • Historical ACLs
      • Historical ACL Bindings
      • Historical Annotations
      • Historical Attributes
    • History Operations
      • History Range Discovery
      • History Range Truncation
      • Amend Historical ACLs
      • Amend Historical ACL Bindings
      • Amend Historical Annotations
      • Redact Historical Attributes
      • Suppress Table History Collection
  • Object Storage (Hatrac)
    • Installing
      • Install code
      • Basic testing
      • Configure web stack
      • Example Hatrac configuration
      • REST API testing
      • Example Webauthn2 configuration
      • Example Apache configuration
      • Working with SE-Linux
        • Errors regarding WSGI socket
    • How-tos
      • Background
      • Examples
        • Deploying
        • Authentication using Database Providers
        • Listing the Root Namespace
        • Creating a Nested Namespace
        • Creating an Object
        • Retrieving an Object
        • Updating an Object
        • Retrieving an Object Version
        • Deleting an Object Version
        • Deleting an Object
    • REST API
      • Contents
        • Main Topics
        • Quick Links to Operations
      • URL Conventions
      • Resource Naming and Lifecycle Rules
        • Object Version Naming
        • Referential Stability
      • Root Namespace Resource
        • Namespace Listing Retrieval
        • Namespace Listing Metadata Retrieval
        • Deletion of Root Namespace Forbidden
      • Nested Namespace Resources
        • Nested Namespace Creation
        • Nested Namespace Listing Retrieval
        • Nested Namespace Deletion
      • Object Resources
        • Object Creation and Update
        • Object Retrieval
        • Object Metadata Retrieval
        • Object Version List Retrieval
        • Object Deletion
      • Object Version Resources
        • Object Version Creation
        • Object Version Retrieval
        • Object Version Metadata Retrieval
        • Object Version Deletion
      • Metadata Sub-Resources
        • Lifecycle and Ownership
        • Metadata Collection Retrieval
        • Metadata Value Retrieval
        • Metadata Value Creation and Update
        • Metadata Value Deletion
      • Access Control List Sub-Resources
        • Lifecycle and Ownership
        • Access Control Retrieval
        • Access Control List Retrieval
        • Access Control List Entry Retrieval
        • Access Control List Update
        • Access Control List Deletion
      • Chunked Upload Resources
        • Chunked Upload Job Creation
        • Chunked Upload Job Listing Retrieval
        • Chunk Upload
        • Chunked Upload Job Finalization
        • Chunked Upload Job Status Retrieval
        • Chunked Upload Job Cancellation
  • JavaScript Client Libraries (ERMrestJS)
    • Base Model Annotation
      • Notation and Usage
        • Example to Set Annotation
      • Annotations
        • Tag: 2015 Display
        • Tag: 2015 Vocabulary (deprecated)
        • Tag: 2016 Ignore (Deprecated)
        • Tag: 2016 App Links
        • Tag: 2016 Immutable
        • Tag: 2016 Generated
        • Tag: 2016 Non Deletable
        • Tag: 2016 Visible Columns
        • Tag: 2017 Key Display
        • Tag: 2016 Foreign Key
        • Tag: 2016 Column Display
        • Tag: 2016 Table Display
        • Tag: 2016 Visible Foreign Keys
        • Tag: 2016 Table Alternatives
        • Tag: 2019 Export
        • Tag: 2016 Export (Deprecated)
        • Tag: 2017 Asset
        • Tag: 2018 Citation
        • Tag: 2018 Required
        • Tag: 2018 Indexing Preferences
        • Tag: 2019 Chaise Config
        • Tag: 2019 Source Definitions
      • Tag: 2021 Google Dataset
        • Context Names
      • Pattern Expansion
    • Annotation Contexts Used By Chaise
    • Template and Markdown Guide
      • Using annotations to create templates with Markdown
      • Mustache Templating
        • Variables
        • Examples
      • Handlebars Templating
        • Handlebars Paths
        • Helpers
        • Block Helpers
        • Using Arrays
        • Accessing keys with spaces and special characters
        • Subexpressions:
        • Boolean Helpers
      • Markdown Formatting
        • Table of Contents
        • Inline Vs. Block
        • Attributes
        • Examples
    • PreFormat Column Value
      • Syntax
        • Flags field:
        • Width field
        • Precision Field
        • Type Field
        • Named arguments:
      • Examples:
      • Syntax for Dates and Timestamps
        • Most common Year, Month, and Day tokens:
        • Most common Hour, minute, second, millisecond, and offset tokens:
    • Facet JSON Structure
      • Structure
      • Logical operators
      • Data Source
      • Constraints
      • Extra Attributes
        • entity v.s. scalar facet
        • markdown_name
        • comment
        • hide_null_choice/hide_not_null_choice
        • bar_plot
        • order
      • hide_num_occurrences
      • Examples
        • Example 1
        • Example 2
        • More Examples
    • Facet Examples
      • Entity vs. Scalar
      • Choice Vs. Range
      • Facet Title
      • Open Vs. Close
      • Change order of scalar values
    • Pseudo-Columns Logic And Heuristics
      • Table of contents
      • Where To Use
      • Syntax
        • Simple Syntax
        • General Syntax
        • Source Definition Attributes
        • Display Attributes
      • Logic And Heuristics
        • Displayname
        • Value
        • Sort
      • Virtual Column
      • Examples
        • Visible Column List
        • Visible ForeignKey List
        • Specific Pseudo Columns
    • Pseudo-Column Display
      • Accessing Pseudo-Column Values Using By Utilizing Source Definitions
      • Accessing Current Pseudo-Column Data
        • $self Data Structure
        • Examples
    • Pseudo Column in Templating Environments
      • 1. Defining Sources
      • 2. Defining Pseudo-Column Display
        • Accessing Columns and Fkeys
        • Accessing Sources (Wait For)
      • Examples
    • Table Alternatives
      • Definition and assumptions
      • Related Entities
      • Understanding Recordset
      • ERMrestJS Restrictions
      • Faceting
        • 1. Alternative Table For Main Table
        • 2. Alternative Table in Facet List
    • Export Annotation
      • Structure
      • How ERMrestJS Interperts It
  • Web interface (Chaise)
    • Configuration File: chaise-config.js
    • Record Page
      • Related Entities
        • Add Related Entity
    • Custom CSS Styling Tips
      • Changing chaise styles
        • Character replacement
        • General HTML structure:
        • Examples:
        • More specific examples (using RBK …/#2/Gene_Expression:Specimen):
      • Custom Markdown Styles
      • Bootstrap styles
    • Useful Browser Console Commands
      • Browser Console
      • Useful Functions
      • Global Variables
    • Use navbar app in external HTML pages
      • Table of Contents
      • How to add navbar app
        • 1. Install Chaise
        • 2. Include Chaise dependencies
        • 3. Use navbar
      • Sample HTML page
      • 4. Configuring Navbar
      • 5. How to customize navbar UI
      • Notes
    • Use login app in external HTML pages
      • Table of Contents
      • How to add login app
        • 1. Install Chaise
        • 2. Include Chaise dependencies
        • 3. Use login
      • Sample HTML page
      • Notes
    • Logging
      • Table of Contents
      • Overview
      • Attributes
      • Action definition
      • List of requests
        • Server vs. Client
        • Stack structure
      • Facet compressed syntax
      • Error log
      • Truncation
      • Analysis
        • Asset Download & CSV Default Export
        • Finding the displayed recordset request
        • Find number of clicks on the “view” button in recordset app
      • Change Log
        • 04/02/21
        • 08/26/20
        • 02/12/20
    • Logging (Pre-February 2020)
      • Attributes
      • Action List
      • Client Button Action List
        • Button Action Attributes
      • Error Log
      • Truncation
      • Analysis
        • Asset Download & CSV Default Export
      • PCID list
      • Change Log
        • 10/??/19
        • 07/16/19
        • 07/12/19
        • 07/10/19
        • 05/06/19
        • 04/18/19
        • 03/08/19
  • Auxiliary web services for DERIVA (deriva-web)
    • deriva-web Configuration Guide
      • deriva_config.json
      • wsgi_deriva.conf
    • REST API reference: deriva/export
      • Exporting Files
        • Export file(s)
        • Retrieve exported file(s)
        • Retrieve log file for exported file(s)
      • Exporting Bags
        • Export bag
        • Retrieve an exported bag
        • Retrieve log file for exported bag
    • Chaise Integration
      • Export
        • root (object)
        • template (object)
        • output (object)
        • Output details
        • source (object)
        • destination (object)
        • Supported output formats
        • Output format details
DERIVA Docs
  • Docs »
  • Catalog Management Command-line Clients and API (deriva-catalog-manage)
  • View page source

Catalog Management Command-line Clients and API (deriva-catalog-manage)¶

Overview

  • TBD

Command Line Interfaces

  • TBD
  • TBD
  • TBD

API Reference

  • deriva.utils.catalog package
    • Subpackages
      • deriva.utils.catalog.components package
        • Submodules
        • deriva.utils.catalog.components.configure_catalog module
        • deriva.utils.catalog.components.deriva_model module
      • deriva.utils.catalog.manage package
        • Submodules
        • deriva.utils.catalog.manage.config module
        • deriva.utils.catalog.manage.deriva_csv module
        • deriva.utils.catalog.manage.dump_catalog module
        • deriva.utils.catalog.manage.graph_catalog module
        • deriva.utils.catalog.manage.update_catalog module
        • deriva.utils.catalog.manage.utils module
Next Previous

© Copyright 2019, USC Information Sciences Institute, Informatics Systems Research Division

Built with Sphinx using a theme provided by Read the Docs.