Base Model Annotation

This document defines a set of annotations we suggest may be useful in combination with ERMrest. We define a set of annotation keys, any associated JSON annotation values, and their semantics. Communities may use these conventions to modify their interpretation of ERMrest catalog content.

These annotations do not affect the behavior of the ERMrest service itself but merely inform clients about intended use beyond that captured in the entity-relationship model. Further, as described in the REST API docs, the annotation system is openly extensible so communities MAY use other annotation keys not described here; in those cases, the community SHOULD publish similar documentation on their use and interpretation.

Notation and Usage

Each annotation key is defined in a section of this document and shown as a literal string. We prepend a date in each key name and promise not to modify the semantics of an existing annotation key, once published to GitHub. We may publish typographical or other small textual clarifications, but if we need to change the proposed semantics we will define a new key with a different date and/or key text. We will follow the date stamp conventions from RFC 4151 which allow for abbreviated ISO dates such as 2015, 2015-01, and 2015-01-01.

Example to Set Annotation

This example sets the 2015 Display annotation:

PUT /ermrest/catalog/1/schema/MainContent/annotation/tag%3Amisd.isi.edu%2C2015%3Adisplay HTTP/1.1
Host: www.example.com
Content-Type: application/json

{"name": "Main Content"}

TBD changes to propose for ERMrest:

  1. Allow non-escaped characters in the annotation key since it is the final field of the URL and does not have a parsing ambiguity?
  2. Allow an empty (0 byte) request body to represent the same thing as JSON null?

Annotations

Some annotations are supported on multiple types of model element, so here is a quick matrix to locate them.

Annotation Schema Table Column Key FKR Summary
2015 Display X X X X - Display options
2015 Vocabulary - X - - - Table as a vocabulary list
2016 Table Alternatives - X - _ _ Table abstracts another table
2016 Column Display - - X - - Column-specific display options
2017 Key Display - - - X - Key augmentation
2016 Foreign Key - - - - X Foreign key augmentation
2016 Generated X X X - - Generated model element
2016 Ignore X X X - - Ignore model element
2016 Immutable X X X - - Immutable model element
2016 Non Deletable X X - - - Non-deletable model element
2016 App Links X X - - - Intra-Chaise app links
2016 Table Display - X - - - Table-specific display options
2016 Visible Columns - X - - - Column visibility and presentation order
2016 Visible Foreign Keys - X - - - Foreign key visibility and presentation order
2016 Export X X - - - Describes export templates
2017 Asset - - X - - Describes assets
2018 Citation - X - - - Describes citation
2018 Required - X - - - Required model column
2018 Indexing Preferences - X X - - Specify database indexing preferences

For brevity, the annotation keys are listed above by their section name within this documentation. The actual key URI follows the form tag:misd.isi.edu, date : key where the key part is lower-cased with hyphens replacing whitespace. For example, the 2015 Display annotation key URI is actually tag:misd.isi.edu,2015:display.

Tag: 2015 Display

tag:misd.isi.edu,2015:display

This key is allowed on any number of schemas, tables, columns, and keys. This annotation indicates display options for the indicated element and its nested model elements.

Supported JSON payload patterns:

  • {"name": name}: The name to use in place of the model element’s original name.
  • {"markdown_name": markdown }: The markdown to use in place of the model element’s original name.
  • {"name_style": { "underline_space": uspace , "title_case": tcase , "markdown": render }}: Element name conversion instructions.
  • {"show_nulls": { ncontext : nshow ,}: How to display NULL data values.

Supported JSON uspace patterns:

  • true: Convert underline characters (_) into space characters in model element names.
  • false: Leave underline characters unmodified (this is also the default if the setting is completely absent).

Supported JSON tcase patterns:

  • true: Convert element names to “title case” meaning the first character of each word is capitalized and the rest are lower cased regardless of model element name casing. Word separators include white-space, hyphen, and underline characters.
  • false: Leave character casing unmodified (this is also the default if the setting is completely absent).

Supported JSON render patterns:

  • true: Interpret the model element’s actual name as a Markdown string. This MAY include rendering visually in applications with such capability.
  • false: Present the model element’s actual name verbatim (this is also the default if the setting is completely absent).

Supported JSON nshow patterns:

  • true (or ""): Show NULL values as an empty field.
  • " marker " (a quoted string literal): For any string literal marker, display the marker text value in place of NULLs.
  • false: Completely eliminate the field if feasible in the presentation.

See Context Names section for the list of supported JSON ncontext patterns.

Tag: 2015 Display Settings Hierarchy

  • The "name" and "markdown_name" setting applies only to the model element which is annotated. They bypass the name_style controls which only apply to actual model names.
    • The "markdown_name" setting takes precedence if both are specified.
  • The "name_style" setting applies to the annotated model element and is also the default for any nested element.
  • The "show_nulls" settings applies to the annotated model element and is also the default for any nested element.
    • The annotation is allowed on schemas in order to set the default for all tables in the schema.
    • Each ncontext : nshow instruction overrides the inherited instruction for the same ncontext while still deferring to the inherited annotation for any unspecified ncontext. The "*" wildcard ncontext allows masking of any inherited instruction.
    • A global default is assumed: {"show_nulls": { "detailed": false, "*": true}

This annotation provides an override guidance for Chaise applications using a hierarchical scoping mode:

  1. Column-level name
  2. Column-level name_style.
  3. Table-level name_style.
  4. Schema-level name_style.

Note:

  • An explicit setting of null will turn off inheritence and restore default behavior for that modele element and any of its nested elements.
  • The name_style has to be derived separately for each field e.g. one can set underline_space=true at the schema-level and doesn’t have to set this again.

Tag: 2015 Vocabulary

tag:misd.isi.edu,2015:vocabulary

This key is allowed on any number of tables in the model, where the table contains at least one key comprised of a single textual column. A vocabulary table is one where each row represents a term or concept in a controlled vocabulary.

Supported JSON payload patterns:

  • null or {}: Default heuristics apply.
  • {"uri": uri}: The uri indicates the global identifier of the controlled vocabulary. The uri MAY be a resolvable URL.
  • {"term": column}: The named column stores the preferred textual representation of the term. The referenced column MUST comprise a single-column key for the table.
  • {"id": column}: The named column stores the preferred compact identifier for the term, which MAY be textual or numeric. The referenced column MUST comprise a single-column key for the table.
  • {"internal": [column, …] …}: The one or more named columns store internal identifiers for the term, used for efficient normalized storage in the database but not meaningful to typical users. The referenced columns MUST each comprise a single-column key for the table.
  • {"description": column}: The named column stores a longer textual representation of the term or concept. The referenced column SHOULD comprise a single-column key for the table.

Heuristics

  1. In the absence of an internal assertion, assume all keys are potentially meaningful to users.
  2. In the absence of a term assertion
  • Try to find a single-column key named term
  • Try to find a single-column key named name
  • If no term column is found table SHOULD NOT be interpreted as a vocabulary.
  1. In the absence of an id assertion
  • Try to find a column named id
  • Try to find an unambiguous single-column numeric key
  • If no id column is found, use the term column as the preferred compact identifier.
  1. In the absence of a description assertion
  • Try to find a column named description
  • If no description column is found, proceed as if there is no description or use some other detailed or composite view of the table rows as a long-form presentation.

In the preceding, an “unambiguous” key means that there is only one key matching the specified type and column count.

The preferred compact identifier is more often used in dense table representations, technical search, portable data interchange, or expert user scenarios, while the preferred textual representation is often used in prose, long-form presentations, tool tips, or other scenarios where a user may need more natural language understanding of the concept.

Tag: 2016 Ignore

tag:isrd.isi.edu,2016:ignore

This key is allowed on any number of Schema, Table, or Column model elements. The only part of chaise that is using this annotation is search application. It does not have any effects on other applications (i.e., record, record-edit, and recordset).

This key was previously specified for these model elements but such use is deprecated:

This annotation indicates that the annotated model element should be ignored in typical model-driven user interfaces, with the presentation behaving as if the model element were not present. The JSON payload contextualizes the user interface mode or modes which should ignore the model element.

Supported JSON payload patterns:

  • null or true: Ignore in any presentation context. null is equivalent to tag:misd.isi.edu,2015:hidden for backward-compatibility.
  • [] or false: Do not ignore in any presentation context.
  • [ context ,]: Ignore only in specific listed contexts, otherwise including the model element as per default heuristics. See Context Names section for the list of supported context names.

This annotation provides an override guidance for Chaise applications using a hierarchical scoping mode:

  1. Hard-coded default behavior in Chaise codebase.
  2. Server-level configuration in chaise-config.js on web server overrides hard-coded default.
  3. Schema-level annotation overrides server-level or codebase behaviors.
  4. Table-level annotation overrides schema-level, server-level, or codebase behaviors.
  5. Annotations on the column or foreign key reference levels override table-level, schema-level, server-level, or codebase behaviors.

Tag: 2016 Immutable

tag:isrd.isi.edu,2016:immutable

This key indicates that the values for a given model element may not be mutated (changed) once set. This key is allowed on any number of columns, tables, and schemas. There is no content for this key.

Tag: 2016 Generated

tag:isrd.isi.edu,2016:generated

This key indicates that the values for a given model element will be generated by the system. This key is allowed on any number of columns, tables and schemas. There is no content for this key.

Tag: 2016 Non Deletable

tag:isrd.isi.edu,2016:non-deletable

This key indicates that the schema or table is non-deletable. This key is allowed on any number tables and schemas. There is no content for this key.

Tag: 2016 Visible Columns

tag:isrd.isi.edu,2016:visible-columns

This key indicates that the presentation order and visibility for columns in a table, overriding the defined table structure.

Supported JSON payload pattern:

  • {context : columnlist ,}: A separate columnlist can be specified for any number of context names.
  • {context1 : context2 ,}: Configure context1 to use the same columnlist configured for context2.
  • {"filter": { "and": [ facetlist ,]} } : Configure list of facets to be displayed.

For presentation contexts which are not listed in the annotation, or when the annotation is entirely absent, all available columns SHOULD be presented in their defined order unless the application has guidance from other sources.

See Context Names section for the list of supported context names.

Supported columnlist patterns:

  • [columnentry ,]: Present content corresponding to each columnentry, in the order specified in the list. Ignore listed columnentry values that do not correspond to content from the table. Do not present table columns that are not specified in the list.

Supported columnentry patterns:

  • columnname: A string literal columnname identifies a constituent column of the table. The value of the column SHOULD be presented, possibly with representation guided by other annotations or heuristics.
  • [ schemaname , constraintname ]: A two-element list of string literal schemaname and constraintname identifies a constituent foreign key of the table. The value of the external entity referenced by the foreign key SHOULD be presented, possibly with representation guided by other annotations or heuristics. If the foreign key is representing an inbound relationship with the current table, it SHOULD be presented in a tabular format since it can represent multiple rows of data.
  • { "source": sourceentry }: Defines a pseudo-column based on the given sourceentry. For detailed explanation and examples please refer to here. Other optional attributes that this JSON document can have are:
    • markdown_name: The markdown to use in place of the default heuristics for title of column.
    • comment: The tooltip to be used in place of the default heuristics for the column.
    • entity: If the sourceentry can be treated as entity (the source column is key of the table), setting this attribute to false will force the scalar mode.
    • aggregate: The aggregate function that should be used for getting an aggregated result. The available aggregate functions are min, max, cnt, cnt_d, array, and array_d.
      • array will return ALL the values including duplicates associated with the specified columns. For data types that are sortable (e.g integer, text), the values will be sorted alphabetically or numerically. Otherwise, it displays values in the order that it receives from ERMrest. There is no paging mechanism to limit what’s shown in the aggregate column, therefore please USE WITH CARE as it can incur performance overhead and ugly presentation.
      • array_d will return distinct values. It has the same performance overhead as array, so pleas USE WITH CARE.
      • Using array or array_d aggregate in entity mode will provide an array of row-names instead of just the value of the column.
    • array_display: If you have "aggregate": "array" or "aggregate": "array_d" in the pseudo-column definition, a comma-seperated value will be presented to the user. You can use array_display attribute to change that. The available options are,
      • olist for ordered bullet list.
      • ulist for unordered bullet list.
      • csv for comma-seperated values.
      • raw for space-seperated values.

Supported sourceentry pattern:

  • columnname: : A string literal. columnname identifies a constituent column of the table.

  • path: An array of foreign key path that ends with a columnname that will be projected. foreign key path is in the following format:

      "`{` _direction_ `:[` *schema name*`,` *constraint name* `]}` "
    

    Where direction is either inbound, or outbound.

Supported facetlist pattern:

  • [facetentry ,]: Present content corresponding to each facetentry, in the order specified in the list. Ignore invalid listed facetentry. Do not present other facets that are not specified in the list.

facetentry must be a JSON payload with the following attributes:

Required attributes:

  • source: Source of the filter. If it is not specified or is invalid the facetentry will be ignored. It has the same pattern as sourceentry defined above.

Constraint attributes (optional):

You can use these attributes to define default preselected facets (Combination of these attributes are not supported yet, you cannot have both choices and ranges specified on a facet).

  • choices: Discrete choice e.g. maps to a checklist or similar UX. Its value MUST be an array of values.
  • ranges: Half-open or closed intervals, e.g. maps to a slider or similar UX. Its value MUST be an array of JSON payload, with min and max attributes. The min and max values will translate into inclusive range filters. In order to force exclusive range, you can use min_exclusive: true, or max_exclusive: true.
  • not_null: Match any record that has a value other than null. Its value MUST be true. If you have this constraint defined in your annotation, other constraints will be ignored (other than "choice": [null]`. In this case both of the filters will be ignored).

Configuration attributes (optional):

  • markdown_name: The markdown to use in place of the default heuristics for facet title.
  • entity: If the facet can be treated as entity (the column that is being used for facet is key of the table), setting this attribute to false will force the facet to show scalar mode.
  • open: Setting this attribute to true, will force the facet to open by default.
  • bar_plot: This attribute is meant to be an object of properties that control the display of the histogram. Setting this attribute to false will force the histogram to not be shown in the facet in the facet panel. If unspecified, default is true (or show the histogram).
  • ux_mode: choices, ranges, or check_presence. If a multi-modal facet control UX is available, it will specify the default UX mode that should be used (If ux_mode is defined, the other type of constraint will not be displayed even if you have defined it in the annotation). In check_presence mode only two options will be available to the users, “not-null” and “null”.
  • hide_null_choice and hide_not_null_choice: By default, we are going to add null and not-null options in the choice picker. Setting any of these variables to true, will hide its respective option.

bar_plot attributes (optional):

  • n_bins: Used to define the number of bins the histogram uses to fetch and display data. If undefined, default is 30 bins.

The following is an example of visible-columns annotation payload for defining facets. You can find more examples in here.

"filter": {
    "and" : [
        {"source": "column", "ranges": [{"min": 1}, {"min":5, "max":10}] ,"markdown_name": "**col**"},
        {"source": [{"outbound": ["S", "FK2"]}, "id"], "choices": [1, 2]},
        {"source": [{"inbound": ["S", "FK1"]}, {"outbound": ["S", "FK2"]}, "term"], "entity": false}
    ]
}

Tag: 2017 Key Display

tag:isrd.isi.edu,2017:key-display

This key allows augmentation of a unique key constraint with additional presentation information.

Supported JSON payload patterns:

  • { context: option}: Apply each option to the presentation of referenced content for any number of context names.

Supported display option syntax:

  • "markdown_pattern": pattern: The visual presentation of the key SHOULD be computed by performing Pattern Expansion on pattern to obtain a markdown-formatted text value which MAY be rendered using a markdown-aware renderer.
  • "column_order": [ columnorder_key]: An alternative sort method to apply when a client wants to semantically sort by key values.
  • "column_order": false: Sorting by this key psuedo-column should not be offered.

Supported columnorder_key syntax:

  • { "column": columnname , "descending": true }: Sort according to the values in the columnname column opposite of the order of current sort. For instance if asked to sort the key in descending order, sorting will be based on the ascending values of columnname column.
  • { "column": columnname , "descending": false }: Sort according to the values in the columnname column.
  • { "column": columnname }: If omitted, the "descending" field defaults to false as per above.
  • columnname: A bare columnname is a short-hand for { "column": columnname }.

Key pseudo-column-naming heuristics (use first applicable rule):

  1. Use key name specified by 2015 Display if name attribute is specified.
  2. For simple keys, use effective name of sole constituent column considering 2015 Display and column name from model.
  3. Other application-specific defaults might be considered (non-normative examples):
  • Anonymous pseudo-column may be applicable in some presentations
  • A fixed name such as Key
  • The effective table name
  • A composite name formed by joining the effective names of each constituent column of a composite key

Key sorting heuristics (use first applicable rule):

  1. Use the key’s display column_order option, if present.
  2. Determine sort based on constituent column, only if key is non-composite.
  3. Otherwise, disable sort for psuedo-column.

The first applicable rule MAY cause sorting to be disabled. Consider that determination final and do not continue to search subsequent rules.

Tag: 2016 Foreign Key

tag:isrd.isi.edu,2016:foreign-key

This key allows augmentation of a foreign key reference constraint with additional presentation information.

Supported JSON payload patterns:

  • {"from_name": fname}: The fname string is a preferred name for the set of entities containing foreign key references described by this constraint.
  • {"to_name": tname}: The tname string is a preferred name for the set of entities containing keys described by this constraint.
  • {"display": { context: option}}: Apply each option to the presentation of referenced content for any number of context names.
  • {"domain_filter_pattern": pattern}: The pattern yields a filter via Pattern Expansion. The filter is a URL substring using the ERMrest filter language, which can be applied to the referenced table. The filter MUST NOT use any

Supported display option syntax:

  • "column_order": [ columnorder_key]: An alternative sort method to apply when a client wants to semantically sort by foreign key values.
  • "column_order": false: Sorting by this foreign key psuedo-column should not be offered.

Supported columnorder_key syntax:

  • { "column": columnname , "descending": true }: Sort according to the values in the columnname column opposite of the order of current sort.For instance if asked to sort the foreign key in descending order, sorting will be based on the ascending values of columnname column. columnname can be the name of any columns from the table that the foreign key is referring to.
  • { "column": columnname , "descending": false }: Sort according to the values in the columnname column.
  • { "column": columnname }: If omitted, the "descending" field defaults to false as per above.
  • columnname: A bare columnname is a short-hand for { "column": columnname }. columnname can be the name of any columns from the table that the foreign key is referring to.

Set-naming heuristics (use first applicable rule):

  1. A set of “related entities” make foreign key reference to a presentation context:
  • The fname is a preferred name for the related entity set.
  • The name of the table containing the related entities may be an appropriate name for the set, particularly if the table has no other relationship to the context.
  • The name of the table can be composed with other contextual information, e.g. “Tablename having columnname = value”.
  1. To name a set of “related entities” linked to a presentation context by an association table:
  • The tname of the foreign key from association table to related entities is a preferred name for the related entity set.
  • The name of the table containing the related entities may be an appropriate name for the set, particularly if the table has no other relationship to the context.

Foreign key sorting heuristics (use first applicable rule):

  1. Use the foreign key’s display column_order option, if present.
  2. Use the referenced table display row_order option, if present.
  3. Determine sort based on constituent column, only if foreign key is non-composite.
  4. Otherwise, disable sort for psuedo-column.

The first applicable rule MAY cause sorting to be disabled. Consider that determination final and do not continue to search subsequent rules.

Domain value presentation heuristics:

  1. If pattern expands to filter and forms a valid filter string, present filtered results as domain values.
    • With filter F, the effective domain query would be GET /ermrest/catalog/N/entity/S:T/F or equivalent.
    • The filter SHOULD be validated according to the syntax summary below.
    • If a server response suggests the filter is invalid, an application SHOULD retry as if the pattern is not present.
  2. If filter is not a valid filter string, proceed as if pattern is not present.
  3. If pattern is not present, present unfiltered results.

Supported filter language is the subset of ERMrest query path syntax allowed in a single path element:

  • Grouping: ( filter )
  • Disjunction: filter ; filter
  • Conjunction: filter & filter
  • Negation: ! filter
  • Unary predicates: column ::null::
  • Binary predicates: column op value
    • Equality: =
    • Inequality: ::gt::, ::lt::, ::geq::, ::leq::
    • Regular expressions: ::regexp::, ::ciregexp::

Notably, filters MUST NOT contain the path divider / nor any other reserved syntax not summarized above. All column names and value literals MUST be URL-escaped to protect any special characters. All column names MUST match columns in the referenced table and MUST NOT be qualified with table instance aliases.

Tag: 2016 Column Display

tag:isrd.isi.edu,2016:column-display

This key allows specification of column data presentation options at the column level of the model.

Supported JSON payload patterns:

  • {context : { option}}: Apply each option to the presentation of column values in the given context.
  • {context1 : context2}: Short-hand to allow context1 to use the same options configured for context2.

See Context Names section for the list of supported context names.

Supported option syntax:

  • "pre_format": format: The column value SHOULD be pre-formatted by evaluating the format string with the raw column value as its sole argument. Please refer to Pre Format Annotation document for detailed explanation of supported syntax.
  • "markdown_pattern": pattern: The visual presentation of the column SHOULD be computed by performing Pattern Expansion on pattern to obtain a markdown-formatted text value which MAY be rendered using a markdown-aware renderer.
  • "column_order": [ columnorder_key]: An alternative sort method to apply when a client wants to semantically sort by this column.
  • "column_order": false: Sorting by this column should not be offered.

Supported columnorder_key syntax:

  • { "column": columnname , "descending": true }: Sort according to the values in the columnname column opposite of the order of current sort. For instance if asked to sort the column in descending order, sorting will be based on the ascending values of columnname column.
  • { "column": columnname , "descending": false }: Sort according to the values in the columnname column.
  • { "column": columnname }: If omitted, the "descending" field defaults to false as per above.
  • columnname: A bare columnname is a short-hand for { "column": columnname }.

All pre_format options for all columns in the table SHOULD be evaluated prior to any markdown_pattern, thus allowing raw data values to be adjusted by each column’s format option before they are substituted into any column’s pattern.

The column_order annotation SHOULD always provide a meaningful semantic sort for the presented column content. column_order MAY be present because the preferred semantic sort may differ from a lexicographic sort of the storage column, e.g. a secondary “rank” column might provide a better order for coded values in the annotated storage column.

Column sorting heuristics (use first applicable rule):

  1. Use the column’s display column_order option, if present.
  2. Sort by presented column value.

The first applicable rule MAY cause sorting to be disabled. Consider that determination final and do not continue to search subsequent rules.

Tag: 2016 Table Display

tag:isrd.isi.edu,2016:table-display

This key allows specification of table presentation options at the table or schema level of the model.

  • {context : { option}}: Apply each option to the presentation of table content in the given context.
  • {context1 : context2}: Short-hand to allow context1 to use the same options configured for context2.

See Context Names section for the list of supported context names.

Supported JSON option payload patterns:

  • "row_order": [ sortkey]: The list of one or more sortkey defines the preferred or default order to present rows from a table. The ordered list of sort keys starts with a primary sort and optionally continues with secondary, tertiary, etc. sort keys. The given sortkey s will be used as is (columnorder SHOULD not be applied recursivly to this).
  • "page_size": _number_: The default number of rows to be shown on a page.
  • "page_markdown_pattern": pagepattern: Render the page by composing a markdown representation only when page_markdown_pattern is non-null.
    • Expand pagepattern to obtain a markdown representation of whole page of dat via [Pattern Expansion](#pattern-expansion. In the pattern, you have access to a $page object that has the following attributes:
      • values: An array of values. You can access each column value using the {{{$page.values.<index>.<column>}}} where <index> is the index of array element that you want (starting with zero), and <column> is the column name ({{{$page.values.0.RID}}}).
      • parent: This variable is available when used for getting table content of related entities. Currently the row_markdown_pattern in compact context is used to provide a brief summary of table data. When used in this context, you can access the parent attributes under $page.parent. The attributes are:
        • values: the parent data {{{$page.parent.values.RID}}}.
        • table: the parent table name {{{$page.parent.table}}}.
        • schema: the parent schema name {{{$page.parent.schema}}}.
  • "row_markdown_pattern": rowpattern: Render the row by composing a markdown representation only when row_markdown_pattern is non-null.
    • Expand rowpattern to obtain a markdown representation of each row via Pattern Expansion. The pattern has access to column values after any processing implied by 2016 Column Display.
  • "separator_markdown": separator: Insert separator markdown text between each expanded rowpattern when presenting row sets. (Default new-line "\n".)
    • Ignore if "row_markdown_pattern" is not also configured.
  • "prefix_markdown": prefix: Insert prefix markdown before the first rowpattern expansion when presenting row sets. (Default empty string "".)
    • Ignore if "row_markdown_pattern" is not also configured.
  • "suffix_markdown": suffix: Insert suffix markdown after the last rowpattern expansion when presenting row sets. (Default empty string "".)
    • Ignore if "row_markdown_pattern" is not also configured.
  • "module": module: Activate module to present the entity set. The string literal module name SHOULD be one that Chaise associates with a table-presentation plug-in.
  • "module_attribute_path": pathsuffix: Configure the data source for activated module. Ignore if module is not configured or not understood.
    • If pathsuffix is omitted, use the ERMrest /entity/ API and a data path denoting the desired set of entities.
    • If pathsuffix is specified, use the ERMrest /attribute/ API and append pathsuffix to a data path denoting the desired set of entities and which binds S as the table alias for this entire entity set.
      • The provided pathsuffix MUST provide the appropriate projection-list to form a valid /attribute/ API URI.
      • The pathsuffix MAY join additional tables to the path and MAY project from these tables as well as the table bound to the S table alias.
      • The pathsuffix SHOULD reset the path context to $S if it has joined other tables.

It is not meaningful to use page_markdown_pattern, row_markdown_pattern, and module in for the same context. If they co-exist, the application will prefer module over page_markdown_pattern and page_markdown_pattern over row_markdown_pattern.

Supported JSON sortkey patterns:

  • { "column": columnname , "descending": true }: Sort according to the values in the columnname column in descending order. This is equivalent to the ERMrest sort specifier @sort( columnname ::desc:: ).
  • { "column": columnname , "descending": false }: Sort according to the values in the columnname column in ascending order. This is equivalent to the ERMrest sort specifier @sort( columnname ).
  • { "column": columnname }: If omitted, the "descending" field defaults to false as per above.
  • columnname: A bare columnname is a short-hand for { "column": columnname }.

Table Display Settings Hierarchy

The table display settings apply only to tables, but MAY be annotated at the schema level to set a schema-wide default, if appropriate in a particular model. Any table-level specification of these settings will override the behavior for that table. These settings on other model elements are meaningless and ignored.

For hierarchically inheritable settings, an explicit setting of null will turn off inheritance and restore default behavior for that model element and any of its nested elements.

Tag: 2016 Visible Foreign Keys

tag:isrd.isi.edu,2016:visible-foreign-keys

This key indicates that the presentation order and visibility for foreign keys referencing a table, useful when presenting “related entities”.

Supported JSON payload pattern:

  • {context : fkeylist ,}: A separate fkeylist can be specified for any number of context names.
  • {context1 : context2}: Short-hand to allow context1 to use the same fkeylist configured for context2.

For presentation contexts which are not listed in the annotation, or when the annotation is entirely absent, all available foreign keys SHOULD be presented unless the application has guidance from other sources. See Context Names section for the list of supported context names.

Supported fkeylist patterns:

  • [ [ schema name, constraint name ] ,]: Present foreign keys with matching schema name and constraint name, in the order specified in the list. Ignore constraint names that do not correspond to foreign keys in the catalog. Do not present foreign keys that are not mentioned in the list. These 2-element lists use the same format as each element in the names property of foreign keys in the JSON model introspection output of ERMrest. The foreign keys MUST represent inbound relationships to the current table.
  • { "source": sourceentry }: Defines a pseudo-column based on the given sourceentry. For detailed explanation and examples please refer to here. Other optional attributes that this JSON document can have are:
    • markdown_name: The markdown to use in place of the default heuristics for title of column.

Supported sourceentry pattern in here:

  • path: An array of foreign key path that ends with a columnname that will be projected. foreign key path is in the following format:

      "`{` _direction_ `:[` *schema name*`,` *constraint name* `]}` "
    

    Where direction is either inbound, or outbound.

Tag: 2016 Table Alternatives

tag:isrd.isi.edu,2016:table-alternatives

This key indicates that the annotated table (e.g. the base storage table) has abstracted views/tables that should be used as alternataive tables in different contexts. This means that they both represent the same entity set but the alternative one has modified the representation of each entity in some way.

Supported JSON payload patterns:

  • {context : [ sname, tname] ,}: The table identified by sname:tname is an alternative table to be used instead of the annoted table in the specified context.

A alternative table or view which abstracts another table SHOULD have a non-null (psuedo) primary key which is also a foreign key to the base storage table. The base storage table is the one bearing this annotation. Otherwise, a consuming application would not know how to navigate from one abstracted representation of an entity to another representation from the base storage tables.

See Context Names section for the list of supported context names. It is assumed that any application context that is performing mutation (record creation, deletion, or editing) MUST use a base entity storage table that is not an abstraction over another table. However, the use of the detailed or compact context MAY offer an abstraction that augments the presentation of an existing record. An application offering mutation options while displaying an existing entity record might then present the data from the detailed or compact abstraction but only offer editing or data-entry controls on the fields available from the base storage table.

Tag: 2016 Export

tag:isrd.isi.edu,2016:export

This key can be used to define export templates that will be used for ioboxd service integration with the client tools. For more information about the annotation payload please visit the iobodx integration document.

Supported JSON payload patterns:

  • { "templates": [template] }: An array of template objects to export.

Supported template patterns:

  • {"displayname:" displayname}: The display name that will be used to populate the Chaise export drop-down for this template.
  • {"type:" type} One of two keywords; “FILE” or “BAG”, used to determine the container format for results.
  • {"outputs": [output]}: An array of output objects. If the template type is “BAG” you MAY leave this attribute and not define it. In this case, the default outputs that the client generates will be used.

Supported output patterns:

  • {"source:" sourceentry}: An object that contains parameters used to generate source data by querying ERMrest.
  • {"destination": destinationentry}: An object that contains parameters used to render the results of the source query into a specified destination format.

Supported sourceentry patterns:

  • {"api:" api}: The type of ERMrest query projection to perform. Valid values are entity, attribute, and attributegroup.
  • {"path": path}: An optional ERMrest path predicate. The string MUST be escaped according to RFC 3986 if it contains user-generated identifiers that use the reserved character set. See the ERMRest URL conventions for additional information.

Supported destinationentry patterns:

  • {"name": name}: The base name to use for the output file.
  • {"type": type}: A type keyword that determines the output format. Supported values are dependent on the template.type selected. For the FILE type, the values csv, json, are currently supported. For the BAG type, the values csv, json, fetch and download are currently supported.
  • {"params": params}: An optional object containing destination format-specific parameters. Some destination formats (particularly those that require some kind of post-processing or data transformation), may require additional parameters to be specified.

Export Annotation Hierarchy

This annotation only applies to table but MAY be annotated at the schema level to set a schema-wide default. If the annotation is missing on the table, we will get the export definition from the schema.

Tag: 2017 Asset

tag:isrd.isi.edu,2017:asset

This key indicates that the annotated column stores asset locations. An asset is a generic, fixed-length octet-stream of data, i.e. a “file” or “object” which can be stored, retrieved, and interpreted by consumers.

An asset location is a globally unique and resolvable string, used to reference and retrieve the identified asset either directly or indirectly through a resolution service. For example, an HTTP URL is both globally unique and resolvable. In the case of a relative URL, the client should resolve the URL within the context from which it was retrieved. Persistent identifier schemes MAY be used such as MINID, DOI, ARK, or PURL. It is up to client tooling to recognize and resolve identifiers in such schemes.

A new asset location may be specified via a pattern to induce a prospective asset location based on known metadata values, i.e. to normalize where to upload and store a new asset in a data-submission process. Only meaningful where clients can request creation of new assets with a desired location.

Supported JSON payload patterns:

  • {"url_pattern": pattern}: A desired upload location can be derived by Pattern Expansion on pattern. This attribute is required for browser upload and if it is not specified the client will not provide the browser upload feature. See implementation notes below.
  • {"browser_upload": False}: If url_pattern is availale and valid browser upload feature will be enabled. If you want to force disabling this feature set it to False.
  • {"filename_column": column}: The column stores the filename of the asset.
  • {"byte_count_column": column}: The column stores the file size in bytes of the asset. It SHOULD be an integer typed column.
  • {"md5": column | True}: If column, then the column stores the checksum generated by the ‘md5’ cryptographic hash function. It MUST be ASCII/UTF-8 hexadecimal encoded. If True, then the client SHOULD generate a ‘md5’ checksum and communicate it to the asset storage service according to its protocol.
  • {"sha256": column | True}: If column, then the column stores the checksum generated by the ‘sha256’ cryptographic hash function. It MUST be ASCII/UTF-8 hexadecimal encoded. If True, then the client SHOULD generate a ‘sha256’ checksum and communicate it to the asset storage service according to its protocol. See implementation notes below.
  • {"filename_ext_filter": [ { filename extension [, filename extension ]* } ]}: This property specifies a set of filename extension filters for use by upload agents to indicate to the user the acceptable filename patterns (.jpg, .png, .pdf, …). For example, .jpg would indicate that only JPEG files should be selected by the user.

Default heuristics:

  • The 2017 Asset annotation explicitly indicates that the associated column is the asset location.
  • url_pattern MUST be specified for browser upload. If it is not specified or if it produces a null value, the browser upload will be disabled.
  • Column MUST be text typed. Otherwise the asset annotation will be ignored.
  • In addition to native columns, the following properties are also available under the annotated column object and can be referred in the pattern e.g. {{{_URI.md5_hex}}} where URI is the annotated column (notice the underscore before the column name).
    • md5_hex for hex
    • md5_base64 for base64
    • filename for filename
    • size for size in bytes
  • Nothing may be inferred without additional payload patterns present.

Protocol-specific metadata retrieval MAY be applied once an asset location is known. How to present or reconcile contradictions in metadata found in multiple sources is beyond the scope of this specification.

  • Some applications may treat ERMrest data as prefetched or cached metadata.
  • Some applications may treat ERMrest data as authoritative metadata registries.
  • Some location schemes may define authoritative metadata resolution procedures.

At present, the Chaise implementation of the asset annotation has the following limitations:

  1. ‘generated’ column(s) participating in the url_pattern are only supported in the entry/edit context and not in the entry/create context. This is because the generated column values are usually generated by the server during the record creation and will not be available to Chaise while the users are supplying information. If you wish to use ‘generated’ column(s) in the url_pattern, you will need to use the 2016 Visible Columns annotation and leave the asset column out of the list of visible columns for its entry/create context.
  2. sha256 is not presently supported.
  3. If url_pattern is not available or browser_upload is False Chaise will show a disabled form field for the asset column. It will still provide the download button in read-only contexts.

Tag: 2018 Citation

tag:isrd.isi.edu,2018:citation

This key indicates that the annotated table has a format for defining citations for the rows. A citation defines the given row in a way that it can be shared and referenced in other works. Each pattern in the citation annotation is consumed by the client and presented in a way defined by the client.

Supported JSON payload patterns:

  • {"journal_pattern": pattern}: A desired journal value can be derived by Pattern Expansion on pattern. This attribute is required for the citation feature and if it is not specified, the client will not provide the citation display feature. See implementation notes below.
  • {"author_pattern": pattern}: A desired author value can be derived by Pattern Expansion on pattern.
  • {"title_pattern": pattern}: A desired title value can be derived by Pattern Expansion on pattern.
  • {"year_pattern": pattern}: A desired year value can be derived by Pattern Expansion on pattern. This attribute is required for the citation feature and if it is not specified, the client will not provide the citation display feature. See implementation notes below.
  • {"url_pattern": pattern}: A desired url value can be derived by Pattern Expansion on pattern. This attribute is required for the citation feature and if it is not specified, the client will not provide the citation display feature. See implementation notes below.
  • {"id_pattern": pattern}: A desired id value can be derived by Pattern Expansion on pattern.

Default heuristics:

  • journal_pattern, year_pattern, and url_pattern MUST be specified for citation. If any of the 3 are not specified or if one of them produces a null value, citation will be disabled.
  • If any of the other values are not present or produce a null value, it is up to the client to decide how to display the citation.

At present, the Chaise implementation of the citation annotation has the following limitations:

  1. If journal_pattern, year_pattern, or url_pattern is not available, Chaise will not show a Citation list option in the Share dialog.
  2. Chaise will try to show the 3 non-required fields if their are present and their templates don’t produce a null value.

Tag: 2018 Required

tag:isrd.isi.edu,2018:required

This key indicates that the values for a given model element will be required by the system. This key is allowed on any number of columns. There is no content for this key.

Tag: 2018 Indexing Preferences

tag:isrd.isi.edu,2018:indexing-preferences

This key indicates that the annotated table or column should follow a different indexing strategy. At the time of writing, this is the only annotation recognized by ERMrest which affects service behavior (all others are opaque key-value storage only affecting clients).

Meaning on different model elements:

  • On tables: requests a table-wide indexing strategy
  • On columns: requests a column-specific indexing strategy (may override table-wide preferences)

Supported JSON payload patterns:

  • {"btree": preference}: Specifies a preference for PostgreSQL btree indexing.
  • {"trgm": preference}: Specifies a preference for PostgreSQL pg_trgm (text tri-gram) indexing.

Supported preference patterns:

  • true: An index is desired.
  • false: An index is not desired.
  • null (or field absent): The default is desired (currently all indexing is enabled by default).

If a column-level annotation sets a preference of null, this suppresses any table-wide preference for the same indexing type, requesting built-in service defaults for the column.

This annotation is a hint to ERMrest during table or column creation, when indexes are built. Therefore, administrators SHOULD supply the annotation within table or column creation requests. Manipulation of the annotation on existing tables or columns will not change the indexes which are already present (or absent) on those existing models. However, changes to the table annotation will affect any columns added later, unless their column-creation requests override the table-wide preferences.

Context Names

List of context names that are used in ERMrest:

  • "compact": Any compact, tabular presentation of data from multiple entities.
    • "compact/brief": A limited compact, tabular presentation of data from multiple entities to be shown under the detailed context. In this context, only a page of data will be shown with a link to the access the compact context for more detail.
    • "compact/select": A sub-context of compact that is used for selecting entities, e.g. when prompting the user for choosing a foreign key value.
  • "detailed": Any detailed read-only, entity-level presentation context.
  • "entry": Any data-entry presentation context, i.e. when prompting the user for input column values.
    • "entry/edit": A sub-context of entry that only applies to editing existing resources.
    • "entry/create": A sub-context of entry that only applies to creating new resources.
  • "filter": Any data-filtering control context, i.e. when prompting the user for column constraints or facets.
  • "row_name": Any abbreviated title-like presentation context.
    • "row_name/title": A sub-context of row_name that only applies to title of page.
    • "row_name/compact": A sub-context of row_name that only applies to compact, tabluar presentation of a row (When a foreignkey value is displayed in a tabular presentation).
    • "row_name/detailed": A sub-context of row_name that only applies to entity-level presentation of a row (When a foreignkey value is displayed in the entity-level page).
  • "*": A default to apply for any context not matched by a more specific context name.

If more than one context name in the annotation payload matches, the options should be combined in the following order (first occurrence wins):

  1. Prefer option set in matching contexts with exact matching context name.
  2. Prefer option set in matching contexts with longest matching prefix, e.g. an option for entry can match application context entry/edit or entry/create.
  3. Use default option set in context *.

The following matrix illustrates which context is meaningful in which annotation.

Annotation compact compact/brief compact/select detailed entry entry/edit entry/create filter row_name *
2015 Display X - X X X X X X - X
2016 Ignore X - X X X X X X - X
2016 Visible Columns X - X X X X X X - X
2016 Column Display X - X X X X X X - X
2016 Table Display X X X X - - - X X X
2016 Visible Foreign Keys X - - X X X X X - X
2016 Table Alternatives X - X X - - - X - X

Pattern Expansion

When deriving a field value from a pattern, the pattern MAY contain markers for substring replacements of the form {{column name}} or {{{ column name}}} where column name MUST reference a column in the table. Any particular column name MAY be referenced and expanded zero or more times in the same pattern. Each pattern is passed through a templating environment. By default, this templating environment is Mustache. A template_engine parameter can be defined alongside any pattern to define which templating engine to use. Currently you can choose between handlebars and mustache. For detailed explanation on template and markdown language please refer to Mustache Templating and Handlebars Templating documents.

As an example, a column may have a tag:isrd.isi.edu,2016:column-display annotation containing the following payload:

{
   "*" : {
       "markdown_pattern": "[{{{title}}}](https://dev.isrd.isi.edu/chaise/search?name={{{_name}}})",
       "template_engine": "handlebars"
   }
}

A web user agent that consumes this annotation and the related table data would likely display the following as the value of the column:

<p>
    <img src="https://dev.isrd.isi.edu/chaise/search?name=col%20name" alt="Title of Image">
</p>