Skip to main content

Model-Prime fields

A defined set of fields, referred to as Model-Prime fields, support special functionality within the Model-Prime platform.

The following Model-Prime fields are created for your organization during onboarding and are populated by the metadata ingestion process:

  • Robolog ID - The unique identifier for a set of data recorded by a robot as defined by the Model-Prime platform.
  • Log Name - The file path of the robolog stored in your ingest store.
  • Ingested At - The time of ingest of the robolog into the Model-Prime platform.
  • Trace ID - The AWS Trace ID used to track ingestion of a robolog.
  • Robolog URI - The URI of the robolog storage resource.
  • Robolog Type - The type indicator for a robolog, which can be one of the following: ROS1Bag, ROS2Bag, ROS2BagMCAP, JSON, GENERATED, or UNKNOWN.

Other Model-Prime fields can be associated with an organization's custom fields when they are created. By creating this association, the Model-Prime platform is able to support special functionality using your organization's fields.

In addition to the Model-Prime fields associated during onboarding, the following additional Model-Prime fields are available:

  • Robot ID - A unique identifier for a robot or hardware instance.
  • Storage ID - The storage identifier describing the robolog write configuration.
  • Robolog Format Version - The format version for a set of data recorded by a robot.
  • Software Version - The software version running on the robot recording the data.
  • Hardware Version - The hardware version of the robot recording the data.
  • Location ID - A location identifier where the robot recorded the data.
  • Compression Format - The compression format used when recording data.
  • Start Time - The start time of the log or channel in UTC.
  • Duration - The duration of the log in numeric format.
  • Message Count - The number of messages in the robolog.

Organization fields

Your organization can create any number of custom metadata fields. Fields can have the following types:

  • BOOLEAN
  • DECIMAL
  • INTEGER
  • ONTOLOGY
  • TEXT
  • TIMESTAMP
  • DURATION

Field definition​

Fields parsed during ingest require a definition attribute, a jq filter used to find the field value. This definition is valid for parsing attributes from both JSON and YAML files.

ROS 1 Example​

The field definitions are used to parse the .yaml output of rosbag info -y.

  • Start Time: .start
  • Message Type: .types[].type
  • Channel Name: .topics[].topic

ROS 2 Examples​

The field definitions are used to parse metadata.yaml.

  • Start Time: .rosbag2_bagfile_information.starting_time.nanoseconds_since_epoch
  • Message Count: .rosbag2_bagfile_information.message_count
  • Channel Name: .rosbag2_bagfile_information.topics_with_message_count[].topic_metadata.name

Input format​

Some fields require an input_format specifier to inform the Model-Prime platform of the expected format and/or unit of measurement for that field during ingest. For instance, logs may have timestamp data formatted in Unix Epoch time.

Example ROS 1 timestamp:

start: 1687464426.789989

Example ROS 2 timestamp:

starting_time:
    nanoseconds_since_epoch: 1636583623701239167

Supported specifiers are listed below:

Type TIMESTAMP:

  • RFC3339 (example: 2023-10-12T19:28:19.123456Z)
  • EPOCH_SECONDS (used by ROS 1, example: 1697138899.123456)
  • EPOCH_NANOSECONDS (used by ROS 2, example: 1697138899123456000)

Type DURATION:

  • SECONDS (example: 123.456789)
  • NANOSECONDS (example: 123456789000)

Note:

  • For type TIMESTAMP, a value in RFC3339 format is unambiguous and therefore always accepted even if the input_format is not RFC3339.
  • For type TIMESTAMP, RFC3339 is considered the default format if no format specifier is indicated. In this case, only a value in
  • RFC3339 format is accepted.
  • For type DURATION, there is no default format and an input_format specifier is required.
  • For all other data types, no input_format is required.

Value conversion​

When a value is converted according to its input_format specifier, the Model-Prime platform persists the original raw value alongside the converted value. You can access both values from the Model-Prime API using the GET /robologs/{id}/values endpoint.

value is the value which has been converted as follows, regardless of input_format:

  • RFC3339 for type TIMESTAMP
  • Nanoseconds (as an integer) for type DURATION.

raw_value is the datum in its original format. It will only be present if different from value.

These properties are also available in the Analytics Platform as the SQL columns field_value and raw_value, respectively.

Creating and updating metadata fields

You can create and update custom metadata fields for your organization in the Field Configuration page in the web UI.

Web App UI Field Config Page

Alternatively, fields can be created using the POST /fields endpoint and can be updated using the PUT /fields/{id} endpoint.

More details on using the API are included in the API documentation via the above links.