# Post Data Retrieval

# Data Processing Best Practices

There are many different environments (local storage, data warehouses, data lakes, etc.), technologies, and third party data vendors that customers can leverage after they have retrieved their data from the CSOD system. It would be foolhardy to provide rules that would adequately apply to all environments/scenarios. Nevertheless, here are some general guidelines and considerations as you process your data.

  • Store your unprocessed data in one location and copy it to another location before processing it. Or if you are going to transform the data, store the transformed data in an alternate location without modifying the original data. This is particularly true when initially building and testing your data solution. Many times there will be an unexpected business logic error used during post-processing and the error will not be discovered until after the fact. If you retain the originally received data for a certain period of time, you can 1) verify the data in question was received in the first place, and 2) retry and troubleshoot your process. CSOD cannot vouch for the reliability of customer data after it has been delivered and processed by another entity. The alternative is to pull another full download of the relevant core objects, which defeats many of the efficiencies that the DEAPI provides.

  • Remember data is likely ordered/optimized for delivery, not necessarily for merge/upsert into the receiving system. Received data may be ordered by _last_touched_dt_utc, particularly if you are taking advantage of change tracking capabilities. Many merge/upsert processes are most effective when the two data sets to be combined use the same index. Therefore, you may wish to reorder or index your data before merge. See the next section to determine the unique keys for each core object.

Additional guidance can/will be added as they are forwarded by DEAPI customers.

# Unique Keys and IDs

Unlike RAPI views which may or may not have a unique key associated with them, all DEAPI core objects have a unique key / constraint that can be applied to the receiving data store (also useful for efficient merge/upsert operations). There are two ways to retrieve these keys:

  • Via object metadata. The Key collection will identify those fields that you can use to create your primary key / unique constraint. The Indexes collection can also identify alternate indexes, although not unique, that you may wish to add to your objects.

CSOD Reporting Environments

  • DEAPI documentation. The Key field(s) are listed above each table that describes the fields of each core object.

CSOD Reporting Environments

# Relationship Diagrams

Once individual objects are loaded into the customer's local data store, they can be joined together for further querying and analysis. The DEAPI's $cs_diagram endpoint can generate diagrams that show the relationship between objects in an instant, without the need to navigate lengthy text documents:

https://{environment}.csod.com/api/x/objects/$cs_diagram?mode=N&obj= X,Y,Z

Two Inputs are required:

  • Mode -- Choose from Basic, Detail, and Relations.
  • Obj -- Add the name of the core objects for which you want to see relationships, separated by a comma. Up to 32 objects may be requested per call

Each relationship is represented with an arrow which shows navigation from one object to another object. The object where the arrow originates is considered (To) and the object where the arrow points towards is considered (From) object. If the arrow points to itself, it represents that there is a self-referencing navigation relationship. Each relationship also shows the Multiplicity that relationship represents between the two entities. Multiplicity defines how object participate in a relationship and it is the number of instances of one object related to one instance of the other object.

  • 0..* - An object instance can have zero or many related instances in another object. for example, A user can have zero or more transcripts.

  • 1 - An object instance has exactly 1 related instance in another object. For example, a user can have only one manager.

  • 0..1 - An object instance can have zero or one related instance in another object. For example, a user can have zero or one indirect manager.

Relationships are ultimately derived from DEAPI metadata. Therefore, to generate a diagram the calling application should have access to the $metadata endpoint (i.e., the 'obj_metadata:read' scope when configuring application permissions). If you ever forget the relationship diagram URLs, you can inspect each object's metadata to retrieve the ObjectBasicDiagramUrl, the ObjectDetailDiagramUrl and the ObjectRelationsDiagramUrl.

# Basic Mode

Example: https://{environment}.csod.com/api/x/objects/$cs_diagram?mode=basic&obj=users_core,users_cf_core,

This mode simply illustrates if objects are related. The relationship is illustrated by the presence of a connecting line. Looping lines mean that there is an internal self-reference, meaning that there are fields within the object that are related.

CSOD Reporting Environments

# Detail Mode

Example: https://{environment}.csod.com/api/x/objects/$cs_diagram?mode=detail&obj=users_core,users_cf_core,

This mode shows every relationship between the selected objects, along with a more human- readable relationship name and the field names that form the relationship.

CSOD Reporting Environments

# Relations Mode

Example: https://{environment}.csod.com /api/x/objects/$cs_diagram?mode=relations&obj=users_core,users_cf_core,

This mode simply illustrates if objects are related. The relationship is illustrated by the presence of a connecting line. Looping lines mean that there is an internal self-reference, meaning that there are fields within the object that are related.

CSOD Reporting Environments

# Custom Fields

# Encoded Field ID

Whenever a custom field is created in the portal, it is assigned a database-generated encoded ID. Please note this encoded ID may not be the same between environments. The ID will never change within its respective environment except for copy downs.

The general rule for locating custom fields is by looking for '_cf' appended to the name of the object. For example, "users_cf_core" will contain all the user custom fields. Please note that custom fields are categorized by type/entity and will not be found in a single object. To see which objects contain user defined custom fields, please refer to the Data Exporter API reference documentation. If an object contains custom fields, it will be categorized as "Yes" under the user defined fields column.

CSOD Reporting Environments

The custom field Objects include the encoded field ID and the value selected for that field. In order get the mapping between the encoded field ID and the name of the custom field as seen in your CSOD portal, please refer to the metadata for that object. In the metadata, the portal's common name of the field will be listed under ExtendedMetadata.FieldHeading.

CSOD Reporting Environments

For dropdown, radio button, multiple checkbox, and branched hierarchy custom fields, the object may provide you with an ID in your return. To map the ID with the description of the record, you must map the ID to one of the local objects.