Complex record streams

In Data Management, most connectors between tools will transport a single record type. When you double-click any data connector, you will see its record schema displayed in the schema window. However, the complex record file formats may contain hundreds of record types in a single schema, especially in the standard formats like X12 and EDIFACT. Since it would be impractical to have that many separate data connectors on a tool, Data Management implements a complex record stream to transport multiple record types between tools.

A complex record stream contains meta-records with six fields:

Field	Meaning
__Type	The type of the record.
__ID	A unique record number.
__ParentID	The __ID value of the parent record in the nesting hierarchy, or zero for root records.
__Sequence	The relative order of this record with respect to all of child records of the same parent.
__Name	Used for special cases child records are named instead of simply ordered in a list. This is only used in XML.
__Variant	A binary field containing the raw data of the actual record. Never alter this value.

You normally do not manipulate these record streams directly. Instead, use a Complex Splitter tool to generate simple record types to work with. There are a few cases where you might operate on the complex record stream directly such as:

• Using a Filter tool to remove some record types based on the __Type field value.

• Using a Calculate tool to alter __Sequence values to change the relative order of children by __Type.