Tree To Composite Projector

Note. See Tree and Composite Projections, IO Projectors for projections overview.

Converting model stored in XML to a Composite graph is a multi-phase process. To better illustrate individual steps, we will use a simple simple component model Booking.

Xml To Tree reading

The model above will be stored in XMLs with the following content. We use plain strings to store references without additional (meta)annotations – such information is stored in the NS metamodel.

<application name="bookingApp">
  <components>
    <component name="bookingComp" version="1.0"/>
    <component name="utils" version="1.0"/>
  </components>
</application>

<component name="bookingComp">
  <version>1.0</version>
  <componentDependencies>
    <componentDependency name="booking:utils">
      <dependsOn name="utils" version="1.0"/>
    </componentDependency>
  </componentDependencies>

  <dataElements>
    <dataElement name="Airline">
      <packageName>org.example</packageName>
      <fields>
        <field name="name">
          <valueField>
             <!-- ValueFieldType reference -->
            <valueFieldType component="" name="String"/>
          </valueField>
        </field>
      </fields>
      <finders>
        <finder name="findByNameEq">
          <fieldOperatorPairs>
            <fieldOperatorPair name="Eq:name">
              <!-- Field reference -->
              <field name="name"/>
            </fieldOperatorPair>
        </finder>
      </finders>
    </dataElement>
    <dataElement name="Booking">
      <packageName>org.example</packageName>
      <fields>
        <field name="airline">
          <linkField>
            <!-- DataElement reference split into two tags -->
            <targetPackage>org.example</targetPackage>
            <targetClass>Airline</targetClass>
        </field>
      </fields>
    </dataElement>
  </dataElements>

  <taskElements>
    <taskElement name="BookingConfirm">
      <!-- Class reference -->
      <targetClass>org.example.BookingDetails</targetClass>
      <!-- DataElement reference -->
      <targetElement component="bookingComp" name="Booking"/>
    </taskElement>
  </taskElements>
</component>

Reading/Writing classes are expanded based on the NS metamodel. They convert the model between the XML and the Tree representation.

Note that XML tags that are nested in the XMLs will be typically nested in the Trees too – the nesting is based on aggregation information in the metamodel.

Reference Representation

References and links are stored in the Trees in several different forms.

class ComponentTree {
  List<DataElementTree> dataElements;
}

class FieldTree {
  ValueFieldTree valueFieldTree;
  LinkFieldTree linkFieldTree;
}

2. DataRef

Crosslinks and references are represented via DataRefs, or via Strings to be resolved later.

class TaskElementTree {
  DataRef targetElement;
}

class LinkFieldTree {
  private String mTargetClass;
  private String mTargetPackage;
}

If the element referenced by a DataRef uses a functional key, an appropriate DataRef subtype will be used at runtime.

As the model may not contain enough information and constraints to do a full conversion, some references are incomplete.

The following sections demonstrate the variants based on the bookingComp Component example.

2a. Unique DataRef (TaskElementTree BookingConfirm)

TaskElement’s targetElement (of type DataElement) uses a component::name functional key. The name and component information is retrieved from the XML (when available) Data<targetElement component="bookingComp" name="Booking"/>

This is the ideal case, as we have an unique reference to the target and therefore the mapping can be performed without custom code.

2b. Non-unique DataRef (FieldOperatorPairTree Eq:name)

FieldOperatorPair’s field (of type Field) uses a component::dataElement::name functional key. However, the XML contains only the field’s name (<field name="name"/>).

This makes the reference non-unique, as it can be a name field in any dataElement – there is no explicit constraint mandating that it is a field of the enclosing dataElement. Thus we need at least some custom code to extend the key or interate over the element, but we can expand helper methods for fields lookup.

3. Derived Reference (LinkFieldTree targetPackage+targetClass)

For legacy reasons, LinkField stores information about a target element in two separate String fields. As there is no additional explicit information, we need to do an explicit lookup from custom code.

Processing Phases

All phases are performed recursively based on the aggregation information: Application calls Components, Component calls DataElements, TaskElements, etc.

  1. Registration
  2. Mapping
  3. Option Conversion

Registration

Registration phase is responsible for building a global cache of all Composite classes, from which composites can be retrieved based on the element’s name or other lookup key. This global cache is passed between TreeToComposite classes via a single TreeToCompositeMappingContext instance.

Registration phase traverses the tree structure through <elementTreeToComposite::registerTree methods, creates an empty <element>Composite for the currently traversed <element>Tree and registers the instance into a context cache using the <element>Tree DataRef.

Preceding the traversal is also registration of prime-data, and stubbing a specific subset of elements – namely all element options types to ensure that we can use new option types without having to declare them in prime-data or elsewhere.

Default lookup key is the element’s DataRef, but you can register the same instance under different keys to simplify lookup later.

// DataElementTreeToComposite.java
DataElementTreeToComposite::registerTree(DataElementTree tree) {
  ...
  // anchor:custom-registration:start
  DataRef qualifiedNameRef = DataRef.withName(tree.getPackageName() + "." + tree.getName();
  mappingContext.getElementsMappingContext().addDataElement(qualifiedNameRef), composite);
  // anchor:custom-registration:end
  ...
}

Additionally during child traversal, the parent sets itself to the child via a DataRef.

Thus

becomes

This is necessary to ensure that functional key DataRefs are complete – TaskElementDataRef("::BookingConfirm") becomes TaskElementDataRef("bookingComp::BookingConfirm").

At the end of the registration phase, the mapping context will contain all composite instances of all data element types. Now we can safely retrieve these instances and modify them without worrying about duplicate instances for the same elements.

  • TreeToCompositeMappingContext
    • FoundationMappingContext
    • ElementsMappingContext
      • dataElementCache : Map<LookupKey, DataElementTree> | key | value | | — | — | | bookingComp::Airline | DataElementComposite{allAttributes = null}@1111 | | bookingComp::Booking | DataElementComposite{allAttributes = null} @5555 | | org.example.Booking | DataElementComposite{allAttributes = null} @5555 |
      • fieldCache : Map<LookupKey, FieldTree> | key | value | | — | — | | bookingComp::Airline::name | FieldComposite{allAttributes = null}@2222 | | bookingComp::Booking::airline | FieldComposite{allAttributes = null}@8888 |

Note. @[...] is for illustration and represents an unique Java instance hash, i.e. classes with same instance hash are the same instances.

Mapping

Mapping phase is responsible for refor copying ValueField values, resolving LinkField references, and extending models where necessary via *Completer classes (custom code).

ValueFields Mapping

ValueField values (name, package name, dates, …) are copied directly in <element>TreeToComposite::transformValues() without additional logic.

transformValues(ComponentTree tree, ComponentComposite composite) {
  composite.setName(tree.getName());
  composite.setVersion(tree.getVersion());
}

Note. ComponentComposite instance in the method above was retrieved from the context cache; we do not create new instances.

Regular LinkFields Mapping

For link fields, we may need to (1) process children in a specific order, and (2) replace default processing with a custom one. anchor:custom-default-mapping can be used to specify ordering and choice of default processing.

Non-child link fields are retrieved from the cache, as non-parent is not responsible for invoking TreeToComposite of the child.

dataElementComposite.setType(getDataElementTypeRef(dataElementTree.getType()));

Whether DataElementType has been transformed yet or not does not concern us, as we use the same instance from the cache.

Note also that link to parent is processed like any other field

dataElementComposite.setComponent(getComponentRef(dataElementTree.getComponent()));

Child link fields are processed in a similar way, however in addition to retrieving the instances from cache, the parent invokes TreeToComposite on the child.

// pseudocode
dataElementComposite.setFields(dataElementTree.getFields().map((fieldTree) -> {
  fieldComposite = getFieldRef(fieldTree);
  new FieldTreeToComposite(mappingContext).mapTree(fieldTree, fieldComposite);
  return fieldComposite;
}));

Custom Mapping

Last step of the mapping is anchor:custom-transform-composite where additional conversion or model extension can be added.

For example linkField’s targetElement’s mapping can be performed as follows (simplified):

// DataElementTreeToComposite.java
DataElementTreeToComposite::registerTree(DataElementTree tree) {
  ...
  // anchor:custom-registration:start
  DataRef qualifiedNameRef = DataRef.withName(tree.getPackageName() + "." + tree.getName());
  mappingContext.getElementsMappingContext().addDataElement(qualifiedNameRef), composite);
  // anchor:custom-registration:end
  ...
}

// LinkFieldTreeToComposite.java
LinkFieldTreeToComposite::mapTree(LinkFieldTree tree, LinkFieldComposite composite) {
  ...
  // anchor:custom-transform-composite:start
  DataRef qualifiedNameRef = DataRef.withName(tree.getPackageName() + "." + tree.getName());
  composite.setTargetElement(getDataElementRef(qualifiedNameRef));
  // anchor:custom-transform-composite:end
  ...
}

Note. In some cases it is necessary to use the pre-mapping anchor:custom-default-mapping. However, it is strongly discouraged for any other uses that preparing simple references for regular mapping, as it operates on limited values (child values are not mapped yet), and complicates reasoning about the conversion order.

At the end of the mapping phase, all instances have their values converted and all dataRefs, backlinks and cross links are resolved to the correct instances.

Option Conversion

Final phase is conversion of complex options. Each option type that requires a conversion has a <OptionName><ElementName>Converter class that should return a structured, data layer-like object (plain POJO without any logic).

For example, the value of dataOption hasDisplayName contains a list of Field names of the same DataElement.

<dataElement name="Location">
  <fields>
    <field name="city">...</field>
    <field name="country">...</field>
  </fields>
  <dataOptions>
    <dataOption name="Location:hasDisplayName">
      <dataOptionType name="hasDisplayName"/>
      <value>city_country</value>
    </dataOption>
  </dataOptions>
  ...
</dataElement>

Up until now, this information has not been explicitly captured anywhere, and the parsing and lookup was performed during expansion itself.

To minimize expansion errors, provide better feedback and improve testability, we encapsulate the transformation in a Converter. The returned object will contain all information relevant to the option.

public class HasDisplayNameOptionConverter implements OptionConverter<DataElementComposite, DataOptionComposite> {
  @Override
  public ElementOption<DataElementComposite> convert(DataOptionComposite dataOption, TreeToCompositeMappingContext notYetSupported) {
    DataElementComposite dataElement = dataOption.getDataElement();
    HasDisplayNameOptionValue optionValue = new HasDisplayNameOptionValue(new FieldListDataOptionConverter().findFields(dataOption));
    return ElementOption.some(dataElement, dataOption.getDataOptionType().getName(), optionValue);
  }

  public static class HasDisplayNameOptionValue {
    private final List<FieldComposite> fields;

    public HasDisplayNameOptionValue(List<FieldComposite> fields) {
      this.fields = fields;
    }

    public List<FieldComposite> getFields() {
      return fields;
    }
  }
}

In the expanders, we can now directly refer to the properties of the OptionValue using OGNL.

<mapping>
  <list name="fields" eval="dataElement.getOption('hasDisplayName').value.fields" param="eval">
    ...
  </list>
</mapping>

After a 4.0 release, conversion information should be gradually pushed into the metamodel/new model to allow expansion of the converter skeletons and OptionValue classes, and better UI, validation, etc. in our modeling tools.


References