Creating external expanders with a separate metamodel

Goal

In this guide we will set up a project that will let us create expanders which run on a custom model instead of, or, in addition to, the standard model of data-elements, task-elements etc.

We will do this by using the expanders of the metamodel-stack to expand a metamodel project. This project is designed to build a set of expansion-resources in a similar way to how we create applications with the default stack.

The end result will be 2 jars:

  • <application>-core: contains the runtime classes for the external metamodel
  • <application>-expanders: contains the actual expanders

By separating the two, we make it possible for the metamodel to be reused with different expanders. It will also, hopefully, encourage developers to separate the logic of reading and enriching the model from the actual logic of the epxanders.

How it works

The generated project will contain classes to run the expansion on instances of our custom elements. Simply put, we need classes to perform the following 4 steps:

  1. An ExpansionStep to hook into the regular expansion on the level of Application, Component or a more specific element.
  2. In this ExpansionStep, a ModelReader will need to read all xml files representing the instances of a custom expandable element. These are then read into Tree objects
  3. Then, all Tree objects should be transformed into Composite objects using the TreeToComposite objects. Composite objects have direct pointers instead of DataRefs. E.g. DataElementComposite::getComponent returns ComponentComposite instead of a DataRef.
  4. Finally, an ExpansionRunner will need to execute all the applicable expanders for the custom element

Setup

Prerequisites

To get started you will need the following:

  • An NS model with data-elements describing your metamodel
  • The version of expanders-maven-plugin should be 2020.12.0 or higher (can be configured here)
  • The elements and expansionControl components imported into the Prime Radiant

If you don’t have access to the elements and expansionControl components, download the prime-metamodel jar and unzip it into your workspace. You can then import it into the Prime Radiant.

If you’re not using the Prime Radiant to expand, you can ignore this. Instead, add the elements and expansionControl components to your application. Adding prime-metamodel as an expansion-resource (see below) will then provide the expanders with the model at expand time.

Step 1: preparing the model

Add the following options to your metamodel component(s):

  • enableComposites: this will expand Composite classes for your metamodel and surrounding infrastructure
  • enableIoXml: this will expand classes for reading and parsing XML representations of your metamodel
  • meta.groupId(optional): if defined, this will set the groupId for each maven artifact for this component to the provided value

Also add a dependency on elements and expansionControl to your component.

Next, add these options to the data-elements you wish to expand:

  • functionalKey: define a number of fields that can be used to uniquely identify an instance (see more)
  • isExpandableElement: expands the infrastructure to create expanders for this element
  • hasParentElement: references the parent element (e.g. elements::Component), so that these elements will always expand in the context of its parent element (required)
  • generateExpansionHook (optional): this generate a *ExpansionStep that will hook into the expansion based on the defined parent (see option hasParentElement)

If some of the expandable elements contain child elements, the field option aggregation should be added to the linkFields of the parent with the value composite. Look at the DataElement-_Field_ relation to see an example. If these options are missing, the child elements will not be read from the model and will not appear in the Tree projection.

Finally, add these options to your application instance:

  • meta.component.exclude: exclude components that you don’t want expanded. Set the value to elements,expansionControl
  • meta.module.distribution: generates a distribution module, which packages all layers together and creates a single expansion-resource jar containing all the classes needed to expand with the metamodel
  • meta.module.expanders: generates an expanders module, to which you can add new expanders using the metamodel
  • meta.groupId(optional): if defined, this will set the groupId for each maven artifact for the application

Step 2: meta expansion

  1. Make sure your application contains the elements and expansionControl components
  2. Add the following expansion-resources:
    • net.democritus:base-components-model@2020.7.0
    • net.democritus:prime-metamodel@2020.9.1
    • net.democritus:metamodel-expansion-stack@2.0.1
  3. Remove nsx-default-stack if present
  4. Export changes and expand

This should generate a new project with the following structure:

Component Layers

Each component has multiple layers:

  • composite-shared: contains composite and tree classes as well as a TreeToCompositeMappingContext class which is used for lookup
  • io-xml: contains XML readers and writers. XmlToTree can be customized to add additional parsing logic.
  • composite-logic: contains TreeToComposite and CompositeToTree classes. If you need some more complex enrichment of the composite objects, this can be added to the TreeToComposite classes.
  • expansion-control: contains the classes used to run the expanders for the custom element. It also contains the ModelReader class, which will read the XML files and transform the trees into composites.

In addition, there are some layers included for testing purposes:

  • onion-spec: contains Specs class, which creates specs to be used with the ModelSpecBuilder to create composite and tree objects in tests
  • io-xml-tests: a separate layer to test io-xml and composite-logic logic and customizations. These tests can use the Specs class from the onion-spec layer.

Application modules

On application level, we have 2 modules:

  • distribution: This will build a single jar, <application>-core-<version>.jar. This jar contains all non-test layers of each of the components. The jar will also contain an expansion-resource manifest. Thus, *-core can be added to your expansionSettings to include it during expansion. However, most of the time it will be added as a dependency in an expansion-resource with expanders so that it is automatically loaded by nsx-prime.
  • expanders: Here you can add new expanders, which will be built into a <application>-expanders-<version>.jar. It contains a dependency on *-core by default.

Step 3: building and using the expansion resources

First thing we need to do to make the project buildable is to define the versions of expanders and expanders-maven-plugin. This can be done by defining the following properties in the root pom.xml:

<!-- anchor:custom-properties:start -->
<expanders.version>4.7.0</expanders.version>
<expanders-maven-plugin.version>2020.12.0</expanders-maven-plugin.version>
<!-- anchor:custom-properties:end -->

You might also want to define a parent pom in the custom-parent anchor at the top of the file.

Next, you should be able to build the project with mvn install.

If succeeded, you can add the expansion-resource to your expansion-settings and run the expansion! Note that you will need at least expanders-maven-plugin 2020.12.0 or nsx-prime 2020.9.0 to run these expanders. (otherwise you will probably run into classloading issues).

Examples

You can find a ‘hello world’ example here.

Also, the data-validation-expanders repository contains an example of an external metamodel: https://bitbucket.org/normalizedsystems/data-validation-expanders/src/master/

References