Skip to main content

2 posts tagged with "guide"

View All Tags

Expanders using Data Resources

· 3 min read
Koen De Cock
Koen De Cock
R&D Engineer

DataResources have become a useful tool for expansion-resources to provide information to the Expanders. This post aims to describe how we can use the information provided by the data-resources in a set of Expanders, to gain insight in how the Expanders work. It gives a number of examples of Expanders using DataResources to make a list of libraries, option types and validation rules.

The data from DataResources is stored in a DataRegistryComposite. This object is available in mapping file as dataRegistry. The examples here will all work around the getComposites(String) method, which returns a list of all instances of an ElementType.

Expander 1. Listing libraries​

Library elements define runtime libraries used by the expanded projects. The logic deciding which libraries are added to the pom.xml files depends on the LayerImplementations of each LayerTypes. This example omits that complexity and instead filters on Technologies.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/1/0/mapping">
  <list name="libraries" eval="dataRegistry.getComposites('net.democritus.settings.Library')"
        param="lib">
    <filter name="onlyApplicableLibraries" eval="technologies.has(lib.technology.name)"/>
    <value name="name" eval="lib.name"/>
    <value name="groupId" eval="lib.groupId"/>
    <value name="artifactId" eval="lib.artifactId"/>
    <value name="version" eval="lib.version"/>
    <value name="technology" eval="lib.technology.name"/>
    <value name="provider" eval="lib.getMetadata('origin.expansionResource').orElse('-')"/>
  </list>
</mapping>
ExpansionResource metadata

Most DataResource elements contain metadata about the location where the data was found. To use, the most interesting information is the expansion-resource that declares this DataResource. This can be found by calling composite.getMetadata('origin.expansionResource'), which returns Optional<String>.

Expander 2. Listing deprecated option types​

Most up-to-date expansion-resources declare the options they support as OptionTypes.

We can filter on OptionTypes with a deprecation warning or an expiration time. The example separates the options that have yet to be expired and the ones that have already expired.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <let name="expirationHelper" eval="new org.normalizedsystems.example.OptionTypeExpirationHelper()"/>
    <list name="deprecatedOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="deprecatedOptions" eval="not opt.deprecationWarning.empty or not opt.validUntil.empty"/>
        <filter name="isNotExpired" eval="not expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
    <list name="expiredOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="hasExpirationTime" eval="not opt.validUntil.empty"/>
        <filter name="isExpired" eval="expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
</mapping>

Expander 3. Listing option types​

We can also get a lot more information from the OptionType elements.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <list name="optionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="notDeprecated" eval="opt.deprecationWarning.empty and opt.validUntil.empty"/>

        <value name="name" eval="opt.name"/>
        <value name="docLink" eval="opt.documentationLink"/>
        <value name="hasDocLink" eval="not opt.documentationLink.empty"/>
        <value name="alias" eval="opt.alias.replaceAll('\\s*,\\s*', ', ')"/>
        <value name="hasAlias" eval="not opt.alias.empty"/>

        <value name="description" eval="opt.description.replace('\n', ' ')"/>

        <list name="properties" eval="{'alwaysEnabled', 'redundant', 'hidden', 'cascading'}" param="prop">
            <filter name="isEnabled" eval="opt[prop]"/>
            <value name="name" eval="prop"/>
        </list>

        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>

        <value name="defaultValue" eval="opt.defaultValue"/>
        <value name="hasDefaultValue" eval="not opt.defaultValue.empty"/>
        <group name="value" if="opt.valueConstraint neq null">
            <value name="isRequired" eval="opt.valueConstraint.isRequired"/>
            <value name="noValue" eval="opt.valueConstraint.noValue"/>
            <value name="regex" eval="opt.valueConstraint.matchRegularExpression.replace('|', '&amp;#124;')"/>
            <value name="hasRegex" eval="not opt.valueConstraint.matchRegularExpression.empty"/>
        </group>

        <list name="elementTypes" eval="opt.elementTypes" param="elementType">
            <value name="name" eval="elementType.name"/>
        </list>
    </list>
</mapping>

Expander 4. Listing validation rules​

ValidationRules (as well as Transmuters) are a bit special. We create ValidationRules by implementing classes and annotating them with @ValidationRule. However, behind the scenes an annotation-processor creates a DataResource files describing these ValidationRules.

And thus, the ValidationRules are also available as data in the DataRegistry.

To make everything run smoothly, it's best to add some dependencies on the validation metamodel:

<dependencies>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-core</artifactId>
    <version>3.3.2</version>
  </dependency>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-test-support</artifactId>
    <version>3.3.2</version>
    <scope>test</scope>
  </dependency>
</dependencies>
<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
  <list name="groups" eval="dataRegistry.getComposites('net.democritus.validations.ValidationGroup')"
        param="group">
    <value name="name" eval="group.name"/>
    <value name="provider" eval="group.getMetadata('origin.expansionResource').orElse(null)"/>
    <list name="rules" eval="group.rules" param="rule">
      <value name="name" eval="rule.name"/>
      <value name="element" eval="rule.element"/>
      <value name="severity" eval="rule.severity.name"/>
      <value name="description" eval="rule.description.replace('\n', ' ')"/>
    </list>
  </list>
</mapping>

Decoupling expanders

· 6 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

As described in the post for the release of Expanders 5.33.0, we've recently introduced a new system to handle imports in expanders. We've since been slowly migrating more and more expanders to use this new system, which does bring frequent breaking changes with it in the form of missing imports. We don't specifically document these changes, since adding an import in custom code is merely a formality. But this change has exposed unwanted coupling in some expanders that could be avoided. In this post I'd like to go over some of these issues and how they can be avoided.

caution

It is important to keep in mind that expansion resources are independent projects, but may extend each other. They should be designed with care to make sure that if there are dependencies, you build that dependency in such a way that there's sufficient flexibility for them to evolve independently. Every resource will depend on a specific version of the other, so updating this minimum dependency is what should guarantee that they are still compatible. Stack resources can also be created to group multiple expansion resources that are compatible, to make it more convenient to use them together.

Over-specifying expander feature conditions​

An expander feature is a way of extending generated code with new functionality that often relates to a crosscutting concern. The way this works is similar to adding custom code into an expanded artifact. There are special anchors in the code, that starts with @anchor, which are places where expander features can be added. A common pitfall in designing features is over-specifying the enable condition for the feature.

Example

Let's say that you have a feature that adds some code in class A, this class is generated by an expander that checks if option enableA is present on an element. If you want to inject code into that class for a feature, you will typically enable that feature with a specific option, such as enableMyFeature. If this feature extends some functionality that is more constrained to that class A, often the condition for that feature will be enableA && enableMyFeature. What is important to realize here is that your feature should only check the conditions for enabling the feature itself. If enableA is not present, then the artifact A will not be generated and as a result, the feature will not insert any code in that artifact anyway.

Now on the other end, the expander represents a specific artifact, which is why you want to add a feature to it. If the condition to enable that expander would be changed, you typically will still want to add that feature, since the meaning of the artifact should not be different. But if you were checking the original condition of that expander along with that of the feature, this may no longer be compatible and the feature might not be added.

Example

As in the previous example, the expander for A could now be enabled with enableMyFunctionality that also enables an expander for B. If you had the condition enableA && enableMyFeature for your feature, it will no longer be injected, even though A is still generated. If you just checked enableMyFeature, it would still work, since your feature still injects into that same expander for artifact A.

Classpath dependencies​

If you depend on another expansion resource, that resource's contents will be available to you on the classpath, as this is how Java libraries work. This is required for you to have access to the expander and modelling frameworks. But this also comes with a risk. It is tempting to use helper classes or shared mapping files from an expansion resource dependency directly in your own helpers or mapping files, but this should be avoided at all cost.

These kinds of files have no guarantee of stability and are intended to be used for functionality of a specific expansion resource. By depending on them directly, you either constrain the flexibility of that parent resource, or you risk your resource breaking at random when these files change.

Anti-pattern

Including other mapping files using <include/> statements or delegating a mapping to Java code is an anti-pattern and should be avoided unless there's no good way of representing the mapping using OGNL statements in the mapping file for an expander. There are exceptions, but these are rare and the need to use these constructs is often a sign that there is something missing in the model, since mapping files should usually not be very complicated.

Feature mappings​

When creating a feature expander, you must specify a mapping for this feature. The feature does however have access to the mapping of the expander it injects into. While by itself this is not such an issue, as some implicit mappings are always present, a feature expander should not use the mappings of the expander it injects code into. All mappings for a feature should be provided by that feature itself in its mapping file, so it does not break when the mapping of the parent expander changes. This means that the mapping for the feature will only depend on the model itself, which is overall very stable.

Imports in features​

Originally we just expanded imports for Java code in the files directly and anchors are typically present to add more for custom code and with a feature expander. We've recently added support to define imports in the mapping file to have a more clean and consistent way of adding these imports and also allowing us to remap imports in cases where it is needed, such as for the transition from javax to jakarta.

With the new import system we've also switched to a new approach of aiming to only add imports in expanded code that are used by that code, so this is self-contained. This has however also shown that a lot of feature expanders do not do this and rely on the imports of the expander they inject code into.

With the new system, the aim should be for a feature expander to define EVERY import it needs for the code to compile in its mapping file with <uses/> statements. All imports will be merged into one pool and only unique imports will be added to the expanded file in the end, so this does not risk adding a bunch of duplicate imports. With this approach, any changes to imports in the parent expander will not affect the expanded feature and will allow them to evolve independently more freely.

tip

The FeatureExpanderTester has a new method testImports(CompositeProjection), that can be used to create a test for the imports generated by a feature. This works the same way as testBase(CompositeProjection), in that it will test against a string template in the test template file.