Skip to main content

Expanders 6.6.0

· One min read
Koen De Cock
Koen De Cock
R&D Engineer

Resources​

The expansion resources below provide Expanders 6.6.0.

ResourceVersion
Expanders6.6.0
nsx-default-stack2024.2.0

Expander development changes​

Expanders 6.6.0 mainly focuses on improvements for Expander development.

Type attribute for uses​

Uses statements in mapping file can now have a type attribute. The way the type is resolved in specific to the import strategy.

For java import, you can now add static imports by adding type="static".

  <uses type="static" eval="'net.democritus.sys.NullDataRef.EMPTY_DATA_REF'"/>

@import parameters​

The @imports statement in Expander templates can now have one or more parameters. These parameters in case you have multiple types of imports in the same artifact (e.g. in Angular).

Read more.

ExpanderRelocation​

There is now support for renaming Expanders, changing packages or changing the artifact location. By defining an ExpanderRelocation DataResource, you can redirect Features and harvest files to the new location, so that everything still works the same way after the change.

Read more.

Angular Expanders 4.3.0

· 2 min read
Jan Hardy
Jan Hardy
R&D Engineer

Changes and improvements​

This update contains a lot of improvements for wiring capabilities, making it easier to reuse expanded elements in custom pages.

List page refactored​

The expanded list page has been updated to use the new table with multi sort capabilities. The table is extracted from the page component and resides in its separate DataViewTable component. It now uses the upgraded filter component, that is generated based on the QuerySearch setup. Filtering state between the ngx-ns-core component and the generated dropdown form is managed by the new DataConnectorFilterSource service. Instead of managing the pagination, filtering and sorting manually in the list component, a DataConnectorDataSource service has been introduced to make this easier.

Table refactored​

Breaking change: Existing components nsx-table, nsx-table-column and directive [nsxTableCell] have been replaced with counterparts from the ngx-ns-core library.

The table now has multi sort capabilities, supports sticky headers and columns, and accepts a NscDataSource as well as data arrays.

Paginator refactored​

Breaking change: Existing component nsx-paginator has been replaced with counterpart from the ngx-ns-core library.

Filter refactored​

Breaking change: Existing component nsx-table-search has been replaced with counterpart from the ngx-ns-core library. The filtering capabilities have been upgraded.

Expanders 6.4.0

· 2 min read
Koen De Cock
Koen De Cock
R&D Engineer

Resources​

The expansion resources below provide Expanders 6.4.4.

ResourceVersion
Expanders6.4.4
nsx-default-stack2024.0.2
rest-jaxrs-stack4.11.0

Changes and improvements​

Java 17​

Since Expanders 6.0.0, the expanders jars are compiled for Java 17. This means that expansion can only be executed with JDK 17 or higher.

However, expanded code is still java 8 compatible.

enteredBy, enteredAt, lastModifiedBy and lastModifiedAt​

There are some field names that, when used, would add additional behavior:

  • enteredAt: filled in with a timestamp when the instance is created
  • enteredBy: filled in with a dataRef to the user that created the instance
  • lastModifiedAt: updated with a timestamp each time the instance is modified
  • lastModifiedBy: updated with a dataRef to the user that modified it

There are now options that add the same behavior and can be added to any field (with the correct type).

  • enteredAt: option audit.create.timestamp
  • enteredBy: option audit.create.author
  • lastModifiedAt: option audit.modify.timestamp
  • lastModifiedBy: option audit.modify.author

The options are still implicitly added to the fields with those specific names.

Maven deprecation warnings​

Maven builds will now show warnings when deprecated code is used. Can be disabled with maven.hideDeprecationWarnings.

Angular Expanders 4.2.0

· 3 min read
Jan Hardy
Jan Hardy
R&D Engineer

Changes and improvements​

Upgrade to Angular 17 and Angular Material 17​

In the previous release (4.1.x) we upgraded to from Angular 14 to 16, but used legacy Angular Material 14 components. Now we have upgraded completely to the latest 17 version.

Breaking change: in routing change name and isBreadcrumb data entries to breadcrumb. The former is replaced by breadcrumb and the latter has become obsolete.

Styling setup refactored​

Breaking change: _theme-to-material-component-exposer.scss has been replaced with _theme-to-component-exposer.scss. Material expose mixin _theme-to-css-classes-exposer.scss has been deleted. Behaviour can be re-added custom if wanted.

Switch to Material symbols​

We switched to Material symbols instead of Material icons as default icon font set.

Added use of Angular library​

Created the ngx-ns-core library containing runtime components, services and directives. Some of the expanded components and services have already been migrated, e.g. sidebar, top-bar, breadcrumbs, etc.

REST Expanders 4.0.0

· 3 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

Changes and improvements​

Migrating to QuerySearch metamodel​

Since version 3.x of rest-expanders, which was released just over a year ago, rest-expanders has integrated with querysearch-expanders by using the QuerySearch metamodel. This was introduced in such a way that it was 100% backwards compatible and also added to the model when expanding implicitly, so no change could really be observed from the perspective of an application developer.

Today the QuerySearch metamodel has matured to the point that what currently exists of the model is stable and will no longer have any big structural changes that are not additive. So with this version of rest-expanders, the implicitly added model will be removed. This means that it has to be defined in the model by the developer and coupled to the rest-expanders using the relevant jaxrs.querySearch option that has been implicitly used for quite some time.

The transmuters to add default REST APIs to the model have been modified to do this automatically, so in that sense, there will be no change to the way of interacting with rest-expanders in the majority of cases.

DataCommand options​

In rest-expanders version 3.8.0, options were introduced to mark the DataCommands that are to be used for the CRUD operations in the REST API. Starting with this version, the validation to check if this has been migrated has been removed, as well as them being implicitly added to the model. We assume that every project has been migrated by now, but this will be the last version where the Upgrade transmutation for rest-expanders does this migration automatically.

Expanders using Data Resources

· 3 min read
Koen De Cock
Koen De Cock
R&D Engineer

DataResources have become a useful tool for expansion-resources to provide information to the Expanders. This post aims to describe how we can use the information provided by the data-resources in a set of Expanders, to gain insight in how the Expanders work. It gives a number of examples of Expanders using DataResources to make a list of libraries, option types and validation rules.

The data from DataResources is stored in a DataRegistryComposite. This object is available in mapping file as dataRegistry. The examples here will all work around the getComposites(String) method, which returns a list of all instances of an ElementType.

Expander 1. Listing libraries​

Library elements define runtime libraries used by the expanded projects. The logic deciding which libraries are added to the pom.xml files depends on the LayerImplementations of each LayerTypes. This example omits that complexity and instead filters on Technologies.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/1/0/mapping">
  <list name="libraries" eval="dataRegistry.getComposites('net.democritus.settings.Library')"
        param="lib">
    <filter name="onlyApplicableLibraries" eval="technologies.has(lib.technology.name)"/>
    <value name="name" eval="lib.name"/>
    <value name="groupId" eval="lib.groupId"/>
    <value name="artifactId" eval="lib.artifactId"/>
    <value name="version" eval="lib.version"/>
    <value name="technology" eval="lib.technology.name"/>
    <value name="provider" eval="lib.getMetadata('origin.expansionResource').orElse('-')"/>
  </list>
</mapping>
ExpansionResource metadata

Most DataResource elements contain metadata about the location where the data was found. To use, the most interesting information is the expansion-resource that declares this DataResource. This can be found by calling composite.getMetadata('origin.expansionResource'), which returns Optional<String>.

Expander 2. Listing deprecated option types​

Most up-to-date expansion-resources declare the options they support as OptionTypes.

We can filter on OptionTypes with a deprecation warning or an expiration time. The example separates the options that have yet to be expired and the ones that have already expired.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <let name="expirationHelper" eval="new org.normalizedsystems.example.OptionTypeExpirationHelper()"/>
    <list name="deprecatedOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="deprecatedOptions" eval="not opt.deprecationWarning.empty or not opt.validUntil.empty"/>
        <filter name="isNotExpired" eval="not expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
    <list name="expiredOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="hasExpirationTime" eval="not opt.validUntil.empty"/>
        <filter name="isExpired" eval="expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
</mapping>

Expander 3. Listing option types​

We can also get a lot more information from the OptionType elements.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <list name="optionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="notDeprecated" eval="opt.deprecationWarning.empty and opt.validUntil.empty"/>

        <value name="name" eval="opt.name"/>
        <value name="docLink" eval="opt.documentationLink"/>
        <value name="hasDocLink" eval="not opt.documentationLink.empty"/>
        <value name="alias" eval="opt.alias.replaceAll('\\s*,\\s*', ', ')"/>
        <value name="hasAlias" eval="not opt.alias.empty"/>

        <value name="description" eval="opt.description.replace('\n', ' ')"/>

        <list name="properties" eval="{'alwaysEnabled', 'redundant', 'hidden', 'cascading'}" param="prop">
            <filter name="isEnabled" eval="opt[prop]"/>
            <value name="name" eval="prop"/>
        </list>

        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>

        <value name="defaultValue" eval="opt.defaultValue"/>
        <value name="hasDefaultValue" eval="not opt.defaultValue.empty"/>
        <group name="value" if="opt.valueConstraint neq null">
            <value name="isRequired" eval="opt.valueConstraint.isRequired"/>
            <value name="noValue" eval="opt.valueConstraint.noValue"/>
            <value name="regex" eval="opt.valueConstraint.matchRegularExpression.replace('|', '&amp;#124;')"/>
            <value name="hasRegex" eval="not opt.valueConstraint.matchRegularExpression.empty"/>
        </group>

        <list name="elementTypes" eval="opt.elementTypes" param="elementType">
            <value name="name" eval="elementType.name"/>
        </list>
    </list>
</mapping>

Expander 4. Listing validation rules​

ValidationRules (as well as Transmuters) are a bit special. We create ValidationRules by implementing classes and annotating them with @ValidationRule. However, behind the scenes an annotation-processor creates a DataResource files describing these ValidationRules.

And thus, the ValidationRules are also available as data in the DataRegistry.

To make everything run smoothly, it's best to add some dependencies on the validation metamodel:

<dependencies>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-core</artifactId>
    <version>3.3.2</version>
  </dependency>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-test-support</artifactId>
    <version>3.3.2</version>
    <scope>test</scope>
  </dependency>
</dependencies>
<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
  <list name="groups" eval="dataRegistry.getComposites('net.democritus.validations.ValidationGroup')"
        param="group">
    <value name="name" eval="group.name"/>
    <value name="provider" eval="group.getMetadata('origin.expansionResource').orElse(null)"/>
    <list name="rules" eval="group.rules" param="rule">
      <value name="name" eval="rule.name"/>
      <value name="element" eval="rule.element"/>
      <value name="severity" eval="rule.severity.name"/>
      <value name="description" eval="rule.description.replace('\n', ' ')"/>
    </list>
  </list>
</mapping>

Decoupling expanders

· 6 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

As described in the post for the release of Expanders 5.33.0, we've recently introduced a new system to handle imports in expanders. We've since been slowly migrating more and more expanders to use this new system, which does bring frequent breaking changes with it in the form of missing imports. We don't specifically document these changes, since adding an import in custom code is merely a formality. But this change has exposed unwanted coupling in some expanders that could be avoided. In this post I'd like to go over some of these issues and how they can be avoided.

caution

It is important to keep in mind that expansion resources are independent projects, but may extend each other. They should be designed with care to make sure that if there are dependencies, you build that dependency in such a way that there's sufficient flexibility for them to evolve independently. Every resource will depend on a specific version of the other, so updating this minimum dependency is what should guarantee that they are still compatible. Stack resources can also be created to group multiple expansion resources that are compatible, to make it more convenient to use them together.

Over-specifying expander feature conditions​

An expander feature is a way of extending generated code with new functionality that often relates to a crosscutting concern. The way this works is similar to adding custom code into an expanded artifact. There are special anchors in the code, that starts with @anchor, which are places where expander features can be added. A common pitfall in designing features is over-specifying the enable condition for the feature.

Example

Let's say that you have a feature that adds some code in class A, this class is generated by an expander that checks if option enableA is present on an element. If you want to inject code into that class for a feature, you will typically enable that feature with a specific option, such as enableMyFeature. If this feature extends some functionality that is more constrained to that class A, often the condition for that feature will be enableA && enableMyFeature. What is important to realize here is that your feature should only check the conditions for enabling the feature itself. If enableA is not present, then the artifact A will not be generated and as a result, the feature will not insert any code in that artifact anyway.

Now on the other end, the expander represents a specific artifact, which is why you want to add a feature to it. If the condition to enable that expander would be changed, you typically will still want to add that feature, since the meaning of the artifact should not be different. But if you were checking the original condition of that expander along with that of the feature, this may no longer be compatible and the feature might not be added.

Example

As in the previous example, the expander for A could now be enabled with enableMyFunctionality that also enables an expander for B. If you had the condition enableA && enableMyFeature for your feature, it will no longer be injected, even though A is still generated. If you just checked enableMyFeature, it would still work, since your feature still injects into that same expander for artifact A.

Classpath dependencies​

If you depend on another expansion resource, that resource's contents will be available to you on the classpath, as this is how Java libraries work. This is required for you to have access to the expander and modelling frameworks. But this also comes with a risk. It is tempting to use helper classes or shared mapping files from an expansion resource dependency directly in your own helpers or mapping files, but this should be avoided at all cost.

These kinds of files have no guarantee of stability and are intended to be used for functionality of a specific expansion resource. By depending on them directly, you either constrain the flexibility of that parent resource, or you risk your resource breaking at random when these files change.

Anti-pattern

Including other mapping files using <include/> statements or delegating a mapping to Java code is an anti-pattern and should be avoided unless there's no good way of representing the mapping using OGNL statements in the mapping file for an expander. There are exceptions, but these are rare and the need to use these constructs is often a sign that there is something missing in the model, since mapping files should usually not be very complicated.

Feature mappings​

When creating a feature expander, you must specify a mapping for this feature. The feature does however have access to the mapping of the expander it injects into. While by itself this is not such an issue, as some implicit mappings are always present, a feature expander should not use the mappings of the expander it injects code into. All mappings for a feature should be provided by that feature itself in its mapping file, so it does not break when the mapping of the parent expander changes. This means that the mapping for the feature will only depend on the model itself, which is overall very stable.

Imports in features​

Originally we just expanded imports for Java code in the files directly and anchors are typically present to add more for custom code and with a feature expander. We've recently added support to define imports in the mapping file to have a more clean and consistent way of adding these imports and also allowing us to remap imports in cases where it is needed, such as for the transition from javax to jakarta.

With the new import system we've also switched to a new approach of aiming to only add imports in expanded code that are used by that code, so this is self-contained. This has however also shown that a lot of feature expanders do not do this and rely on the imports of the expander they inject code into.

With the new system, the aim should be for a feature expander to define EVERY import it needs for the code to compile in its mapping file with <uses/> statements. All imports will be merged into one pool and only unique imports will be added to the expanded file in the end, so this does not risk adding a bunch of duplicate imports. With this approach, any changes to imports in the parent expander will not affect the expanded feature and will allow them to evolve independently more freely.

tip

The FeatureExpanderTester has a new method testImports(CompositeProjection), that can be used to create a test for the imports generated by a feature. This works the same way as testBase(CompositeProjection), in that it will test against a string template in the test template file.

Expanders 5.35.1

· 2 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

This version of the expanders updates the groupId of all Maven modules in the expanded application project, to be more consistent and configurable.

Resources​

The expansion resources below provide Expanders 5.35.1.

ResourceVersion
Expanders5.35.1
nsx-default-stack2023.18.1
rest-jaxrs-stack3.28.1

Changes and improvements​

Maven groupIds​

In the past, all modules in the application project had the groupId org.normalizedsystems, except for the ear file module, which was org.normalized. To finally bring some consistency into this and differentiate them from those in other applications (to avoid collisions between base-component artifacts), this has now been changed.

The new groupId is now uniform across the entire project and is specifically org.normalizedsystems with the application shortname added after it in lowercase. So for shortname testApp it is org.normalizedsystems.testapp.

These new groupIds are merely the default settings though, it can now be configured with the application option maven.groupId, for which the value will be used as the groupId instead.

PascalCase for expander templating​

We've now added PascalCase as a format for all engines in the expander framework, the same way it is available for other formats.

Workflows 1.6.0 & ProcessAutomation 0.1.0

· 2 min read
Jorren Hendriks
Jorren Hendriks
R&D Engineer

We have been hard at work on the process automation component, which supersedes the workflow component.

Resources​

ResourceVersion
workflow-model-stack 1.6.1
process-automation-component 0.1.2

Workflow model​

The first step is moving away from database config into the model. The workflow model allows you to define all workflows statically. It also provides bindings for the workflow component such that it automatically updates the database configuration when changes are made to the model. Having the information in the model also gives the MicroRadiant the ability to visualize your workflows.

We also provide a tool for importing existing database configuration into the model.

Stack Rename

From version 1.6.0 onwards, the stack resource is called net.democritus.workflow:workflow-model-stack.

This previously was net.democritus.workflow:workflows-metamodel-stack.

Process automation component​

A preview release is now available for the new process automation component!

note

The version starts with 0.x.x. This is to indicate it is still likely breaking changes are introduced as we move towards a stable 1.0.0 release.

The component is a new approach to workflows, designed with more flexibility in mind. The key distinction is queue based processing, where the previous component could only handle timed polling. This means we can now start supporting a wide range of triggers, such as CRUD events, transitions, cron schedules and even the flow engines you're already familiar with.

Migration​

Migration can be quite involved depending on the current state of your application. To ease the difficulty, we provide a migration guide with smaller steps so applications can incrementally adopt the new component.

It is recommended for all applications to at least perform Step 1 (workflow model) and Step 2 (task outcome model). This will already give some advantages like model-based configuration and makes it easier to adopt process automation on a stable release.

Expanders 5.33.0

· 2 min read
Koen De Cock
Koen De Cock
R&D Engineer

This version of the expanders introduces improved support for imports in Expanders.

Pruned Imports

This version includes a cleanup of several of the unused imports in Expanders. It is possible you may need to add imports for classes you use in custom code after upgrading.

Resources​

The expansion resources below provide Expanders 5.33.2.

ResourceVersion
Expanders5.33.2
nsx-default-stack2023.16.4

Changes and improvements​

Better support of Imports in mapping files​

The mapping file syntax has been extended with 2 new keywords: artifact and uses.

Mapping files often had a separate list statement to resolve imports:

FinderDetailsExpanderMapping.xml
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
  <list name="valueTypes"
        eval="finder.fieldOperatorPairs.{ field }.{? valueField neq null }.{ javaType }"
        param="javaType" unique="javaType">
    <filter name="requiresImport" eval="javaType.requiresImport()"/>
    <value name="qualifiedType" eval="javaType.qualifiedTypeName"/>
  </list>
  <list name="fields" eval="finder.fieldOperatorPairs"
          param="fieldOperatorPair" filter="fieldOperatorPair.field neq null">
    <let name="field" eval="fieldOperatorPair.field"/>
    <value name="name" eval="field.name"/>
    <!-- ... -->
  </list>
  <!-- ... -->
</mapping>

This can be replaced by adding uses statements for the imported classes. The artifact definition describes which import resolver to use.

FinderDetailsExpanderMapping.xml
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/1/0/mapping">
  <artifact this="classBuilder.from(finder).qualifiedName + 'Details'" importStrategy="java"/>
  <list name="fields" eval="finder.fieldOperatorPairs"
          param="fieldOperatorPair" filter="fieldOperatorPair.field neq null">
    <let name="field" eval="fieldOperatorPair.field"/>
    <value name="name" eval="field.name"/>
    <uses eval="field.javaType"/>
    <!-- ... -->
  </list>
  <!-- ... -->
</mapping>

The imports can then be inserted into the template with @imports:

FinderDetailsExpander.stg

base() ::= <<
package <dataElement.type.packageName>;

// <expanderComment>

// anchor:imports:start
@imports
// anchor:imports:end
@anchor:imports
...
>>

Check out the article to learn more.