Skip to main content

17 posts tagged with "expanders"

View All Tags

Metamodels 2024.3.9

· 5 min read
Koen De Cock
Koen De Cock
R&D Engineer

Resources​

This version introduces a new expansion-resource ontology-metamodel-expanders, which is included in the metamodel stack.

ResourceVersion
nsx-metamodel-stack2024.3.9

Changes​

Expanders 2024.3.x introduce the ontology-metamodel-expanders. These expanders have been to made to replace the metamodel-expanders. They use the ontology-based metamodel directly instead of first requiring the GenerateMetamodelApplication transmuter.

Different Project Structure​

The new expanders generate a simplified project.

  • The different submodules for each component/ontology have been merged. There is now a single {ontology}-model submodule for each ontology
  • The onion-spec classes have been moved to the test-support module.
  • Harvesting ext files works differently. There are no ext/gen directories. Instead, the expanders harvest every file that was not expanded. The project contains a number of .harvestInclude files, which contain glob patterns for which files to harvest.
  • The expanders module has been removed. Instead, expanders should be implemented in a separate project similar to regular expander projects.

Merged with CompositeBuilder Expanders​

Before, it was necessary to expand with model-builder-expanders in a separate project to generate helper classes for transmuters. These expanders have now been merged into the ontology-metamodel-expanders so that this is no longer needed.

Profiles​

The new expanders make use of profiles to configure cross-cutting concerns. The Standard Metamodel enables every feature, but it is possible to create new profiles if necessary.

<dataResource type="expansionControl::ExpansionProfile">
  <profile name="Standard Metamodel">
    <tags>
      <tag>#meta.maven</tag>
      <tag>#meta.xml</tag>
      <tag>#meta.transmutation</tag>
    </tags>
  </profile>
</dataResource>

XML and XSD​

The expanders generate XSD files to describe the ElementClass XML files. The default namespace is https://schemas.normalizedsystems.org/xsd/${ontology}/${version}, which can be modified with the meta.xml.namespace option on ontology. If it contains ${version}, this will be replaced with the actual version.

There are also some changes to the XML representation.

  • References are represented as text-content with the full functional key.
  • Exported XML files have one or more xmlns attributes. In case of mixed metamodels, the XML tags can contain prefixes.

The XMLReaders are implemented to be backwards compatible (they can read both old and new formats).

Miscellaneous​

  • MappingContext: Now uses Optional instead of Option. There is an option meta.compatibility.mappingContext.option for backwards compatibility.
  • Dropped support for valueFieldTypes. Attributes now only support DataTypes with String, Boolean or Integer.
  • Dropped DataRefConverter. Keys can be parsed with a method on the {element}DataRef class.
  • Dropped CompositeRegistrar. This was not used.
  • Dropped OptionConverter.
  • CompositeScanner now also scans aggregate relations, e.g. Application -> Component.
  • SpecBuilders no longer support embedding parent specs into the child spec.
  • Now uses dataResource notation instead of _data.xml for most DataResources.
  • expansion directory uses expansionMetamodel name
  • Ontologies now have versions, and imported ontologies without versions are interpreted as being expanded with older expanders.

Expanders 6.6.0

· One min read
Koen De Cock
Koen De Cock
R&D Engineer

Resources​

The expansion resources below provide Expanders 6.6.0.

ResourceVersion
Expanders6.6.0
nsx-default-stack2024.2.0

Expander development changes​

Expanders 6.6.0 mainly focuses on improvements for Expander development.

Type attribute for uses​

Uses statements in mapping file can now have a type attribute. The way the type is resolved in specific to the import strategy.

For java import, you can now add static imports by adding type="static".

  <uses type="static" eval="'net.democritus.sys.NullDataRef.EMPTY_DATA_REF'"/>

@import parameters​

The @imports statement in Expander templates can now have one or more parameters. These parameters in case you have multiple types of imports in the same artifact (e.g. in Angular).

Read more.

ExpanderRelocation​

There is now support for renaming Expanders, changing packages or changing the artifact location. By defining an ExpanderRelocation DataResource, you can redirect Features and harvest files to the new location, so that everything still works the same way after the change.

Read more.

Expanders 6.4.0

· 2 min read
Koen De Cock
Koen De Cock
R&D Engineer

Resources​

The expansion resources below provide Expanders 6.4.4.

ResourceVersion
Expanders6.4.4
nsx-default-stack2024.0.2
rest-jaxrs-stack4.11.0

Changes and improvements​

Java 17​

Since Expanders 6.0.0, the expanders jars are compiled for Java 17. This means that expansion can only be executed with JDK 17 or higher.

However, expanded code is still java 8 compatible.

enteredBy, enteredAt, lastModifiedBy and lastModifiedAt​

There are some field names that, when used, would add additional behavior:

  • enteredAt: filled in with a timestamp when the instance is created
  • enteredBy: filled in with a dataRef to the user that created the instance
  • lastModifiedAt: updated with a timestamp each time the instance is modified
  • lastModifiedBy: updated with a dataRef to the user that modified it

There are now options that add the same behavior and can be added to any field (with the correct type).

  • enteredAt: option audit.create.timestamp
  • enteredBy: option audit.create.author
  • lastModifiedAt: option audit.modify.timestamp
  • lastModifiedBy: option audit.modify.author

The options are still implicitly added to the fields with those specific names.

Maven deprecation warnings​

Maven builds will now show warnings when deprecated code is used. Can be disabled with maven.hideDeprecationWarnings.

Expanders using Data Resources

· 3 min read
Koen De Cock
Koen De Cock
R&D Engineer

DataResources have become a useful tool for expansion-resources to provide information to the Expanders. This post aims to describe how we can use the information provided by the data-resources in a set of Expanders, to gain insight in how the Expanders work. It gives a number of examples of Expanders using DataResources to make a list of libraries, option types and validation rules.

The data from DataResources is stored in a DataRegistryComposite. This object is available in mapping file as dataRegistry. The examples here will all work around the getComposites(String) method, which returns a list of all instances of an ElementType.

Expander 1. Listing libraries​

Library elements define runtime libraries used by the expanded projects. The logic deciding which libraries are added to the pom.xml files depends on the LayerImplementations of each LayerTypes. This example omits that complexity and instead filters on Technologies.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/1/0/mapping">
  <list name="libraries" eval="dataRegistry.getComposites('net.democritus.settings.Library')"
        param="lib">
    <filter name="onlyApplicableLibraries" eval="technologies.has(lib.technology.name)"/>
    <value name="name" eval="lib.name"/>
    <value name="groupId" eval="lib.groupId"/>
    <value name="artifactId" eval="lib.artifactId"/>
    <value name="version" eval="lib.version"/>
    <value name="technology" eval="lib.technology.name"/>
    <value name="provider" eval="lib.getMetadata('origin.expansionResource').orElse('-')"/>
  </list>
</mapping>
ExpansionResource metadata

Most DataResource elements contain metadata about the location where the data was found. To use, the most interesting information is the expansion-resource that declares this DataResource. This can be found by calling composite.getMetadata('origin.expansionResource'), which returns Optional<String>.

Expander 2. Listing deprecated option types​

Most up-to-date expansion-resources declare the options they support as OptionTypes.

We can filter on OptionTypes with a deprecation warning or an expiration time. The example separates the options that have yet to be expired and the ones that have already expired.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <let name="expirationHelper" eval="new org.normalizedsystems.example.OptionTypeExpirationHelper()"/>
    <list name="deprecatedOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="deprecatedOptions" eval="not opt.deprecationWarning.empty or not opt.validUntil.empty"/>
        <filter name="isNotExpired" eval="not expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
    <list name="expiredOptionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="hasExpirationTime" eval="not opt.validUntil.empty"/>
        <filter name="isExpired" eval="expirationHelper.isExpired(opt)"/>
        <value name="name" eval="opt.name"/>
        <value name="validUntil" eval="opt.validUntil"/>
        <value name="description" eval="opt.description.replace('\n', ' ')"/>
        <value name="reason" eval="opt.deprecationWarning"/>
        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>
    </list>
</mapping>

Expander 3. Listing option types​

We can also get a lot more information from the OptionType elements.

<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
    <list name="optionTypes" eval="dataRegistry.getComposites('net.democritus.elements.OptionType')"
          param="opt">
        <filter name="notDeprecated" eval="opt.deprecationWarning.empty and opt.validUntil.empty"/>

        <value name="name" eval="opt.name"/>
        <value name="docLink" eval="opt.documentationLink"/>
        <value name="hasDocLink" eval="not opt.documentationLink.empty"/>
        <value name="alias" eval="opt.alias.replaceAll('\\s*,\\s*', ', ')"/>
        <value name="hasAlias" eval="not opt.alias.empty"/>

        <value name="description" eval="opt.description.replace('\n', ' ')"/>

        <list name="properties" eval="{'alwaysEnabled', 'redundant', 'hidden', 'cascading'}" param="prop">
            <filter name="isEnabled" eval="opt[prop]"/>
            <value name="name" eval="prop"/>
        </list>

        <value name="provider" eval="opt.getMetadata('origin.expansionResource').orElse('-')"/>

        <value name="defaultValue" eval="opt.defaultValue"/>
        <value name="hasDefaultValue" eval="not opt.defaultValue.empty"/>
        <group name="value" if="opt.valueConstraint neq null">
            <value name="isRequired" eval="opt.valueConstraint.isRequired"/>
            <value name="noValue" eval="opt.valueConstraint.noValue"/>
            <value name="regex" eval="opt.valueConstraint.matchRegularExpression.replace('|', '&amp;#124;')"/>
            <value name="hasRegex" eval="not opt.valueConstraint.matchRegularExpression.empty"/>
        </group>

        <list name="elementTypes" eval="opt.elementTypes" param="elementType">
            <value name="name" eval="elementType.name"/>
        </list>
    </list>
</mapping>

Expander 4. Listing validation rules​

ValidationRules (as well as Transmuters) are a bit special. We create ValidationRules by implementing classes and annotating them with @ValidationRule. However, behind the scenes an annotation-processor creates a DataResource files describing these ValidationRules.

And thus, the ValidationRules are also available as data in the DataRegistry.

To make everything run smoothly, it's best to add some dependencies on the validation metamodel:

<dependencies>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-core</artifactId>
    <version>3.3.2</version>
  </dependency>
  <dependency>
    <groupId>net.democritus.validations.model</groupId>
    <artifactId>validations-test-support</artifactId>
    <version>3.3.2</version>
    <scope>test</scope>
  </dependency>
</dependencies>
<?xml version="1.0" encoding="UTF-8" ?>
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
  <list name="groups" eval="dataRegistry.getComposites('net.democritus.validations.ValidationGroup')"
        param="group">
    <value name="name" eval="group.name"/>
    <value name="provider" eval="group.getMetadata('origin.expansionResource').orElse(null)"/>
    <list name="rules" eval="group.rules" param="rule">
      <value name="name" eval="rule.name"/>
      <value name="element" eval="rule.element"/>
      <value name="severity" eval="rule.severity.name"/>
      <value name="description" eval="rule.description.replace('\n', ' ')"/>
    </list>
  </list>
</mapping>

Expanders 5.35.1

· 2 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

This version of the expanders updates the groupId of all Maven modules in the expanded application project, to be more consistent and configurable.

Resources​

The expansion resources below provide Expanders 5.35.1.

ResourceVersion
Expanders5.35.1
nsx-default-stack2023.18.1
rest-jaxrs-stack3.28.1

Changes and improvements​

Maven groupIds​

In the past, all modules in the application project had the groupId org.normalizedsystems, except for the ear file module, which was org.normalized. To finally bring some consistency into this and differentiate them from those in other applications (to avoid collisions between base-component artifacts), this has now been changed.

The new groupId is now uniform across the entire project and is specifically org.normalizedsystems with the application shortname added after it in lowercase. So for shortname testApp it is org.normalizedsystems.testapp.

These new groupIds are merely the default settings though, it can now be configured with the application option maven.groupId, for which the value will be used as the groupId instead.

PascalCase for expander templating​

We've now added PascalCase as a format for all engines in the expander framework, the same way it is available for other formats.

Expanders 5.33.0

· 2 min read
Koen De Cock
Koen De Cock
R&D Engineer

This version of the expanders introduces improved support for imports in Expanders.

Pruned Imports

This version includes a cleanup of several of the unused imports in Expanders. It is possible you may need to add imports for classes you use in custom code after upgrading.

Resources​

The expansion resources below provide Expanders 5.33.2.

ResourceVersion
Expanders5.33.2
nsx-default-stack2023.16.4

Changes and improvements​

Better support of Imports in mapping files​

The mapping file syntax has been extended with 2 new keywords: artifact and uses.

Mapping files often had a separate list statement to resolve imports:

FinderDetailsExpanderMapping.xml
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/0/0/mapping">
  <list name="valueTypes"
        eval="finder.fieldOperatorPairs.{ field }.{? valueField neq null }.{ javaType }"
        param="javaType" unique="javaType">
    <filter name="requiresImport" eval="javaType.requiresImport()"/>
    <value name="qualifiedType" eval="javaType.qualifiedTypeName"/>
  </list>
  <list name="fields" eval="finder.fieldOperatorPairs"
          param="fieldOperatorPair" filter="fieldOperatorPair.field neq null">
    <let name="field" eval="fieldOperatorPair.field"/>
    <value name="name" eval="field.name"/>
    <!-- ... -->
  </list>
  <!-- ... -->
</mapping>

This can be replaced by adding uses statements for the imported classes. The artifact definition describes which import resolver to use.

FinderDetailsExpanderMapping.xml
<mapping xmlns="https://schemas.normalizedsystems.org/xsd/expanders/2023/1/0/mapping">
  <artifact this="classBuilder.from(finder).qualifiedName + 'Details'" importStrategy="java"/>
  <list name="fields" eval="finder.fieldOperatorPairs"
          param="fieldOperatorPair" filter="fieldOperatorPair.field neq null">
    <let name="field" eval="fieldOperatorPair.field"/>
    <value name="name" eval="field.name"/>
    <uses eval="field.javaType"/>
    <!-- ... -->
  </list>
  <!-- ... -->
</mapping>

The imports can then be inserted into the template with @imports:

FinderDetailsExpander.stg

base() ::= <<
package <dataElement.type.packageName>;

// <expanderComment>

// anchor:imports:start
@imports
// anchor:imports:end
@anchor:imports
...
>>

Check out the article to learn more.

Expanders 5.32.0

· 6 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

In this release of Expanders, we've cleaned up some long-deprecated code that still uses UserContext as the main context transport object. There's also been several improvements in the infrastructure to improve ease of use for functional keys.

Resources​

The expansion resources below provide Expanders 5.32.0.

ResourceVersion
Expanders5.32.0
nsx-default-stack2023.15.2
rest-jaxrs-stack3.27.0

Changes and improvements​

Functional key infrastructure​

FinderResolver​

A new class FinderResolver has been introduced, which is called from the Bean.find() method in the logic layer. This class will automatically resolve the DataRefs for any field operator pairs that target link fields. Resolving means that it will resolve the DataRef against the database in case there's no database identifier present in it. While much of the methods in the stack already did this, the finders still did not, which often caused some confusion. The finder would not work as intended if the database identifier was not present in the DataRefs. This resolution will only occur if there is no database identifier in the DataRef, so the impact on performance should be negligible in existing code.

Agent.getDetails()​

The Agent.getDetails(DataRef) method did not behave quite the same as LocalAgent.getDetails(DataRef). Even though the same code was called down the line, the way it worked on the remote agent would lose all information in the DataRef except for the database identifier. This meant that it had to be resolved prior to use. This issue has now been resolved as well, by having it call getProjection() instead, just like its LocalAgent counterpart.

Context transport​

In a distant past, there was no Context class, rather the only context was UserContext ans it was used to pass all context information. This has been replaced for years in favor of the Context class and the UserContext has since only been used for actual information about the user. Therefore, we've started removing a lot of this deprecated infrastructure that still relied on UserContext where Context should be used instead. Some of these methods include Agent.getAgent(UserContext) and LocalAgent.getLocalAgent().

Deprecating old methods​

We intend to remove a lot of clutter that has been created in the stack over the years, with methods for backwards compatibility that should no longer be used. We've started marking many of these methods as deprecated for removal, so it is already clear to developers that these should not be used in new code and should be refactored if they are still in use in older code. A prime example of these are methods in the stack that still use a database identifier as a Long parameter, rather than a DataRef. We'll be marking more of these as deprecated in future releases.

Typed finders​

The typed finders system is now the default for all applications. The option finders.enableTypedFinders is now no longer required. Typed finders means that generics are used through the stack for finders, to ensure that the compiler will verify that they are not used with infrastructure of the wrong DataElement. It also changes the FinderBean to use mapped method references for the finder implementations, rather than reflection. This means that it is no longer possible to defined custom finders that are not present in the model.

Expanders 5.30.0

· 3 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

This release of Expanders is the final push to remove all uses of net.palver.util.Options.Option in our JEE stack.

Resources​

The expansion resources below provide Expanders 5.30.0.

ResourceVersion
Expanders5.30.1
nsx-default-stack2023.14.1
rest-jaxrs-stack3.23.0
note

Some very specific fixes were made after the release of 5.30.0 itself, hence the discrepancy in versions here. But this is not relevant for projects that were already on 3.28.0 and use Java 17. But it is best to use the latest versions anyway.

Changes and improvements​

Fully deprecating Options.Option​

This last refactoring of Options.Option to Optional in the application stack required reworking code across several projects. Extensive changes were made to nsx-runtime, Expanders, base-components and alerting-component. We did manage however to reduce the impact of the changes to a minimum, so most projects will experience little impact from the migration.

We did choose to introduce a hard breaking change in one location, namely the getProperty() method of the expanded <component>ApplicationSettings class when the option configuration.properties is added to a component. The use of this option is still fairly limited at this time, so it seemed better to go for some minor refactoring in those projects that use it, than to add a new method like we did in Context and SearchResult.

The Options.Option class itself will still be available for quite some time, so projects can migrate to Optional on their own time, but we do intend to remove it in the future.

Expanders 5.29.0

· 3 min read
Frédéric Hannes
Frédéric Hannes
R&D Engineer

This release of Expanders is mostly focused around changes related to Struts2, amongst which the removal of all support for versions 2.3.x and 2.5.x, as well as the update to version 6.2.0.

Resources​

The expansion resources below provide Expanders 5.29.0.

ResourceVersion
Expanders5.29.0
nsx-default-stack2023.13.0
rest-jaxrs-stack3.22.0

Changes and improvements​

Removal of old version support​

We have been working towards removing support for outdated versions of Struts2 for some time now. On July 1st, the deprecated option struts.version expired. We've now removed all support that relied on this option to target Struts 2.3 and 2.5. This was announced last year with the release of Expanders 5.15.0.

Struts 6.2​

Struts version 6.2 was released yesterday as announced by Apache. This version brings mostly bugfixes and some improvements, but it also comes with some breaking changes. There's no official migration guide for this, but we've described some of these changes and the fixes in our migration guide below. Most of these changes will however be handled by our expanders, so the impact on projects should be fairly minimal.

Refactoring​

With the refactoring needed for the new Struts update, we've also taken the time to revisit some of the code in our JEE stack related to Struts and made some minor improvements. This includes replacing remaining code that still pulled the UserContext object directly from the session with Context. Though UserContext remains part of the session for now, this might change in the future

Expanders 5.28.0

· 3 min read
Jorren Hendriks
Jorren Hendriks
R&D Engineer

Efforts to reimagine and improve the way workflows are modeled and implemented within our applications have started a while ago. This release marks the first impact of those efforts on Expanders.

Resources​

The expansion resources below provide Expanders 5.28.0.

ResourceVersion
Expanders5.28.0
nsx-default-stack2023.12.0
rest-jaxrs-stack3.21.0

Changes and improvements​

Task perform and delegation​

Modeling your application should be as simple as possible. This includes providing developers with sensible defaults. Best practices shouln't be hidden behind an option. For this reason the following options have become deprecated and are now the default behavior:

  • includePerform Every task should be able to be performed. The expanded perform method has the advantage that its contract aligns with the TaskElement model.
  • includeDelegation In the JEE application stack a task is accessed as an EJB. Delegation allows us to implement the task independently of this technology. It will also improve the ability to test this implementation. To opt-out of delegate implementation(s) you can use the new noDelegation option.

Additionally, adding the includeRemoteAccess option is no longer recommended. This option exposes a task directly to a Struts api, which doesn't have the best interface and implementation.

Non-dynamic task implementation​

Available task implementations must be known at compile-time, which is why we replaced the dynamic lookup using Class.forName(..) with a switch-statement. This explicitly lists all available implementations and allows for better code navigation.

Multiple task implementations for the same task are infrequently used. However, when applicable it should at least be clear what each implementation does. For this reason we chose to retire the <Task>Impl2.java artifact, custom implementations with meaningful names should be used instead.

Additionally, getting rid of the dynamic lookup allows us to use named implementations instead of fully qualified class names as well. You could for example have a task implemented with and without encryption of some data. These implementations can now be named default and secure, which would then resolve to their respective implementation. Since the implementation name is just a string, you can still use fully qualified names as well.