Changes and improvements​

Refactoring Options.Option​

As net.democritus.sys.Options.Option is being phased out, this version of rest-expanders replaced the vast majority of its uses with java.util.Optional. All of these changes are located in the validation system and may affect custom code.

Changes and Improvements​

Both Model Validations and Model Transmutations have been refactored to make configuration easier.

The annotation processors are now defined in the validations-core and modelTransmutations-core jars and no longer need to be explicitly added to your maven builds.

<plugins>
  <plugin>
    <groupId>net.democritus.maven.plugins</groupId>
    <artifactId>expanders-maven-plugin</artifactId>
    <version>${expanders-maven-plugin.version}</version>
    <executions>
      <execution>
        <goals>
          <goal>expansionResource</goal>
        </goals>
        <configuration>
          <includeClassPathDependencies>true</includeClassPathDependencies>
        </configuration>
      </execution>
    </executions>
  </plugin>
  <!-- Remove this -->
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.10.0</version>
    <configuration>
      <showWarnings>true</showWarnings>
      <annotationProcessors>
        <annotationProcessor>
          net.democritus.transmutations.compilation.TransmutationAnnotationProcessor
        </annotationProcessor>
      </annotationProcessors>
    </configuration>
  </plugin>
  ...
</plugins>

Furthermore, the generated DataResources are generated in the new format.

Migration​

Update your expanders-maven-plugin to 2023.3.1 or higher:

<expanders-maven-plugin.version>2023.3.1</expanders-maven-plugin.version>

Next, remove the DataResource reference in your _data.xml file:

<dataResources>
  ...
  <!-- Remove these -->
  <dataResource>
    <path>data/transmutations/modelTransmutations.xml</path>
    <elementTypeCanonicalName>net.democritus.transmutations.ModelTransmutation</elementTypeCanonicalName>
  </dataResource>
  <dataResource>
    <path>data/validations/validationGroups.xml</path>
    <elementTypeCanonicalName>net.democritus.validations.ValidationGroup</elementTypeCanonicalName>
  </dataResource>
</dataResources>

Changes and Improvements​

The plugin has been refactored to support an easier way to register data-resources with the expansionResource goal.

With the latest version, you can create an XML file with dataResource as root tag and a type attribute:

<dataResource type="elements::Technology">
  <technology name="HSQLDB">
    <subdir>hsql</subdir>
    <concernType name="DATABASE"/>
    <description>-</description>
  </technology>
  <technology name="MARIADB">
    <subdir>mariadb</subdir>
    <concernType name="DATABASE"/>
    <description>MariaDB Database</description>
  </technology>
  <technology name="POSTGRESQL">
    <subdir>postgresql</subdir>
    <concernType name="DATABASE"/>
    <description>PostgreSQL Database</description>
  </technology>
</dataResource>

The file should still be placed in a resources directory or another directory that is included in the build. The expansionResource mojo will scan through the target/classes directory to find these files and register them. Because of this change, we no longer need a _data.xml file to describe the locations of the data-resources.

Changes and improvements​

Legacy constraint methods​

The when(boolean, QueryConstraint) and whenOrElse(boolean, QueryConstraint) methods have been removed after being marked as deprecated for three years. These methods were fundamentally flawed as the constraints are passed as a parameter adn as such evaluated immediately, instead of only when the condition is true as per the design of these constraints.

The exists(qualifiedElementName, variableName, constraint, queryOptions) constraint method has now been marked as deprecated. This method was superseded by a new exists() method in version 1.7.0, which takes a QueryBuilder instance as an argument. Using the QueryBuilder instance if more version transparent and more readable.

Meta-model​

The QuerySearchOptionType element has been removed in favor of using the new ValueType system introduced in the elements metamodel. This change aligns the querySearch metamodel more with the elements model which it extend and will allow for more flexibility as the metamodel is refined further. This change was also planned for quite a long time ago and the reason the model was still considered unstable. With this change now implemented, the querySearch metamodel can now be considered to be stable and will no longer be subjected to large breaking changes in the future.

Over the last year we've put a lot of effort into modernizing our NS application stack and improving its quality. This release of expanders marks the start of a new and important step in that process. We are starting to remove the use of net.palver.util.Options.Option in favor of Java's java.util.Optional.

Resources​

The expansion resources below provide Expanders 5.22.0.

Changes and improvements​

Optional​

We will be rolling out these changes in stages, as they will be considerably impactful. Though the first steps have been taken in this release, we have started by refactoring a bunch of code that has limited external use and cleaning up legacy code that is no longer in use which also used Options.Option.

Most changes for this release are actually part of the nsx-runtime library, version 2023.1.0. The expanders release and release of several other expansion resources is mostly to update affected code in the expanders themselves or expanded by them. If there is any impact from these changes, it will most likely be from the use of the component metadata system. You can refer to the nsx-runtime changelog for a comprehensive list of these changes. Where methods previously returned Options.Option, they now return Optional, which has a few differences in method signatures that will require refactoring:

• Options.Option.isDefined()
Optional.isPresent()
• Options.Option.isEmpty()
Optional.isEmpty() for Java 11 or 17, !Optional.isPresent() for Java 8
• Options.Option.getValue()
Optional.get()
• Options.some()
Optional.of()
• Options.notNull()
Optional.ofNullable()
• Options.none()
Optional.empty()

Where possible, it is prudent to use Options.Option.asOptional() in custom code, which should make it easier to refactor for future changes.

The newest addition to our family of base components, the alerting component, has been released. The alerting component is intended to send simple alerts/notifications to various channels based on a pub-sub system. This is also an experiment to make more modular/extensible components by providing specific expanded integration points. Implementations for new channel to send alerts to can be provided externally in different components and will be integrated seamlessly into the system by the included expanders.

Out-of-the-box, it can send alerts over email, Slack and Pushover. For more information, check out the documentation page.

Changes and Improvements​

New ValueType System​

ValueFieldTypes are now obsolete and will be replaced with ValueTypes.

You can use the MigrateValueFieldsToValueType transmutation to migrate your ValueFields to the new ValueType representation.

The new ValueTypes support adding restrictions to some basic ValueTypes, like String patterns. So, it will be useful to check out the supported restrictions and see where they can be used in the model.

<field name="foo">
  <valueField>
    <type name="Text"/>
  </valueField>
</field>

Mapping Xml Improvements​

Some improvements have been added to simplify mapping files.

Filter and Distinct in List​

First, the <list> tag has 2 new child tags:

• <filter name="filterName" eval="condition">: Evaluate the condition. If the condition is false, the item will not appear in the list.
• <distinct name="distinctionName" eval="value">: Evaluate the value. Items with duplicate values will be de-duplicated.

These feature were previously already available as the filter and unique attributes on the <list> tag. However, these tags can be interleaved with <let> tags to simplify some of the logic.

E.g. this:

<list name="oneToManyLinkFields" eval="dataElement.fields" param="field"
        filter="field.linkField neq null and not field.linkField.linkFieldType.isMultivalued">
    <value name="name" eval="field.name"/>
    <value name="element" eval="field.linkField.linkFieldType.targetElement.name"/>
</list>

becomes:

<list name="oneToManyLinkFields" eval="dataElement.fields" param="field">
    <filter name="isLinkField" eval="field.linkField neq null"/>
    <let name="linkFieldType" eval="field.linkField.linkFieldType"/>
    <filter name="isOneToMany" eval="not linkFieldType.isMultivalued"/>
    <value name="name" eval="field.name"/>
    <value name="element" eval="linkFieldType.targetElement.name"/>
</list>

NotEmpty and Flatten in ognl​

2 methods have been attached to lists in OGNL statements in the mapping files:

<mapping>
    <!-- List#notEmpty returns true iff the list has no items -->
    <value name="hasDataElements" eval="component.dataElements.notEmpty"/>
    <!-- List#flatten will turn a list-of-lists List<List<T>> into a flat list List<T> -->
    <value name="allDataElements" eval="application.components.{ dataElements }.flatten"/>
</mapping>

String manipulation in OGNL​

Several properties have been attached to Strings to give them similar features to the StringTemplates:

<value name="fieldCapitalized" eval="field.name.firstToUpper"/>

Lambdas in OGNL​

OGNL does not support java 8 lambda syntax. And while OGNL did have its own version of lambdas, these did not work together with java 8 functional interfaces. This has been fixed for the Function, Predicate and Supplier functions.

So now, you can use the OGNL lambda syntax with, for example, Optional:

<value name="valueName" eval="someOptionalValue.map(:[ #this.name ]).orElse('no value')"/>

Changes and improvements​

OpenApi 3.0 support​

In this version of rest-expanders, we've introduced support for OpenAPI 3.0. Previously we only supported OpenAPI 2.0, which was Swagger originally. Due to some technical challenges, we were not able to support OpenAPI 3.0 up to this point. Now that these hurdles have been cleared, we've finished the implementation and made it the new default.

The new version of the specification is also integrated with a Swagger UI front-end as was the case with the previous specification. This version does rely on a new major version of the Swagger library, which brings many breaking changes. As a regular, custom code will be heavily impacted by this change. We do provide a model option to switch back to OpenAPI 2.

More information about updating can be found in the migration guide.

SecurityRight annotation​

The SecurityRight annotation is used to define rights for an endpoint. Previously these would only be added to expanded endpoints when use of account::DataAccess was enabled. This made it difficult to add more functionality to the annotation. Now this annotation is added to all expanded endpoints and anchors are available in the annotation to add additional property values.

This version of Expanders is quite impactful. It has a breaking change that could potentially affect several projects if they did not apply the NS theory correctly. This change is the removal of the detailed constructor in the projection POJO classes. We carefully considered the impact of this change and this topic describes the motivation for that and other changes.

Resources​

The expansion resources below provide Expanders 5.19.0.

Changes and improvements​

Removal of detailed constructor​

The projection POJO classes such as <dataElement>Details and <dataElement>Info have always had two constructors. The primary constructor is the constructor with no arguments. There was however also a secondary constructor with the database id and all fields of the projection as arguments. We have now removed the secondary constructor, as there is no technical reason for this to exist anymore.

The reason for removing the secondary constructor is that it violates Data Version Transparency, the first theorem of the NS theory. This theorem states that a changes in a data structure should not affect processing functions that consume them. This constructor, though generated by Expanders, would change whenever the relevant projection or fields in the DataElement change.

MyElementDetails myElement = new MyElementDetails(0L, "some name", "and a value");
...
CrudsResult<Void> result = createMethod(context.withParameter(new MyElementDetails(0L, "please don't", "do this")));

The legacy option legacy.projectionDetailedConstructor.reenable was added to re-enable the old behavior where the secondary constructor was generated. This option will expire on the 1st of September 2023.

Pruning deprecated imports​

Some expanders still contained some deprecated imports for code that was reworked or removed. These have now been removed, so the relevant classes could be deprecated or removed in nsx-runtime.

In version 5.16.0 of Expanders, we removed several expired legacy options. One such option was legacy.searchDataRefMethods.reenable. All related functionality has been removed from the expanders. As such, all related classes have now been removed from nsx-runtime in version 2023.0.0. The remaining related imports have been removed from all expanders for the following classes:

• net.democritus.support.ejb3.SearchDataRefToSearchDetailsMapper
• net.democritus.support.Paging
• net.democritus.sys.SearchDataRef

In 2020 the implementation of the FileUploadFeature, used to upload files through the generated UI, was improved and imports for two now deprecated classes in nsx-runtime were left behind in the Enterer classes. These imports have now been removed, though the classes are at this time still present in nsx-runtime:

• net.democritus.io.NioFileCopier
• net.democritus.io.SimpleUriFormat

Changes and improvements​

GetProjection​

The behaviour of the getProjection method in the element cruds has been changed to no longer return an empty object when the requested element could not be found. Instead it now returns a CrudsError to allow for better error handling.

• The option experimental.cruds.failOnProjectionNotFound has been removed, as it is now the default behaviour.
• A legacy option legacy.cruds.getProjection.emptyOnError was added to revert to old behaviour. (Expires 2023-08-01)

This change shouldn't affect your application if errors were handled previously. If the application assumes the returned CrudsResult to be successful you either:

• Handle the error explicitly, by checking CrudsResult.isError
• Or by providing a default value, for example with CrudsResult.getValueOrElseGet(Projection::new)

Frontend support for Null fieldoperator​

Finders specifying the Null fieldoperator will now be shown as a checkbox in the knockout UI to enable or disable the null check when searching. The label for this checkbox can be customized by overriding the $component.$dataElement.$field.null translation.