Skip to main content

Model Manipulator

danger

This tool has been deprecated in favor of Model Transmutations! It does not receive any more updates and is not compatible with current model infrastructure.

Manipulator for scripted reading and modifications of NS models.

Latest release: 2.2.1

Download​

The easiest way to grab a manipulator is to download it from Nexus, or via Maven:

mvn dependency:copy "-Dartifact=net.democritus:model-manipulator:2.2.1:jar:jar-with-dependencies" "-DoutputDirectory=." "-Dmdep.stripClassifier=true" "-Dmdep.stripVersion=true"

Migration Usage​

Apply all migrations in directory migration-scripts, on project in C:\ws\root\of\my\project, starting from version 2020.5.0, without using or creating conf\migration.xml markers.

java -jar model-manipulator.jar migration --directory C:\ws\root\of\my\project --runAll migration-scripts --from 2020.5.0 --noMarkers

Force execution of a single migration script.

java -jar model-manipulator.jar migration --directory C:\ws\root\of\my\project --runSingle 2020.5.0-1_single-script.groovy --force

Execute all migration scripts in the specified directory ignoring all previously executed ones.

java -jar model-manipulator.jar migration --directory C:\ws\my\other\project --runAll migrations --latest

Migration Markers​

caution

Migration markers are NOT created when using runFrom.

Running a migration (whether a single file, or a directory of files) creates or updates a marker file in {root}/conf/migration.xml.

<migrations>
  <migration>
    <version>2021.0.0-01-make-changes</version>
    <timestamp>2021-01-01T00:00:30Z</timestamp>
  </migration>
  <migration>
    <version>2021.0.0-02-make-other-changes</version>
    <timestamp>2021-05-27T18:20:30Z</timestamp>
  </migration>
</migrations>

This file lists all migrations that were ever applied on the project and when.

Both --runSingle and --runAll supports additional options:

  • --latest (default behavior)
    • Only versions after the latest one in the marker.xml will be executed.
  • --all
    • All non-applied migrations are executed.
  • --force
    • Forcibly re-run all migrations. Options --latest and --all are still followed.

For example, in migration.xml you have versions 1.0.0, 2.0.0, 2.1.0. A new release contains migration versions 1.0.0, 2.0.0, 2.0.1, 2.1.1, 2.1.2.

With --latest option, only 2.1.1 and 2.1.2 are executed (because latest one was 2.1.0). With --all option, also version 2.0.1 is executed (because it wasn't applied yet, but is older than the latest 2.1.0). With --force and --latest option, all versions are executed except for 2.0.1. With --force and --all option, all versions are executed.

Writing Migration Scripts​

Each migration script must be named <SEMVER>_description.groovy, where <SEMVER> must be a valid SemVer version. E.g. 2020.7.0-2_my-changes.groovy.

Scripts are sorted using standard SemVer comparison.

The script itself is a plain groovy script, but has access to a pre-configured ModelManipulator instance and classes within model manipulator.

In order to have intellisense/autocompletion, create a minimal maven or gradle project and add model-manipulator as a provided dependency.

import net.democritus.elements.ApplicationInstanceTree
import net.democritus.manipulator.api.ManipulationTasks
import net.democritus.manipulator.api.ModelManipulator
import net.democritus.sys.DataRef

// facade to simplify common steps, such as adding options
import static net.democritus.manipulator.api.ManipulationTasks.setOption

// obtain a pre-configured ModelManipulator instance for the project we are currently migrating
def m = ModelManipulator.getInstance()

// Find, read, modify, and write all model files of MyElement within the project
// m.updateAll(MyElementTree.class, (MyElementTree myElement) -> { /* do processing */ })

m.updateAll(ApplicationInstanceTree.class, applicationInstance -> {
  // you can directly modify the tree instance
  applicationInstance.setExpanderVersion(DataRef.withName("PROVIDED"))
  // or use helper methods
  setOption(applicationInstance, 'db.global.noSchemas')
})

Method ModelManipulator#updateAll finds all files in the project of the given type and applies the requested change. If you want to apply a change only to some instances, wrap a change in the consumer block with an if statement.

note

Step updateAll reads and writes all files of the requested type. This may result in XML changes even if no content changes were applied. We recommend running normalize task (see further) and committing before running the migrations.

CLI Usage​

Optional. Define a function/alias.

# pwsh
function manipulator() { java -jar model-manipulator.jar $args }
# bash
alias manipulator='java -jar model-manipulator.jar'

All commands understand the --help and -h flags to give you the most up-to-date information.

manipulator --help
manipulator set-option --help

To specify on which files the operations will be performed, you must specify either --file (-f), or --directory (-d).

Specifying a directory (via --directory) excludes the following subdirectories to reduce false matches: target, ext, overlay, .<dir-starting-with-dot>.

Furthermore, when reading a directory, all XML files within, excluding the ones mentioned above, will be attempted to be read with the following effect:

  • related root tag name, related content -- processed as expected
  • related root tag name, but unrelated content -- depending on the content, reading will likely issue warnings, but not necessarily errors (e.g. .idea/codeStyles/Project.xml)
  • unrelated root tag name -- file is ignored (e.g. struts, or project)
  • broken XML file -- error message + file is ignored

To further limit the applicability, you can add flag --elementType (-t) to specify an element such as applicationInstance (but see supported limitations).

Normalize XMLs​

Loads a file and saves it without functional changes. Useful to maintain cleaner XML form. Note that any change in the model (add-option, etc.) will normalize the XML as well.

# normalize specific application instance
manipulator normalize --file applications/primeTest/model/instances/mvn.xml
# normalize all component files
manipulator normalize --directory components --elementType component

Options Manipulation​

At the moment, only modification of the following elements is supported. Other elements are ignored. This does not apply to other manipulation (e.g. command).

  • applicationExpansionSettings
  • applicationInstance
  • component (not all-in-one component files)

Set Option​

Sets an option. If an option with the specified name (optionType) already exists, its value will be updated.

manipulator set-option --name struts.version --value 2.3 --elementType applicationInstance --directory applications/

Remove Option​

Remove all options with the given name. When an option with the same optionType was declared multiple times (e.g. duplicated), both of them will be removed.

manipulator remove-option --name noViewLayer --file components/useCases/model/useCases.xml

OGNL Query and Command​

You can query and modify files using OGNL expressions.

The provided expression will be applied to each matching file and the result will be printed to standard output.

You can use it, model, or <elementType> (e.g. dataElement for -t dataElement) to refer to the root object of the currently processed file.

# search for help
manipulator query --help

# find all dataElements in current directory that don't have a `name` field, and have a `nameNotWanted` option
manipulator query -d components -t dataElement -q "dataElement.fields.{ ? name == 'name' }.empty and dataElement.dataOptions.{ ? dataOptionType.name == 'nameNotWanted' }.size > 0 ? dataElement.name : 'nope'" | Select-String -SimpleMatch nope -NotMatch

# modify modelRepository and customRepository in all component files
manipulator command -d components -t component -q "model.(customRepository.name='mySourceBase',modelRepository.name='mySourceBase')"
# and in applications
manipulator command -d applications -t application -q "model.(customRepository.name='mySourceBase',modelRepository.name='mySourceBase')"