Skip to main content

Strict Mode

expanders-assert1.6.0+

Strict mode applies extra rules to the test model of each test. These rules catch definitions that work, but are incomplete: options without an OptionType, or tags that are not documented. Without strict mode, these issues go unnoticed because the framework fills in the gaps.

Enabling Strict Mode​

Strict mode is a JUnit configuration parameter. Enable it for all tests of a module in src/test/resources/junit-platform.properties:

src/test/resources/junit-platform.properties
expanders.assert.strict=true

JUnit also reads configuration parameters from systemSystemA set of interconnected components that work together to achieve a specific purpose or function. properties, so you can enable it for a single run:

mvn test -Dexpanders.assert.strict=true

The test run logs whether strict mode is enabled, e.g. Strict mode is enabled (expanders.assert.strict).

Warn Mode​

Set the property to warn to run the rules without failing tests:

src/test/resources/junit-platform.properties
expanders.assert.strict=warn

Every error is lowered to a warning. The violations still show up in the report at the end of the test run. Use this to see what strict mode would flag before you enable it.

Valid values

Strict mode properties only accept true, false or warn. Any other value fails the test setup.

Rules​

RuleChecksRuns
optionTypesDefinedEach option in the model has an OptionType.After the model is built.
tagTypeDefinedEach tag used by the test is documented.When the test expands.
tagTypeDescribedEach Profile and Expander API tag type has a description.After the model is built.

optionTypesDefined​

Each option in the test model needs an OptionType defined in an elements::OptionType DataResource. Without one, the framework generates an OptionType, so the test passes either way. This rule makes the missing definition an error.

The rule checks options on elements built by the test and on DataResources of the module under test. DataResources of dependencies are skipped.

Rule (expanders.assert.rule.optionTypesDefined) violated:
- option 'sample.option' on DataElement testComp::City has no OptionType
Fix: define an OptionType dataResource for each option key.

tagTypeDefined​

Each tag used by the test needs to be documented with an ElementTagType. The used tags are:

  • the tags of the tested ExpanderExpanderA unit of generation that combines a data tree instance (functional M1 model) with a template to generate an artifact., Feature or ExpansionStep;
  • the tags passed to TestExpansion.withTags(...).

The rule runs when the test prepares the ExpansionContext, so tests that do not expand are not checked.

TagResult
ElementTagType with a scope (Profile, Expander API, or Internal from this module)OK
Defined by this module, through an ElementTagType without scope or a deriving ElementTagRuleOK
Defined by another module, through an ElementTagType without scope or a deriving ElementTagRuleWarning
ElementTagType with scope Internal from another moduleError
Not defined by any moduleError

A tag without an ElementTagType, or without a scope, is internal to the module that defines it. Using such a tag from another module only gives a warning: the defining module has to document it, not your module. Using a tag that another module explicitly marks as Internal fails the test.

tagTypeDescribed​

Each ElementTagType of the module under test with scope Profile or Expander API needs a description. Other modules and profiles rely on these tags, so they must be documented.

Rule (expanders.assert.rule.tagTypeDescribed) violated:
- tag '#sample.api' with scope 'Expander API' has no description
Fix: add a description to the ElementTagType.
Metamodel version

tagTypeDefined and tagTypeDescribed need a metamodel version where ElementTagType has a scope. With an older version, both rules are skipped.

Configuring Rules​

Each rule can be enabled, disabled or lowered to warnings with its own property: expanders.assert.rule.{ruleName}=(true|false|warn). The rule property takes precedence over expanders.assert.strict.

Enable strict mode, but skip the tag rules:

src/test/resources/junit-platform.properties
expanders.assert.strict=true
expanders.assert.rule.tagTypeDefined=false
expanders.assert.rule.tagTypeDescribed=false

Enable only one rule, e.g. to adopt strict mode one rule at a time:

src/test/resources/junit-platform.properties
expanders.assert.rule.optionTypesDefined=true

Enforce strict mode, but only warn about undocumented tags:

src/test/resources/junit-platform.properties
expanders.assert.strict=true
expanders.assert.rule.tagTypeDefined=warn

Violations​

A violation is either an error or a warning.

  • An error fails the test with a StrictModeViolation. The message lists the violations per rule, with a hint on how to fix them.
  • A warning does not fail the test. In warn mode, every violation of a rule is a warning.

Report​

At the end of the test run, strict mode prints a report with every distinct violation, warnings included:

====================================================================================================
STRICT MODE: 1 error(s) and 1 warning(s) to fix
====================================================================================================
Rule (expanders.assert.rule.tagTypeDefined) violated:
- [ERROR] tag '#sample.feature' used by Expander SampleExpander is not documented
- [WARNING] tag '#cruds.bulk.delete' used by withTags is not documented, but defined by com.example:other-expanders::1.0.0, which should document it
Fix: define an ElementTagType with scope 'Internal', 'Expander API' or 'Profile', or an ElementTagRule that derives the tag.
The tag may belong to another module: let that module document it, do not define its type here.
====================================================================================================

The report is only printed if there are violations. To turn it off, deactivate its JUnit listener:

src/test/resources/junit-platform.properties
junit.platform.execution.listeners.deactivate=net.democritus.expander.junit.StrictModeReportListener