Expander Features

Expander features are a flexible mechanism to introduce additional additive behavior to expanders without impacting the expander’s definition itself. They further improve traceability, debugging, and organization of expanders.

Artifacts and Naming

  • <feature.packageName>.<feature.name>.xml
    • FeatureComposite representation including all ExpanderFeatureComposites
  • <feature.packageName>.<featureName>.<expanderName>.stg
    • Template for the ExpanderFeature
  • <feature.packageName>.<featureName>.<expanderName>Mapping.xml
    • Mapping for the ExpanderFeature
  • <feature.packageName>.<featureName>.<expanderName>.test.xml
    • mapping and template tests in XML
  • <feature.packageName>.<featureName><expanderName>Test.java
    • Java test for the ExpanderFeature, possibly itself expanded from the *.test.xml

New Style (Syntax-independent)

With Expander Mapping and the new Expander style, it is no longer possible to use insertions as everything is driven from the Mapping and the main base() template.

We continue to use STG, however it makes sense to either repurpose existing anchors, or introduce new ones, which would be used as anchor points for feature insertion. Furthermore, with first-class declaration of features, we can improve the traceability.

BeanExpander.stg (main Expander stg file)

package <class.packageName>;

@anchor:imports

@anchor:annotations
public class <class.className>Bean @anchor:interfaces(prefix : "implements ", separator : ", ")@{

  @anchor:variables

  @anchor:methods
}

GlobalCounter.xml (feature XML declaration)

<feature name="GlobalCounter">
  <condition eval="useGlobalIdCounter"/>
  <expanderFeatures>
    <expanderFeature>
      <expander name="BeanExpander" packageName="net.democritus.expander.ejb3.dataElement"/>
    </expanderFeature>
  </expanderFeatures>
</feature>

GlobalCounter.BeanExpander.stg (feature STG for a specific Expander)

base() ::= <<
@hook:imports:start
import net.democritus.sys.IdCounter<ejbType>;
@hook:imports:end

@hook:variables:start
@EJB private IdCounter<ejbType> idCounter<ejbType>;
@hook:variables:end
>>

Note about AOP

From a theoretical perspective, these features are not dissimilar to some concepts from Aspect Oriented Programming (AOP) or AspectJ and provide similar functionality.

@joinpoint:start
@joinpoint:end

@aspect:(aspect options):start
@pointcut:start
advice
@pointcut:end
@aspect:end

Anchor-based Feature Weaving

This section illustrates how code within features are weaved into anchors defined in the base code.

1. Raw STG

AgentExpander.stg

base() ::= <<
package <class.packageName>;

@anchor:imports

public class <class.className>Agent {
  @anchor:variables

  @anchor:methods
}
>>

CsvImport.xml

<feature name="CsvImport">
  <condition>dataElement.getOption('includeCsvImport').defined</condition>
  <expanderFeatures>
    <expanderFeature name="AgentExpander">
  </expanderFeatures>
</feature>

CsvImport.AgentExpander.stg

base() ::= <<
@hook:imports:start
import net.democritus.upload.ImportFile;
import net.democritus.upload.ImportResult;
@hook:imports:end

@hook:methods(group : "Import/Export"):start
public ImportResult importFile(ImportFile importFile) {
    return <class.varName>Proxy.importFile(createParameter(importFile));
}
@hook:methods:end

Commands.xml

<feature name="Commands">
  <condition>!dataElement.dataCommands.empty</condition>
  <expanderFeatures>
    <expanderFeature name="AgentExpander">
  </expanderFeatures>
</feature>

Commands.AgentExpander.stg

@hook:methods:(group : "Handle commands"):start
public \<T extends ICommand\> CommandResult perform(T command) {
  return <class.varName>Proxy.perform(createParameter(command));
}
@hook:methods:end

2. Feature Extraction

Before we can render the template, we must extract the anchors, filter them, and insert them at the appropriate locations.

3. Feature Filtering

The if conditions in feature’s options are processed using the model mapping.

<!-- AgentExpanderMapping.xml -->
<mapping>
  <value name="class" eval="classBuilder.on(dataElement)"/>
  <value name="isCsvImportEnabled" eval="dataElement.getOption('isCsvImportEnabled').defined" />
  <value name="hasCommands" eval="!dataElement.dataCommands.empty" />
</mapping>

which could result into e.g.:

Map<String, Object> model = {
  class : ClassDescriptor{sand.box.Person},
  isCsvImportEnabled : false,
  hasCommands : true
};

All Features that fail their if condition including all their segments are ignored.

Note. Unclear whether filtering on Segments would yield any benefit.

4. Feature Segment Processing

Segments from Features that passed through the filtering are inserted into their declared anchors. The name of the segment matches the name of the desired anchor.

Furthermore, it seems useful to utilize both the options of the Segments and the Anchors.

For example all content of <@methods:(group : "Handle commands"):start> segments would be grouped before inserting into the anchor. For each group, a new heading would be added.

Similarly, for Anchors an extra formatting information is useful.

public class <class.className>Bean <@anchor:interfaces:(prefix : implements, separator : ", "):inline> {

5. Feature Insertion

Once all Segments are processed, they are inserted back into the STG (in-memory, although a temporary file for debugging is also an option).

Example Assembled STG:

base() ::= <<
public class <class.className>Agent {

  /*========= Handle commands ========*/

  public \<T extends ICommand\> CommandResult perform(T command) {
    return <class.varName>Proxy.perform(createParameter(command));
  }

}
>>

6. Traceable Insertion

As we now control the insertion mechanism, we can insert additional information about the traceability.

Trace Info Only

base() ::= <<
// <@anchor:imports:start>
// trace:(if : isCsvImportEnabled = false)
// <@anchor:imports:end>

public class <class.className>Agent {

  // <@anchor:methods:start>
  // <@trace:(if : isCsvImportEnabled = false):inline>
  /*========= Handle commands ========*/

  // <@trace:(if : hasCommands = true):start>
  public \<T extends ICommand\> CommandResult perform(T command) {
    return <class.varName>Proxy.perform(createParameter(command));
  }
  // <@trace:end>

  // <@anchor:methods:end>

}
>>

Full Trace

Because we know the comment syntax for the target language, we can actually insert all code that could be potentially inserted. The STG syntax within the comments must be escaped, as it would lead to errors when trying to render the template.

base() ::= <<
// anchor:imports:start
// trace:(if : isCsvImportEnabled = false):start
//import net.democritus.upload.ImportFile;
//import net.democritus.upload.ImportResult;
// trace:end
// anchor:imports:end

public class <class.className>Agent {

  // anchor:methods:start
  // trace:(if : isCsvImportEnabled = false):start
  //public ImportResult importFile(ImportFile importFile) {
  //    return \<class.varName\>Proxy.importFile(createParameter(importFile));
  //}
  // trace:end

  /*========= Handle commands ========*/

  // trace:(if : hasCommands = true):start
  public \<T extends ICommand\> CommandResult perform(T command) {
    return <class.varName>Proxy.perform(createParameter(command));
  }
  // trace:end

  // anchor:methods:end

}
>>

7. STG Rendering

Finally, the assembled STG is rendered normally. At this point in time it seems unnecessary to split the Mapping for each individual Feature. They can use all data provided from the Mapping.

In-Method Cross-Cutting concerns

To Be Explored.

AST-based Feature Weaving (Concept)

TBD in the future.

The basic idea of AST-based weaving is to declare syntactically valid source within each Feature, render the desired ones, and merge them according to their AST position. That is, imports with imports, methods with methods, etc.

Theoretically optimal, however still would need to be merged in some way with the above, as there are additional concepts that may not be representable with the syntax in all languages. (E.g. grouping of methods.)

package sand.box;

@Stateless()
public class PersonBean {
}

<@astConcern:(name : localEjb, if : localEjb):start>
import javax.ejb.Local;

@Local(PersonLocal.class)
public class PersonBean implements PersonLocal {

}
<@astConcern:end>

<@astConcern:(name : remoteEjb, if : remoteEjb):start>
import javax.ejb.Remote;

@Remote(PersonRemote.class)
public class PersonBean implements PersonRemote {
  @Deprecated
  @PersistenceContext(unitName="sandboxApp_sandboxComp")
  private EntityManager entityManager;
}
<@astConcern:end>
// code produced from the AST
package sand.box;

import javax.ejb.Local;
import javax.ejb.Remote;

@Stateless()
@Local(PersonLocal.class)
@Remote(PersonRemote.class)
public class PersonBean implements PersonLocal, PersonRemote {
  @Deprecated
  @PersistenceContext(unitName="sandboxApp_sandboxComp")
  private EntityManager entityManager;
}

Old Style of Expander Features

Properties of newly introduced aspects are organized by their target location.

Example: Adding CSV import requires adding imports and methods. Both parts are separated in the StringTemplate (STG) and must be guarded with the same condition twice (<if(isCsvImportEnabled)>).

Prone to forget to update/fix other properties of the introduced aspect. Hard to reason about, as the parts are semi-randomly spread out throughout the template. Cannot be traced, as the conditional trace <if(isCsvImportEnabled)> is lost – it is only available internally within the STG parser/renderer.

AgentExpander.stg

package <classNamer.packageName>;

// <expanderComment>

import java.util.List;

// anchor:imports:start
// anchor:imports:end

<if(isCsvImportEnabled)>
import net.democritus.upload.ImportFile;
import net.democritus.upload.ImportResult;
<endif>

public class <classNamer.agent.className> {

  /*========== Import/Export =========*/

<if(isCsvImportEnabled)>
  public ImportResult importFile(ImportFile importFile) {
      return <class.varName>Proxy.importFile(createParameter(importFile));
  }
<endif>  
}

Additional code can be introduced via insertions from Java.

AgentExpander.java

public class AgentExpander {
  ...

  public Collection<Insertion> getInsertions() {
    return collect(
      Insertion.create(insertImports(), anchor("imports"), "")
    );
  }

  public String insertImports() {
    List<String> lines = new ArrayList<String>();

    lines.add("import X.Y.Z");

    return StringUtil.joinLines(lines);
  }
}