FreshMarker – compile it! (3)

“When virtue is lost, benevolence appears, when benevolence is lost right conduct appears, when right conduct is lost, expedience appears. Expediency is the mere shadow of right and truth; it is the beginning of disorder.”

Lao Tsu

In the second article on the FreshMarker Compiler, we looked at the implementation of a pretty printer for Template instances. This implementation serves as the basis for the FreshMarker Compiler, which generates Java code from a Template instance. Before we tackle the Compiler, we need to look at a few more hurdles and details.

A fundamental limitation of the solution approach used is that a large part of the original information from the template source is no longer available in the Template instance. This is not entirely correct, as some constructs rely on the CongoCC parse tree for the error messages, but using this information would be time-consuming.

For various operations, such as comparisons, disjunction, addition etc., only the type within the template is known and not the original notation as ASCII or Unicode. The pretty printer is therefore unable to decide whether an & or a  operator was used in the template. However, we have to choose one of the two variants for the output. At this point, we make a virtue of necessity and decide to enable the pretty printer to convert operators between both notations. Depending on the settings, it writes all operations in ASCII or Unicode notation.

private static final Map<TokenType, String> UNICODE_OPERATORS = Map.ofEntries(
  Map.entry(TokenType.OR, "∨"),
  Map.entry(TokenType.OR2, "||"),
  Map.entry(TokenType.AND, "∧")),
  Map.entry(TokenType.AND2, "&&"),
  // ...
);

private static final Map<TokenType, String> ASCII_OPERATORS = Map.ofEntries(
 Map.entry(TokenType.OR, "|"),
  Map.entry(TokenType.OR2, "||"),
  Map.entry(TokenType.AND, "&")),
  Map.entry(TokenType.AND2, "&&"),
  // ...
  // ...
);

private final Map<TokenType, String> operators;

public ExpressionPrinter(boolean unicode) {
  operators = unicode ? UNICODE_OPERATORS : ASCII_OPERATORS;
}

private String visit(TemplateObject left, TemplateObject right, TokenType operatorType) {
  return left.accept(this) + " " + operators.get(operatorType) + " " + right.accept(this);
}

The display for the operator is selected from a map operator. The corresponding map UNICODE_OPERATORS or ASCII_OPERATORS is selected in the constructor via the boolean flag unicode.

The hurdles that arise with the switch directive are far more difficult. Here too, information from the template source is no longer accessible in the Template. In this case, it is particularly the information about whether it was a case or on statement.

<#switch color>
<#on 'RED', 'Crimson', 'DarkRed'>#FF0000
<#on 'GREEN'>#00FF00
<#on 'BLUE', 'Navy'>#0000FF
<#default>transparent
</#switch>
<#switch color>
<#case 'RED'>#FF0000
<#case 'Crimson'>#FF0000
<#case 'DarkRed'>#FF0000
<#case 'GREEN'>#00FF00
<#case 'BLUE'>#0000FF
<#case 'Navy'>#0000FF
<#default>transparent
</#switch>

Both types are represented by the same internal data structure. The more compact form of the switch/ondirective is converted internally into the form of the switch/casedirective. When it comes to the output, the question arises as to whether we want the switch/casedirective or the switch/on directive to be output. Unfortunately, it is not as easy here as when deciding between ASCII and Unicode, because the information that 'RED''Crimson' and 'DarkRed' generate the same output is no longer directly available.

@Override
public String visit(SwitchFragment fragment, TemplateObject switchExpression, List<ConditionalFragment> fragments, Fragment endFragment) {
  StringBuilder builder = new StringBuilder();
  builder.append("<#switch ").append(switchExpression.accept(expressionPrinter)).append(">");
  builder.append(fragments.stream().map(f -> f.accept(switchPrinter)).collect(Collectors.joining()));
  if (endFragment != null) {
    builder.append("<#default>").append(endFragment.accept(this)).append("\n");
  }
  builder.append("</#switch>");
  return builder.toString();
}

The current implementation for the switch directive runs through all ConditionalFragment instances and generates a separate statement in each case. Alternatively, consecutive ConditionalFragment instances can be checked to see whether they have an identical body.

@Override
public String visit(SwitchFragment fragment, TemplateObject switchExpression, List<ConditionalFragment> fragments, Fragment endFragment) {
  StringBuilder builder = new StringBuilder().append("<#switch ").append(switchExpression.accept(expressionPrinter)).append(">");
  if (switchCase) {
    SwitchCasePrinter visitor = new SwitchCasePrinter(this, expressionPrinter);
    builder.append(fragments.stream().map(f -> f.accept(visitor)).collect(joining()));
  } else {
    SwitchCollectVisitor visitor = new SwitchCollectVisitor(unicode, this);
    fragments.forEach(f -> f.accept(visitor));
    builder.append(visitor);
  }
  if (endFragment != null) {
    builder.append("<#default>").append(endFragment.accept(this));
  }
  return builder.append("</#switch>").toString();
}

This implementation uses a SwitchCollectVisitor, which collects all statements of the switch directive and checks whether their bodies match for consecutive statements. The only shortcoming here is that not all directives have a corresponding equals method for now. This means that not all identical bodies are recognized as equal.

For the Java code generation in the planned FreshMarker Compiler, there is a further difficulty. Unlike Java, the switch directive does not only allow constants to be used in the case- and on-statements. However, a solution is also emerging here, as the switch and if directives differ only slightly internally.

<#switch color>
<#case 'RED'>#FF0000
<#case 'Crimson'>#FF0000
<#case 'DarkRed'>#FF0000
<#case 'GREEN'>#00FF00
<#case 'BLUE'>#0000FF
<#case 'Navy'>#0000FF
<#case value>'${color}'
<#default>transparent
</#switch>
<#if color == 'RED>
#FF0000
<#elseif color == 'Crimson'>
#FF0000
<#elseif color == 'DarkRed'>
#FF0000
<#elseif color == 'GREEN'>
#00FF00
<#elseif color == 'BLUE'>
#0000FF
<#elseif color == 'Navy'>
#0000FF
<#elseif color == value>
'${color}'
<#else>
transparent
</#if>

case statement can be replaced by an if or elseif statement and the default statement by the else statement. The solution for the compiler is to transform it into an if-else cascade if a switch statement is not possible. However, this will be the task of another blog post.

Leave a Comment