The End of Loose Tag End

by

While designing a new feature, I once again stumbled upon a feature of the template language that I haven’t liked for a long time. The syntax of the directives is modeled after HTML elements, not XML elements. As long as you don’t want to change the syntax of the template language, this isn’t a problem. But if you have a great idea for a new feature, you’re in for a world of trouble.

The directives in the template language often consist of a start tag and an end tag, with content between them that needs to be processed in a specific way.

<#list values as value with looper>
${looper?index}) ${value}
</#list>

This list directive has a start tag <#list> and an end tag </#list>.

Other directives consist only of a single tag and can be written as a start tag or an empty tag.

<#var test/>
<#set test='zwei'/>
<#var test>
<#set test='zwei'>

Both forms are valid in the template language, and there is no difference between using one or the other. This also applies to the Nested directive used within a Macro directive.

<#macro copyright from author="Jens Kaiser">
/*
* Copyright © ${from}-${.now?string('yyyy')} ${author}
*
* <#nested>
*/
</#macro>
<@copyright from="1968">All Rights reserved.</@copyright>
<@copyright from="1968">Licensed under the Apache License, Version 2.0</@copyright>

In this example, the copyright User directive generates two different headers. This depends on the content of the User directive, which is inserted into the macro at the position of the Nested directive. In this example, it is included in the form of a start tag for instructional purposes.

The idea behind this post stems from the fact that the user directive above always requires content, and that content is identical in perhaps 90% of calls. Of course, one could simply write two user directives, but where’s the challenge in that?

Why not provide default content for the nested directive?

<#macro copyright from author="Jens Kaiser">
/*
* Copyright © ${from}-${.now?string('yyyy')} ${author}
*
* <#nested>Licensed under the Apache License, Version 2.0</#nested>
*/
</#macro>
<@copyright from="1968">All Rights reserved.</@copyright>
<@copyright from="1968"/>

In this modified example, the second User directive does not require any content because it was specified as default content in the Nested directive. And this is where the problem for the parser begins. How can we distinguish whether the Nested directive ends immediately and the following text belongs to the Macro directive, or whether it still belongs to the Nested directive?

There are two solutions to this problem. The first involves tedious work on the Freshmarker grammar to continue allowing Nested directives as single start tags, or the second: we prohibit this form of Loose Tag End directives.

The name Loose Tag End directive derives from its definition in the grammar.

Nested #NestedInstruction :
    (<FTL_DIRECTIVE_OPEN1>|<FTL_DIRECTIVE_OPEN2>)
    <NESTED>
    [<BLANK> PositionalArgsList]
    LooseTagEnd
;

LooseTagEnd :
    <CLOSE_TAG> | <CLOSE_EMPTY_TAG>
;

These two rules ensure that a nested directive ends with either a CLOSE_TAG (>) or a CLOSE_EMPTY_TAG (/>). What we actually want is the following rule:

Nested #NestedInstruction :
    (<FTL_DIRECTIVE_OPEN1>|<FTL_DIRECTIVE_OPEN2>)
    <NESTED>
    [<BLANK> PositionalArgsList]
    ( <CLOSE_EMPTY_TAG>
    | Block DirectiveEnd("nested")
    )  
;

A nested directive with no content must end with a CLOSE_EMPTY_TAG, or a CLOSE_EMPTY_TAG must be followed by a Block and then a matching end tag. This is checked using DirectiveEnd("nested").

Unfortunately, such a change constitutes a breaking change for the template language and will therefore not be implemented in FreshMarker 2.7.0. However, we can use the capabilities of CongoCC to provide an advance notice of this change in the next version.

LooseTagEnd :
    <CLOSE_TAG> {
      LOGGER.warn("loose end syntax is deprecated use empty tag syntax instead: " + lastConsumedToken.getLocation());
    }
    |
    <CLOSE_EMPTY_TAG>
;

We have inserted a code block after the CLOSE_TAG in the LooseTagEnd rule. This means that our rule executes this code whenever a nested directive is detected that ends with a CLOSE_TAG. This way, users of version 2.7.0 will know that this syntax will no longer be supported in the future and that they should switch to the cleaner version using the empty tag syntax.

To display this warning, we naturally need a logger in our parser. Here, we use the ingenious INJECT feature from CongoCC.

INJECT PARSER_CLASS :
    import java.io.*;
    import java.util.*;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
{
  private static final Logger LOGGER = LoggerFactory.getLogger(FreshMarkerParser.class);

  private boolean looseEndTagMode;

  public void setLooseEndTagMode(boolean looseEndTagMode) {
    this.looseEndTagMode = looseEndTagMode;
  }

  public boolean isLooseEndTagMode() {
    return looseEndTagMode;
  }
}

This INJECT statement inserts the specified code into the parser class for the template language. Here, we define our logger as a static attribute of the class, and we also add a boolean flag called looseEndTagMode.

With this flag, we can already switch to a stricter syntax so that users can test how the template engine will behave in the future.

LooseTagEnd :
    <CLOSE_TAG> {
      if (!isLooseEndTagMode()) {
        throw new ParseException("Expecting empty tag");
      }
      LOGGER.warn("loose end syntax is deprecated use empty tag syntax instead: " + lastConsumedToken.getLocation());
    }
    |
    <CLOSE_EMPTY_TAG>
;

The code block now checks whether the flag is set and, depending on the result, either throws a ParseException or issues a warning. To ensure the flag is set correctly, we retrieve the current value from your new SystemFeature named LOOSE_END_MODE and set it when the parser is called.

FreshMarkerParser parser = new FreshMarkerParser(name, source.getContent());
parser.setLooseEndTagMode(featureSet.isEnabled(SystemFeature.LOOSE_END_MODE));

The feature must be defined as a new constant in the SystemFeature enum.

/**
 * Feature that enables LOOSE_END_MODE in the grammar.
 * When deactivated loose end directives are recognized as a parse error.
 * <p>
 * The default value is 'true', meaning that loose end directives are allowed.
 * <p>
 * The feature is enabled by default.
 */
LOOSE_END_MODE;

@Override
public boolean isEnabledByDefault() {
  return this == LOOSE_END_MODE;
}

The implementation of isEnabledByDefault shows that LOOSE_END_MODE is the only system feature that is enabled by default. This means that the old loose tag end syntax is enabled without any further configuration by the user, and FreshMarker 2.7.0 will continue to behave like its predecessor.

With version 3.0.0, FreshMarker’s behavior will change, and the old loose tag end syntax can only be used if the user enables LOOSE_END_MODE. The only change in the implementation will occur in the isEnabledByDefault method. Another triumph for the FreshMarker feature implementation.

Leave a Comment