“Some people, when confronted with a problem, think ‘I know, I’ll use regular expressions.’ Now they have two problems.”
Jamie Zawinski
Every now and then, I look into the capabilities of other template engines for new features. If something offers added value and fits within FreshMarker’s concept, then it gets adapted. Sometimes the result is something completely different, and sometimes it’s very similar to the original. Since FreshMarker 2.3.0, the built-ins find, matches, and split have been available. Why not add a few new built-ins using regular expressions?
I recently discovered the functions keep_after, keep_after_last, keep_before, keep_before_last, matches (a different one than ours), remove_beginning, and remove_ending. They all operate on a string and take a regular expression as a parameter. The idea was: how can these built-in functions enhance FreshMarker?
The keep_after built-in returns the suffix of a string following the first occurrence of the search pattern. For the text " Chico Harpo Groucho Gummo Zeppo " and the pattern "G\w+", this built-in returns the result " Groucho Gummo Zeppo ". The keep_after_last built-in returns the suffix of a string following the last occurrence of the search pattern. This built-in returns the result " Gummo Zeppo ".
The keep_before built-in returns the prefix of a string preceding the first occurrence of the search pattern. This built-in returns the result " Chico Harpo ". The keep_before_last built-in returns the prefix of a string preceding the last occurrence of the search pattern. It returns the result " Chico Harpo Groucho ".
The built-in remove_beginning is a special case of keep_after and removes the prefix specified by the pattern. For the pattern " C\w+", the built-in returns " Harpo Groucho Gummo Zeppo ". It can also be expressed using remove_beginning with the pattern "^ C\w+". The built-in remove_ending, on the other hand, is a special case of keep_before and removes the suffix specified by the pattern. For the pattern "Z\w+ ", the built-in returns " Chico Harpo Groucho Gummo ". It can also be expressed using remove_ending with the pattern "Z\w+ $".
The matches built-in is a bit more complex, so we’ll take a closer look at it separately later. As a software developer, it feels a bit counterintuitive to implement four very similar built-ins (keep_after, keep_after_last, keep_before, keep_before_last) instead of a single parameterizable one. But not all users who write FreshMarker templates are software developers, so these built-ins serve a purpose.
Using FreshMarker’s extension mechanism and Java’s Pattern/Matcher classes, the first four built-ins can be implemented quickly. Since the four built-ins are all quite similar, only the implementation for keep_after and keep_after_last is described here.
public static TemplateString keepAfter(TemplateString text, List<TemplateObject> parameters) {
String regex = getRegex(parameters);
if(regex.isEmpty()) {
return TemplateString.EMPTY;
}
Matcher matcher = getMatcher(text.getValue(), regex);
return getSuffixOrAll(text, matcher.find() ? matcher.end() : -1);
}
public static TemplateString keepAfterLast(TemplateString text, List<TemplateObject> parameters) {
String regex = getRegex(parameters);
if(regex.isEmpty()) {
return TemplateString.EMPTY;
}
Matcher matcher = getMatcher(text.getValue(), regex);
int lastEnd = -1;
while (matcher.find()) {
lastEnd = matcher.end();
}
return getSuffixOrAll(text, lastEnd);
}
private static String getRegex(List<TemplateObject> parameters) {
BuiltInHelper.checkParametersLength(parameters, 1);
return parameters.getFirst().evaluate(null, TemplateString.class).getValue();
}
private static Matcher getMatcher(String text, String regex) {
return Pattern.compile(regex).matcher(text);
}
private static TemplateString getSuffixOrAll(TemplateString text, int lastEnd) {
return lastEnd < 0 ? text : new TemplateString(text.getValue().substring(lastEnd));
}
The built-ins are implemented in the keepAfter and keepAfterLast methods. In both, the first parameter is first evaluated using the helper method getRegex. If it is empty, an empty string is returned. The constant EMPTY is used for this. Otherwise, a matcher is created and used to determine the end position of the match. For keepAfter, the first match can be used; for keepAfterLast, the while loop must run until the last match is reached. Afterward, the getSuffixOrAll method determines the suffix if a match is found, or returns the entire text.
In order for FreshMarker to recognize these two built-ins, they must still be registered.
register.add("keep_after", (x, y, e) -> keepAfter((TemplateString)x, y));
register.add("keep_after_last", (x, y, e) -> keepAfterLast((TemplateString)x, y));
I haven’t been entirely convinced by the two built-ins, remove_beginning and remove_ending, so I won’t discuss them further here. However, they may still be included in one of the next versions of FreshMarker.
The matches built-in has not yet been discussed. This built-in returns a list of matches for the regular expression. Details for each match can then be accessed. For our example text "Chico Harpo Groucho Gummo Zeppo” and the pattern “(\w)\w+”, we get a list of five matches. The original version has the peculiarity that the return value can be also a boolean, depending on where it’s used. Such possibilities do not exist in FreshMarker and do not seem useful. A simple check for an empty list fulfills the same requirement.
For our new match_results built-in, we’ll start with the matchResults method.
private TemplateListSequence matchResults(TemplateString text, List<TemplateObject> parameters) {
Matcher matcher = getMatcher(text.getValue(), getRegex(parameters));
List<Object> list = matcher.results().map(MatchResultWrapper::new).map(Object.class::cast).toList();
return new TemplateListSequence(list);
}
As before, we create a Matcher using the regular expression from the first parameter. We then retrieve all MatchResult instances from the matcher, wrap them in a MatchResultWrapper record, and store them in a list. We then add this list to a TemplateListSequence, and we have our result. It’s worth noting here that the MatchResultWrapper is necessary. The FreshMarker type system relies almost exclusively on concret classes in many areas. This includes, among other things, mapping to model classes and registering built-ins. These two points are precisely the reason for the existence of the MatchResultWrapper class, since the MatchResult returned by the matcher is a private implementation of the MatchResult
register.add("match_results", (x, y, e) -> matchResults((TemplateString)x, y));
After we’ve registered our new built-in, we still can’t use it, because FreshMarker treats MatchResultWrapper as a regular POJO. For our intended use, we still need a model class called TemplateMatchResult.
public class TemplateMatchResult extends TemplatePrimitive<MatchResult> implements DotHashAddressable {
public TemplateMatchResult(MatchResult value) {
super(value);
}
@Override
public TemplateObject get(ProcessContext context, String name) {
return switch (name) {
case "start" -> TemplateNumber.of(getValue().start());
case "end" -> TemplateNumber.of(getValue().end());
case "group" -> new TemplateString(getValue().group());
case "group_count" -> TemplateNumber.of(getValue().groupCount());
default -> throw new ProcessException("unknown attribute: " + name);
};
}
}
The class extends TemplatePrimitive and implements the DotHashAdressable interface. As a subclass of TemplatePrimitive, TemplateMatchResult can be used as a primitive value in FreshMarker, and because it implements the get method from the DotHashAdressable interface, four of its attributes (start, end, group, and groupCount) are directly accessible via dot notation. Additional access to the MatchResult is enabled through extra built-ins.
The mapping between MatchResultWrapper and TemplateMatchResult takes place in the DefaultTypeMapperProvider.
Map.entry(StringBuiltInProvider.MatchResultWrapper.class, o -> new TemplateMatchResult((MatchResult)o))
This entry wraps every MatchResultWrapper instance in a TemplateMatchResult. In addition, we register three built-ins for TemplateMatchResult to access the capturing groups of the regular expression. We will not go into further detail about their implementation, because they follow the patterns already shown here.
register.add(TemplateMatchResult.class, "start", (x, y, e) -> startGroup((TemplateMatchResult)x, y)); register.add(TemplateMatchResult.class, "end", (x, y, e) -> endGroup((TemplateMatchResult)x, y)); register.add(TemplateMatchResult.class, "group", (x, y, e) -> group((TemplateMatchResult)x, y));
Now we can finally use our new built-in method match_results:
<#list text?match_results('\w+') as match>
<#var v=match.group/>
${v[(v?length-1)..0]}
</#list>
ocihC opraH ohcuorG ommuG oppeZ
In this example, the regular expression matches words, stores each match in a variable, and uses these variables to reverse the words. The expression v[(v?length-1)..0] uses a slice with inverted ranges to reverse the string.
<#list text?match_results('(\w+) (\w+)') as match>
${match?group(2)}-${match?group(1)}
</#list>
Harpo-Chico Gummo-Groucho
In the second example, the regular expression also matches words, but this time, two words separated by a space. In the output, both matches are joined by a hyphen: first, the content of the second capturing group is read, and then the first. The result is the first four names of the Marx Brothers; since Zeppo has no partner, he does not appear in the result.
The built-ins are all part of Freshmarker 3.0.0—enjoy!