Category Archives: Code Generation

Writing JavaScript templates using ejs

There are quite a few (as you’d probably expect) templating engines for use with JavaScript, such as Mustache and handlebars both of which use {{ }} syntax.

The syntax used is really not very important, what matters is the flexibility/functionality, but for nostalgia’s sake (as much as anything) this post will instead use ejs which uses syntax such as <%= %> which is very much like old ASP from my recollections (hence the nostalgia).

Note: I’m not endorsing ejs over any other engine, I just picked this to begin with, if/when I get time I will probably create posts covering using some other templating engines also.

Getting Started

We need to start by installing ejs, so run the following

yarn add ejs

We’re going to create a folder named templates (this is not a requirement of ejs) to store our .ejs files. So the template files will be saved with the .ejs extension.

Let’s just create a very standard Hello World app. so in the file templates/sample.ejs write the following

<%= message %>

Now let’s write the code to use ejs generate some output from our template and supply our data to the generator.

const ejs = require("ejs");

ejs.renderFile("./templates/sample.ejs", {
    message: "Hello World",
  },
  {},
  (err: any, result: string) => {
    if(err != null) {
      console.log(err);
    }
    // use the results of the template combined
    // with the supplied model
    console.log(result);
});

Here we show the basic usage pattern, which uses the asynchronous renderFile function. This takes the template file (we can ofcourse also supply the template as a string using the render function). The second argument is a “model” of the data being passed into the template engine and ofcourse the key’s related to the variable names used within the template file. The third argument supplies any options to the template engine, in this example I’m not applying any options. Finally, as this function is asynchronous, we will get an error or result or the processing of the template.

More…

That was simple enough but we’re not going to want to just output simple values like this, so let’s take a look at some more capabilities.

If we take a look at the Docs (https://ejs.co/#docs), Tags section we can see the tags for embedding our model data, let’s take a look at some of these in usage.

<%

The scriptlet tag is used for flow control and does not embed output into the template itself (as such). Instead it allow us to use conditionals such as if or loops such as for loops on our data model items.

type Names = <% for(let i = 0; i < names.length; i++) { %>
    | '<%= name[i].header %>'<% } %>;

In this example we’re code generating some types by taking an array of names from the model and creating a union of those header names.

Like-wise we might also use this tag for if statements within the <% %gt;

<%_

This will strip whitespace from before the tag. One of the problems we often find with templated code generation is how to control whitespace before and after the generated code. Hence this will remove the preceding whitespace.

<%=

This simply embeds an HTML escaped value into our template, in other words it simply replaces the tag with the value from message in this example below. Ofcourse message is a value stored in our data model.

<%= message %>

<%-

As per <%= but no escaping of values takes place.

<%#

Comment tags, just in case you wanted to add some comments/documentation to your template, this does not get output as part of the generated data.

<%%

Escapes the tag, i.e. outputs <%

%>

Standard end tag for use with the other start tags.

-%>

Trims the following newline.

_%>

Removes whitespace after the tag, again allows us to have some control of the whitespace following our generated output.

Transforming config files using SlowCheetah

It’s not unusual for our applications to require different configuration files (whether App.config or other) which might reflect different environments or build configurations. Under such circumstances I’ve tended to stick to using (good old) Nant to switch tokens in my configuration files as and when required.

An alternative to Nant or Cake etc. is Microsoft’s SlowCheetah, which allows us to create transformations within our configuration files, whether XML or JSON based as part of the build process within Visual Studio. These files will be named after the build configuration that you build your project with, i.e. Debug, Release etc.

To try this out, we first need to install SlowCheetah. So from Tools | Extensions and Updates, search for SlowCheetah and download it – you’ll need to restart Visual Studio to get it to installed.

Once installed we can right mouse click on the App.config and should see the option Add Transform.

Note: When we first add a transform Visual Studio will ask you to install the SlowCheetah NuGet package, select Yes when prompted.

Once you’ve executed Add Transform against the App.config you’ll find child nodes in solution explorer for App.Debug.config and App.Release.config (and any other build configurations you have).

If you right mouse click on either of these new config files, the context menu should show the option to Preview Transform. As the name suggests, this allows us to see what the transform will look like instead of having to build our solution to view the App.config within the bin folder.

Transforming .config files

In our simple implementation/example, we’ve got three App.config files now. The original App.config (which is ultimately what we’ll see in the bin folder as the exe.config) and a version of this file for Debug and Release builds.

What we need to do is add some configuration into the App.config as normal, so for example add the following

<appSettings>
   <add key="environment" value="NONE" />
</appSettings>

We can now add configuration to the Debug and Release config files to override this value. To save on duplicated config let’s concentrate on the App.Debug.config, so when we build using the Debug configuration SlowCheetah will transform our App.config with instructions from the App.Debug.config. Here’s an update to the App.Debug.config file

<appSettings>
   <add key="environment" value="DEBUG" 
      xdt:Transform="Replace" 
      xdt:Locator="Match(key)" />
</appSettings>

Note: the syntax for XML transformations is taken from XDT and for JSON from JDT.

As you can probably work out for yourself, we’re telling the transformation engine to replace the value for the given key. Without these two attributes no transformation takes place. Just adding configuration to this file, i.e. if we added another key/value solely to the App.Debug.config, will simply be ignored unless we tell the transformer what to do. For example if we wanted the App.Debug.config build to insert/add a new app settings key/value we can tell the transformer to do this using

<add key="url" 
   value="http://somewhere.com" 
   xdt:Transform="Insert"/>

Transformation syntax for XDT can be found here Transform Attribute Syntax.

Transform attributes include

  • Replace
  • Insert
  • InsertBefore
  • InsertAfter
  • Remove
  • RemoveAll
  • RemoveAttributes
  • SetAttributes

Transforming .json files

We may be using JSON files for configuration or support files (for example when creating .NET core applications or in any other type of application). SlowCheetah allows transformations on JSON as well using JDT.

Let’s add a JSON file to our test application, mine’s named Config.json and includes the following

{
  "appSettings" : {
     "environment" : "None" 
  }
}

Now in solution explorer within Visual Studio, right mouse click on Config.json and select Add Tranform. Again we’ll concentrate solely on the Config.Debug.json file that’s created as part of this process. Add the following to this file

{
  "appSettings" : {
    "@jdt.replace" : {
      "environment" :  "Debug" 
    } 
  }
}

As I’m sure you can see, we’re aiming to replace the value of the environment key to Debug (as we did with the .config previously). Obviously to see this happen and output to the relevant bin folder, select the Config.json (parent node) and set its property Copy to Output Directory to Copy if newer, now select each of the child nodes and switch those back to Do not copy as Visual Studio will set the child nodes to the same build action as the parent and we do not want/need these environment configurations in our output folder.

JDT includes the following verbs

References

SlowCheetah
SlowCheetah on GitHub

Creating a text templating engine Host

I’m in the process of creating a little application to codegen some source code for me from an XML schema (yes I can do this with xsd but I wanted the code to be more configurable). Instead of writing my own template language etc. I decided to try and leverage the T4 templating language.

I could simply write some code that can be called from a T4 template, but I decided it would be nicer if the codegen application simply acted as a host to the T4 template and allowed the template to call code on the host, so here’s what I did…

Running a T4 template from your application

The first thing I needed was to be able to actually run a T4 template. To do this you’ll need to add the following references

  • Microsoft.VisualStudio.TextTemplating.11.0
  • Microsoft.VisualStudio.TextTemplating.Interfaces10.0
  • Microsoft.VisualStudio.TextTemplating.Interfaces.11.0

Obviously these are the versions at the time of writing, things may differ in the future.

Next we need to instantiate the T4 engine, this is achieved by using the Microsoft.VisualStudio.TextTemplating namespace and with the following code

Engine engine = new Engine();
string result = engine.ProcessTemplate(File.ReadAllText("sample.tt"), host);

Note: The host will be supplied by us in the next section and obviously “sample.tt” would be supplied at runtime in the completed version of the code.

So, here we create a Engine and supply the template string and host to the ProcessTemplate method. The result of this call is the processed template.

Creating the host

Our host implementation will need to derive from MarshalByRefObject and implement the ITextTemplatingEngineHost interface.

Note: See Walkthrough: Creating a Custom Text Template Host for more information of creating a custom text template.

What follows is a basic implementation of the ITextTemplatingEngineHost based upon the Microsoft article noted above.

public class TextTemplatingEngineHost : MarshalByRefObject, ITextTemplatingEngineHost
{
   public virtual object GetHostOption(string optionName)
   {
      return (optionName == "CacheAssemblies") ? (object)true : null;
   }

   public virtual bool LoadIncludeText(string requestFileName, 
            out string content, out string location)
   {
      content = location = String.Empty;

      if (File.Exists(requestFileName))
      {
         content = File.ReadAllText(requestFileName);
         return true;
      }
      return false;
   }

   public virtual void LogErrors(CompilerErrorCollection errors)
   {
   }

   public virtual AppDomain ProvideTemplatingAppDomain(string content)
   {
      return AppDomain.CreateDomain(&quot;TemplatingHost AppDomain&quot;);
   }

   public virtual string ResolveAssemblyReference(string assemblyReference)
   {
      if (File.Exists(assemblyReference))
      {
         return assemblyReference;
      }

      string candidate = Path.Combine(Path.GetDirectoryName(TemplateFile), 
            assemblyReference);
      return File.Exists(candidate) ? candidate : String.Empty;
   }

   public virtual Type ResolveDirectiveProcessor(string processorName)
   {
      throw new Exception(&quot;Directive Processor not found&quot;);
   }

   public virtual string ResolveParameterValue(string directiveId, 
            string processorName, string parameterName)
   {
      if (directiveId == null)
      {
         throw new ArgumentNullException(&quot;directiveId&quot;);
      }
      if (processorName == null)
      {
         throw new ArgumentNullException(&quot;processorName&quot;);
      }
      if (parameterName == null)
      {
         throw new ArgumentNullException(&quot;parameterName&quot;);
      }

      return String.Empty;
   }

   public virtual string ResolvePath(string path)
   {
      if (path == null)
      {
         throw new ArgumentNullException(&quot;path&quot;);
      }

      if (File.Exists(path))
      {
         return path;
      }
      string candidate = Path.Combine(Path.GetDirectoryName(TemplateFile), path);
      if (File.Exists(candidate))
      {
         return candidate;
      }
      return path;
   }

   public virtual void SetFileExtension(string extension)
   {
   }

   public virtual void SetOutputEncoding(Encoding encoding, bool fromOutputDirective)
   {
   }

   public virtual IList&lt;string&gt; StandardAssemblyReferences
   {
      // bare minimum, returns the location of the System assembly
      get { return new[] { typeof (String).Assembly.Location }; }
   }

   public virtual IList&lt;string&gt; StandardImports
   {
      get { return new[] { &quot;System&quot; }; }
   }

   public string TemplateFile { get; set; }
}

Now the idea is that we can subclass the TextTemplatingEngineHost to implement a version specific for our needs.

Before we look at a specialization of this for our purpose, let’s look at a T4 template sample for generating our code

Note: mycodegen is both my assembly name and the namespace for my code generator which itself hosts the T4 engine.

<#@ template debug="false" hostspecific="true" language="C#" #>
<#@ assembly name="mycodegen" #>
<#@ import namespace="mycodegen" #>
<#@ output extension=".cs" #>

<#
   ICodeGenerator cg = ((ICodeGenerator)this.Host);
#>

namespace <#= cg.Namespace #>
{
   <# 
   foreach(var c in cg.Classes) 
   {
   #>
   public partial class <#= c.Name #>
   { 
      <#  
      foreach(Property p in c.Properties)
      {
          if(p.IsArray)
          {
      #>
         public <#= p.Type #>[] <#= p.Name #> { get; set; }
      <#
           }
           else
           {
      #>
         public <#= p.Type #> <#= p.Name #> { get; set; }
      <#
           }
      }
      #>
   }
   <#
   }
   #>
}

So in the above code you can see that our host will support an interface named ICodeGenerator (which is declared in the mycodegen assembly and namespace). ICodeGenerator will simply supply the class names and properties for the classes extracted from the XML schema and we’ll use the T4 template to generate the output. By using this template we can easily change how we output our classes and properties, for example xsd creates fields which are not required if we use auto-implemented property syntax, plus we can change the naming convention, property name case and so on an so forth. Whilst we could add code to the partial classes generated by including other files implementing further partial methods etc. if a type no longer exists in a month or two we need to ensure we deleted the manually added code if we want to keep our code clean. Using the T4 template we can auto generate everything we need.

References

Walkthrough: Creating a Custom Text Template Host
Processing Text Templates by using a Custom Host

Getting started with T4 templates

What are T4 templates

First off, T4 stands for Text Template Transformation Toolkit. It allows us to embed C# (or VB.NET) code into documents and optionally combine the code with plain text, which can then be used to generate new documents – therefore similar to the way ASP or PHP engines work when producing HTML. The output text document might be a C# file, HTML file or pretty much any other file type you want.

A T4 template file ends in a .tt extension.

Within Visual, when a T4 template is executed the resultant file is created as a child node (in the solution explorer) off of the template’s node.

Let’s get started

If you create a solution (or have a solution ready), then on a selected project, select the “Add New Item” option. Then select “Text Template”, give it a name (mine’s named Test.tt) and add to the project.

By default the file will look similar to the following (this is from the Visual Studio 2012/2013 T4 item template)

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ assembly name="System.Core" #>
<#@ import namespace="System.Linq" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ output extension=".txt" #>

Note: by default Visual Studio (well 2013 at least) does not offer syntax highlighting. You can use Tools | Extensions and Updates to add T4 Toolbox which is a free syntax highlighter etc. for T4.

So in the template code above we can see that <#@ #> are used to enclose “directives”.

The template adds a reference to the System.Core assembly and imported the namespaces System.Linq, System.Text and System.Collections.Generic. The template sets the output extension (by default) to .txt. Hence the template will generate a file name Test.txt file (in my case). Obviously the name of the file is a copy of the name of your template file.

The template directive is described here.

We can see that the language attribute denotes whether we want to write VB or C# code within our code statement blocks.

The hostspecific attribute denotes that we want the T4 template to allow us access to the this.Host property of type ITextTemplatingEngineHost. From this we can use methods such as the ResolvePath method, which allows us to get a combined path of our supplied filename or relative path, combined with the current path of the solution. See ITextTemplatingEngineHost Interface for more information.

Finally the debug attribute denotes that we want the intermediate code to include code which helps the debugger to identify more accurately where an exception or the likes has occurred.

Include Directive

We can also include T4 code using the <#@ include file=”mycode.tt” #> See T4 Include Directive.

Blocks

So we’ve looked at the <#@ #> directive, but there’s more…

<# Statement Blocks #>

Statement blocks allow you to embed code into your template. This code can simply be blocks of code or can be used to wrap loops or if statements which in turn wrap around output text, as per

<#
for(int i = 0; i < 3; i++)
{
#>
   Hello
<#
}
#>

or, as mentioned, simply embedding code blocks, such as

<#
for(int i = 0; i < 3; i++)
{
   WriteLine("Hello");
}
#>

which results in the same output at the previous sample. See also Statement Syntax.

<#+ Class Feature Blocks #>

Class feature blocks allow you to create code which you can reuse within your template. Statement blocks are like the inner blocks of a method, but ofcourse it would be useful to be able to actually create reusable methods. This is where Class Blocks come in.

For example

<#= WrapInComment("Hello") #>

<#+
private string WrapInComment(string text)
{
   return "<!-- " + text + " -->";
}
#>

It’s important to note that the statement block calling code is actually before the method declaration.

In this example we return a string, which can be used from a <#= Expression Block #> (see below). If the method returned void then it would equally be callable from a <# Statement Blocks #>.

<#= Expression Blocks #>

Expression blocks allow us to write out results from our code to the generated output text, for example

<#
int sum = 10 + 4;
#>

<#= sum #>

So the first block is a statement block where we declare the sum variable and it is output in situ via the <#= #> code. See also Expression Syntax (Domain-Specific Languages).

Great, now let’s do something with all this!

Let’s write a T4 template which will take a C# class, which contains our model (as a partial class) and has properties marked with XmlElementAttribute and then generates another partial class for the type, with some new ShouldSerializeXXX methods – the idea being any string property which is empty or null, will not be serialized.

This example template is not perfect and I’m sure there are better ways to do things, but I really wanted to create a more meaningful example than a “Hello World”, so bare with me.

<#@ template debug="false" hostspecific="true" language="C#" #>
<#@ assembly name="System" #>
<#@ assembly name="System.Xml" #>
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Text" #>
<#@ import namespace="System.Collections.Generic" #>
<#@ import namespace="System.Xml.Serialization" #>
<#@ import namespace="System.Reflection" #>
<#@ import namespace="System.CodeDom.Compiler" #>
<#@ output extension=".cs" #>
<#
string model = File.ReadAllText(this.Host.ResolvePath("MyModel.cs"));

CodeDomProvider provider = CodeDomProvider.CreateProvider("CSharp");
CompilerParameters cp = new CompilerParameters();
cp.GenerateExecutable = false;
cp.GenerateInMemory = true;
cp.ReferencedAssemblies.Add("System.dll");
cp.ReferencedAssemblies.Add("System.Core.dll");
cp.ReferencedAssemblies.Add("System.Xml.dll");

CompilerResults cr = provider.CompileAssemblyFromSource(cp, new string[] { model });
if (cr.Errors.Count > 0)
{
   Error("Errors detected in the compiled model");
}
else
{
#>
using System;

<#
    bool nsAdded = false;

    Type[] types = cr.CompiledAssembly.GetTypes();
	foreach (Type type in types)
	{
        // time to generate code
        if(nsAdded == false)
        {
#>
namespace <#=type.Namespace#>
{
<#
        }
        nsAdded = true;
#>
    public partial class <#=type.Name#>
    {
<#
		List<PropertyInfo> toGenerate = new List<PropertyInfo>();

		foreach (PropertyInfo pi in type.GetProperties())
		{
		    if(pi.PropertyType == typeof(string))
			{
			    XmlElementAttribute[] a = (XmlElementAttribute[])pi.GetCustomAttributes(typeof(XmlElementAttribute), true);
				if (a != null && a.Length > 0)
				{
#>
        public bool ShouldSerialize<#=pi.Name#>()
        {					
            return !String.IsNullOrEmpty(<#=pi.Name#>);
        }
<#
                }
            }
		}
#>
    }

<#
        }
    }
#>
}

Before we look at the output from this T4 template, let’s quickly review what’s in this template. Firstly we reference the two assembles, System and System.Xml then add our “using” clauses via the import directive. The template is marked as hostspecific so we can use the Host property.

We read the MyModel.cs file, in this instance. Although we could have possibly looked to interact with the Visual Studio environment instead to achieve this. Then we create a C# CodeDomProvider, we could use Roslyn instead for this. The purpose of the CodeDomProvider is solely to use it to compile the MyModel.cs and allow us to use reflection to get the properties with the XmlElementAttribute as I don’t really want to parse the source code myself (this is where Roslyn would have come in).

Now you can see interspersed with our T4 blocks of code is the output text which creates the namespace, class and ShouldSerializeXXX methods.

So the code gets the properties on our MyModel object, then finds those with a string type and also with the XmlElementAttribute attribute applied and then creates the namespace, class and methods to match these properties. Writing output which looks something like the following

using System;

namespace DomainObjects
{
    public partial class MyModel
    {
        public bool ShouldSerializeName()
        {					
            return !String.IsNullOrEmpty(Name);
        }
        public bool ShouldSerializeAge()
        {					
            return !String.IsNullOrEmpty(Age);
        }
        public bool ShouldSerializeAddress()
        {					
            return !String.IsNullOrEmpty(Address);
        }
   }
}

And finally (for now)…

Right mouse click on the T4 .tt file and you will see a Debug T4 template. I actually only discovered this after searching for something else related to this post and came across T4 TEMPLATE DEBUGGING which talks about it.

You can now put break points against your T4 code and run the T4 debugger!