Template Functions & Template Masks

Topics: APIs, C# Language Design, VB Language Design
Sep 20, 2014 at 12:12 AM
Edited Sep 20, 2014 at 12:16 AM

Template Functions

Template Functions will allow you return the expression (syntax node) for the body of the function. I'm including statements as an expression as well. Inside them anything but not inside a template hole %{ } will be treat as a code quotation. Anything inside a template hole %{ } will be treated as executable code and evaluated.

C#
template CS.Expr VarDeclaration<T>( String! name, T value )
{
  var %{name} = %{value};
}
VB.net
Template(Of T)( name As String!, Value As T) As VB.Expr
  Dim %{name} As %{GetType(T)} = %{Value}
End Template

Template Mask

Template Mask is closely related to Template Functions instead of synthesising the expression. It generate code to see if a syntax node matches the pattern of the template.

Example
Template Mask IF_BLOCK() As Mask< VB.Expr >
  If %{0} Then
     %{1}
  End If
End Template
 ' Named Mask Holes
Template Mask IF_BLOCK() As Mask< VB.Expr >
  If %{ pred } Then
     %{ body }
  End If
End Template
Will generate a template mask to see if the node is a If Block.
Also note it will match against Else If , Else, etc because the inner body could be any expression.

So how can they used?
The following example would replace the if-block with an if-statement if it contains only a single line of code.

Template IF_Statement( pred As VB.Expr<Bool>, code As VB.Expr ) As VB.Expre
  If %{ pred } Then %{ code }
End Template


 Dim q = IF_BLOCK()
 
 If q.Matches( node ) Then
    If q(1).Lines.Count = 0 Then node =  IF_Statement( q(0), q(1) ) 
 End If
' Using the name mask holes
 If q.Matches( node ) Then
    If q( "code" ).Lines.Count = 0 Then node =  IF_Statement( q("pred"), q("code") ) 
 End If
Since the compiler is generating the template mask. It maybe also possible to provide IDE support to restrict the possible inputs. Eg If_Statement( q( "foo" ), q( "bar" ) are compile-time errors.
Sep 20, 2014 at 1:06 PM
I'm also contemplating syntax, but largely focused on what we can do now.

How do you see debugging working?

My current perspective is that there are three things:
- stuff we don't care about - IL/native
  • 3GL that we can understand and debug (real code)
  • Shortcuts that better resemble the way we think
This perspective leads me to desire a pre-compile step. It has the side benefit of being something we can build now. I'm sufficiently bad at creating VS languages that I'm currently using legal (C# at the moment) code and [[magic comments for compiler communications]].
Sep 21, 2014 at 11:33 PM
Edited Sep 21, 2014 at 11:34 PM
KathleenDollard wrote
How do you see debugging working?
Template A
  If %{0} = %{1} Then %{2} = %{3}
End Temple
This isn't executable source code, but a code template (think a "code stencil" / "code schematic") that can used to produce executable source code.
It is only checked for "structurally" valid code., any structural errors are highlighted.
Template Function B ( tm As TemplateMatch ) As Template(Of VB.Expr)
  Template
    Select Case %{0}
  End Template
  For Each m In tm.Matches
    Template
      Case %{m(1)} :  %{m[2]} = %(m(3))
    End Template
  Next

  Template 
    End Select
  End Template

End Function
A template function is used when you want to output a template that was generate via code.
The resultant template is checked to see if it is valid code, it is at the point the what as put in the template hole is checked.
If it doesn't produce valid code, the coder show have the option to apply it ( which would cause the compiler to display/highlight the errors ) or cancel.
Dim this_method
Dim if_statements = this_method.FindTemplateMatches(Of A)
Dim trans = if_statements.Where( Function(m) Typeof(m(1)) IsA TypeOf(m(0}) ).WhereContigous(aresame:= {Function(m) m{0}, Function(m) m{2} } )
if trans.Any() Then trans.Transform( into: B )
I admit that this part of the process could be improved. Maybe a API similar to you RoslynDOM could be used?

Example
Let's say we're using the follow code.
Dim item As String
Dim Price As Decimal
If  Item = "Apple " Then Price = 0.30D
If  item = "Banana" Then Price = 0.35D
If  item = "Carrots" Then Price = 0.50D
The transform result would be
Dim Item As String
Dim Price As Decimal
Select Case Item
  Case "Apple" : price = 0.30D
  Case "Banana" : price = 0.35D
  Case "Carrot" : price = 0.50D
End Select
Sep 22, 2014 at 8:08 PM
When I run the transformed code and it hits a bug, what code does the debugger break into?
Sep 22, 2014 at 8:09 PM
BTW, I've been doing similar transforms in prototypes for a while. We can do this now, as long as we can debug the final code.

FWIW, I've come to hate special holes syntax. I think it is unnecessary. The coder knows what they will recognize and the engine can recognize it.
Nov 10, 2014 at 5:11 PM
Edited Nov 10, 2014 at 6:45 PM
I've been tweaking the syntax
  |{  }|    Template
  /{  }/    Template Arg Hole ( Allows identifiers / executable code to be used within templates.
This is to enable executable code to be used. The following example unpacks a tuple as method arguments.
template unpack(this tu : Tuple , method : Action ) : CS.Expr
{
  |{ /{ method }/ ( /{ tu[0] }/ }|
  for(int i = 1; i < tu.Arity; i++)
  {
    |{, /{ tu[i] }/ }|
  }
  |{ ) }|
}
Would produce the syntax node / tree for the following method call assuming a tuple<Int,String,Double>
method( (tu.Item1 As String), (tu.Item2 As String) , (tu.Item3 As Double) )

Very rough grammar for template functions for CSharp
template_function        ::= template return_type method_identifier generic_params?
                             method_args method_body
generic_params           ::= '<' generic_type (',' generic_type )* '>'
method_args              ::= '(' (parameter ( ',' parameter ) *)? ')'
method_body              ::= '{' (template | expr )* '}'
method_identifier        ::=
return_type              ::= 

template                 ::= template_header template_body template_footer
template_header          ::= "|{"
template_footer          ::= "}|"
template_body            ::= template_arg_hole | expr

template_arg_hole        ::= template_arg_hole_header template_arg_hole_body
                             template_arg_hole_footer
template_arg_hole_header ::= "/{"
template_arg_hole_footer ::= "}/"
template_arg_hole_body   ::= identifier | expr
Nov 10, 2014 at 9:23 PM
Another example replace If - Else structure to Inline if structure.
This template mask matches against assignments
template mask simple_assignment() : template_mask
{
  |{ /{ target }/ = /{ source }/ ; |}
}
This template generates an assignment that utilises and inline if expression. cond ? true : false
template assignment_viaCondition_ <T>( target : __      , cond   : Expr<Bool> ,
                                       valueA : Expr<T> , valueB : Expr<T> )  : CS.Expr
{
  |{ var /{ target }/ = (/{ cond }/ ) ? ( /{ valueA }/ ) : ( /{ valueB }/ ) ; |}
}
The following template mask matches against the if else construct.
template mask _IfElse_() : template_mask  
{
  |{ if( /{ cond }/ 
     {
       /{ on_true }/
     }
     else
     {
       /{ on_false }/
     }
   }|
}
So let's utilise the previously construct templates and template masks.
AnalyseNode ( node : SyntaxNode )
{
  var mn = template.Match( node , _IfElse_ )
  if( !mn.Valid ) return
  var part_0 = template.Match( mn["on_true"], simple_assignment )
  var part_1 = template.Match( mn["on_false"], simple_assignment )
  if( part_0.Valid && part_1.Valid )
  {
    if( part_0["target"] == part_1["target"] ) 
    { 
      node.ReplaceWith( assignment_viaCondition_( part_0["target"] , mn["cond"] , part_0["source"],  part_1["source"] );
    }
  }
}
I think that if CSharp and VB.net had templates and template mask, creating diagnostics and code-fix would be a lot simpler.
Nov 11, 2014 at 8:04 PM
This all is already perfectly possible with T4. I personally feel direct text transformations to be out of the language spirit, so IMHO it's better to let them be outside (and bring some more attention of the language team to the T4 further development: e. g. built-in support of class/function model).
Nov 11, 2014 at 10:43 PM
VladD these are not just generating code but also doing a "structural" pattern matching. Like a regex with named groups.
It doesn't return the source code text but the syntax node that represents the code. It's syntatic sugar over syntax nodes and trees.

From designing the grammar and syntax it looks and feels really natural within C#.


Parse code
On finding a template switch context to template so parsing is in isolation. Eg parse code within in the context of template not the surrounding context.
On finding a TemplateArg Hole contents are parsed within the in the template contexrt
Nov 25, 2014 at 9:01 PM
Found a way to do Template Functions (kind of) in C#6 via String Interpolation.
See Blog Post