Declarative Roslyn and Metaprogramming

Topics: APIs, General
Feb 2, 2015 at 5:04 PM
Edited Feb 2, 2015 at 5:06 PM
Hey guys,

I thought I'd share my work on metaprogramming using Roslyn. I recently finished a prototype version of Excess where you can write live extensions for c# such as this awesome match extension.

The work is based on detecting and rewriting syntactical errors, however, on some cases the detection gets tricky due to the unpredictability of the erroneous syntax tree (something Roslyn is awesome at, but even superheroes need help sometimes)

So I decided to rewrite my whole meta-compiler and make it declarative, for instance a "constructor" extension may be written like this:
        method => method.ReturnType.IsMissing && 
                  method.Identifier.ToString() == "constructor")
Where Constructor is a Roslyn function that takes a MethodDeclarationSyntax and transforms it into a ConstructorDeclarationSyntax. Pretty neat, no?

But this still doesn't help Roslyn in complicated cases. Imagine, for instance, your extension being written in a completely foreign language using another parsing technology such as Antlr. The solution I found was to create a brand new lexical pass designed to adapt the text before it gets compiled as so to minimize the weirdness for Roslyn. It looks like this:
    .any('(', '=', ',')
    .token("function", named: "fn")
    .enclosed('(', ')')
    .token('{', named: "brace")
        .insert("=>", before: "brace"))
Venture a guess? Either way (and at risk of sounding cocky) I think this is to metaprogramming what Roslyn is to code analysis: There is nothing like it out there :)

I have put a lot of effort into this and would love feedback from the pros and community alike.

Feb 2, 2015 at 9:14 PM
Edited Feb 2, 2015 at 9:16 PM
Personally, I really don't like stringly-typed code. I think it would be much better if your example was written something like this:
var fn = lexical.match() 
    .any('(', '=', ',')
var brace = fn
    .enclosed('(', ')')
        .insert("=>", before: brace))
Or maybe:
Token fn, brace; // I don't know if you have any type like this already, so I called it Token
    .any('(', '=', ',')
    .token("function", out fn)
    .enclosed('(', ')')
    .token('{', out brace)
        .insert("=>", before: brace))
Feb 2, 2015 at 9:38 PM
This would be fairly hard to implement, giving that this code "describes" a parsing process that will be potentially ran many times. So, I couldn't really store results in single tokens but "token references to be lazily initialized, many times, in different contexts". Doable, but hardly worth the effort, IMO.

Thanks for the suggestion!
Feb 5, 2015 at 1:20 PM
I think you may do a real-world metaprogramming experiment, for example: JSON-able C#?
Feb 5, 2015 at 5:29 PM

Yeah, that should be piece of cake.
Feb 6, 2015 at 6:09 PM
Hey guys/gals!

One last time here, since there isn't a lot of interest :) I just committed a fully functional prototype of the new compiler. It ended up quite marvelous, if I'm allowed to say that about my own code: 15 extensions to C# in under 1000 loc:

1- JS-style function as lambdas
2- JS-style function as method, with return type inference
3- JS-style function inside of code blocks
4- function as types
5- method construct with return type inference and automatic visibility
6- property construct, with type inference, automatic visibility, initialization
7- constructor construct
8- Delegate-less event declaration
9- Syntactic event handlers: on ev (args) {}
10- JS-style arrays: var x = [1, 2, 3];
11- typedef construct, supporting generic parameters, etc. 2 variations.
12- asynch extension
13- synch extension
14- match extension, which generalizes the switch statement
15- contract extension

That would be about 66 lines of code per extension, which is pretty unbeatable. Anyhow, you can see the tests here:

The rest of the code is not too far away, if curious. Unfortunately, this not yet integrated into the web front end, that's on the way (It's only me, help!!)
Now, I'm a little worried that all of this is somehow a terrible idea and I'm just not seeing it. If so, please do tell and save me from myself.

Mar 19, 2015 at 4:11 PM
Knat wrote:
I think you may do a real-world metaprogramming experiment, for example: JSON-able C#?
So, I thought of a couple of ways of making json, which weren't as "piece of cake" as I thought. So I took the plunge and implemented full-blown grammars (ANTLR grammars at that) and now it is truly a piece of cake :)

Check this 2 minute video of how to make the json extension along with the code: