This project is read-only.

Meta-programming case study

Topics: General
Aug 25, 2014 at 9:55 AM
Edited Aug 26, 2014 at 2:25 AM
Hi Roslyn team,
So far, Metah.W: A Workflow Metaprogramming Language is definitely the most imaginative and amazing metaprogramming experiment using Roslyn, take a look at it by an online translator :)
Sep 10, 2014 at 1:10 PM
Edited Nov 2, 2014 at 8:31 AM
Get a feeling of (my) C# metaprogramming in 10 minutes:
  1. You need Visual Studio 2013 professional or advanced version
  2. Download and install Metah.0.51.vsix
  3. Open VS2013 -> new project -> Visual C# -> Metah.W -> create a Metah.W Console Application -> delete Program.cs
  4. Add new item -> Visual C# Items -> Metah.W -> create a .mw file named File1.mw
  5. Copy the following code into File1.mw
  6. Build solution, click "Show All Files" button in Solution Explorer, view File1.mw.cs
  7. Run the program if you like
//File1.mw
using System;
using System.Activities;

namespace HelloMW
{
    public enum DriveAction { Neutral = 1, Forward, Reverse, TurnOff }

    public sealed activity ActivityDemo()
    {
        bool isMoved;
        isMoved = false;
        while (!isMoved)
        {
            statemachine goto InPark
            {
                DriveAction action;
            InPark:
                ~> Console.WriteLine("Enter InPark");
                <~ Console.WriteLine("Exit InPark");
                on action = new Drive().Invoke(1);
                    if (action == DriveAction.Neutral) goto InNeutral;
            InNeutral:
                ~> Console.WriteLine("Enter InNeutral");
                <~ Console.WriteLine("Exit InNeutral");
                on action = new Drive().Invoke(1); {
                    if (action == DriveAction.Forward) goto InForward;
                    if (action == DriveAction.Reverse) goto InReverse;
                    if (action == DriveAction.TurnOff) goto TurnedOff;
                }
            InForward:
                ~> { Console.WriteLine("Enter InForward"); isMoved = true; }
                <~ Console.WriteLine("Exit InForward");
                on action = new Drive().Invoke(2);
                    if (action == DriveAction.Neutral) goto InNeutral;
            InReverse:
                ~> { Console.WriteLine("Enter InReverse"); isMoved = true; }
                <~ Console.WriteLine("Exit InReverse");
                on action = new Drive().Invoke(2);
                    if (action == DriveAction.Neutral) goto InNeutral;
            TurnedOff: break
                ~> Console.WriteLine("TurnedOff");
            }
            Console.WriteLine("isMoved: " + isMoved);
        }
    }

    internal sealed activity Drive([RequiredArgument]int SecondsToDelay) as DriveAction
    {
        delay TimeSpan.FromSeconds(SecondsToDelay);
        Result = (DriveAction)_random.Next((int)DriveAction.Neutral, (int)DriveAction.TurnOff + 1);
        Console.WriteLine("!action: " + Result.ToString());
    }
    ##
    {
        private static readonly Random _random = new Random((int)DateTime.Now.Ticks);
    }

    class Program
    {
        static void Main(string[] args)
        {
            WorkflowInvoker.Invoke(new ActivityDemo());
        }
    }
}
Code talks. The above code is intuitive and easy to understand, I think :)
~~~~~~~~~~~~
If WF runtime is an "advanced CLR" built on CLR, then Metah.W is an "advanced C#" extended from C#. Chapter 1 of Essential Windows Workflow Foundation is a brain storm for unimaginative programmers.
Sep 13, 2014 at 3:21 PM
That is very interesting.

Who's building that project? Are they an MVP/will they be at the summit?

Kathleen
Sep 15, 2014 at 10:28 AM
KathleenDollard wrote:
Who's building that project? will they be at the summit?
Me, 谭克(Tank), I'm learning English...
Sep 15, 2014 at 2:39 PM
Edited Sep 15, 2014 at 3:50 PM
Three ways of metaprogramming:
  • Attributes
[Activity(...)]
public class SomeClass { ... }
  • Magic comments
//@@activity(...)
public class SomeClass { ... }
  • Extended syntax (see File1.mw)
public activity MyActivity(int A, string B) { ... } 
Obviously, the last way is the most expressive and user-friendly, but (very) difficult to implement.
Sep 16, 2014 at 12:51 PM
The first is the least expressive, at least without a full redefinition of a .NET attribute away from being an (assembly) metadata element.

But why do you think the third is the most expressive?

With modern tools, I think we have a fantastic opportunity with the second. Visual Studio is already designed to support two languages simultaneously for IntelliSense, coloration, etc (Razor and VB XML literals for examples). Within some magic delimiters, a new language can exist. Because of the ability to nest languages without comments already demonstrated in these tools, I think the word "comments" should be dropped - perhaps "inside magic delimiters" is a good phrase.

Now I don't really think we should have a new language, because we're trying to be nice to the humans. But building on the ideas of current language and data structures (C#/VB, JSON, XML literals, etc) we could build something pretty sweet. But, while I want massive expressiveness in content, I would really like to see one language behind the magic delimiters.

But I absolutely do NOT see 2 and 3 being in competition. There are things we can say best in formal, IntelliSense and expanded C#/VB code. I've been showing my prototypes (code gen templates) and this Metah project is such a great example of another approach. I agree wholeheartedly that for certain things extended syntax is most user friendly, as shown in this project.

I believe that creating extended syntax for limited scenarios is not difficult to do. It's difficult because Roslyn drops the complexity of accomplishing this by an order of magnitude, but that we need it dropped by another order of magnitude to make creating extended syntax for appropriate problems easy enough. That is why I am building RoslynDom. I hope it will be applicable to other problems, but this is the specific problem I am trying to solve. Unfortunately, I am mired in the syntax nightmare of trying to turn just enough of C# (VB to come later) into a language agnostic tree. But once my tree supports what I need (and I fix the breaks in file management), I anticipate building the template creation (one expanded syntax) in a matter of days, or possibly hours.

(if you want more of my thoughts on other applications of RoslynDom, check out Hallway Conversations and .NET Rocks webcasts this week)

At any rate. We should not choose between 1, 2 and 3. Where information appropriately shared at runtime (contractual) can be harvested, we should do that (some validations, and other current data annotations). Where it is a compiler conversation, we should use expanded syntax where the problem gracefully lends itself to C#/VB extended syntax, we should do that. Where it's a bunch of information, or for some other reason is not gracefully contained in C#, we should use something inside magic delimiters.

Once we get a bang up good transformation language (template language in today's terms), it is possible that the only thing that needs the magic delimiters is the transformation language itself (that everything else can be done with extended syntax), but that is not my expectation.

Kathleen
Sep 16, 2014 at 12:53 PM
Knat wrote:
KathleenDollard wrote:
Who's building that project? will they be at the summit?
Me, 谭克(Tank), I'm learning English...
Thank you for your work.

Thank you for your efforts at learning English.

I apologize that I am so verbose, I suspect that is difficult.

Are you working directly between your expanded syntax and Roslyn, or have you created an abstraction layer for the language? I am in no way suggesting that is the appropriate approach. But if you have, I want to ask you about it.

Kathleen
Sep 16, 2014 at 2:35 PM
Edited Nov 2, 2014 at 8:37 AM
Hi Kathleen,
Here is some explanations for Metah.W. The following meta-code:
//file2.mw
[Serializable]
public sealed activity Activity1<T>([RequiredArgument]int Arg1, out string Arg2, ref SomeClass<T> Arg3) as DateTime?
    where T : class, new()
{
}//It's a mixture of C# class and function
will be translated into the following C# code:
//file2.mw.cs
//generated by Metah.W compiler
[Serializable]
public sealed class Activity1<T> : global::System.Activities.Activity<DateTime?> where T : class, new()
{
    [RequiredArgument]
    public global::System.Activities.InArgument<int> Arg1 { get; set; }
    public global::System.Activities.OutArgument<string> Arg2 { get; set; }
    public global::System.Activities.InOutArgument<SomeClass<T>> Arg3 { get; set; }
}
In meta-thinking, an activity is a function; for implementation, an activity is a C# class.
For my problem domain(WF 4.5 metaprogramming), I think "extended syntax" is the best choice. I'm expecting your "magic delimiters" metaprogramming practice :)
Sep 17, 2014 at 5:12 PM
Edited Nov 2, 2014 at 8:40 AM
In meta-model, an activity is a function, it's body contains statements. Some MW statements are same as C# statements: local declaration statement, expression statement, if, switch, while, foreach, try..., howerver, MW has its unique statements: statemachine(see File1.mw), parallel, parallelforeach, pick, delay, transacted, cancellable, compensable, receive, sendreply, send, receivereply...
Example:
//file3.mw
using System;
using System.Activities;

activity Activity1()
{
    int i;//MW variable declarator cannot have initializer(eg: int i = 0)
    i = 0;
    while (i < 3)
    {
        string s;
        DateTime dt;
        s = "hello";
        parallel
        {
            //branch1: invoke another activity
            dt = new Activity2().Invoke(i, ref s);
            //branch2: delay 2 seconds
            delay TimeSpan.FromSeconds(2);
        }
        Console.WriteLine("S: {0}, DT: {1}", s, dt); 
        i++;
    }
}
activity Activity2(int Arg1, ref string Arg2) as DateTime
{
    Console.WriteLine("Arg1: {0}, Arg2: {1}", Arg1, Arg2);
    Arg2 += " world";
    //MW has no return statement, instead, assign value to the reserved out argument "Result"
    Result = DateTime.Now;
}
class Program
{
    static void Main()
    {
        WorkflowInvoker.Invoke(new Activity1());
    }
}
MW compiler will generate the following C# implementation code from the meta-code:
//file3.mw.cs
using System;
using System.Activities;

class Program
{
    static void Mainu()
    {
        WorkflowInvoker.Invoke(new Activity1());
    }
}

class Activity1 : global::System.Activities.Activity
{
    private global::System.Activities.Activity __GetImplementation__()
    {
        global::System.Activities.Activity __vroot__;
        {
            var __v__0 = new global::System.Activities.Statements.Sequence();
            var i = new global::System.Activities.Variable<int>();
            __v__0.Variables.Add(i);
            var __v__1 = new global::MetahWActionActivity(__ctx__ =>
            {
                i.SetEx(__ctx__, 0);
            }

            );
            __v__0.Activities.Add(__v__1);
            var __v__2 = new global::System.Activities.Statements.While();
            __v__2.Condition = new global::MetahWFuncActivity<bool>(__ctx__ => i.Get(__ctx__) < 3);
            {
                var __v__3 = new global::System.Activities.Statements.Sequence();
                var s = new global::System.Activities.Variable<string>();
                __v__3.Variables.Add(s);
                var dt = new global::System.Activities.Variable<DateTime>();
                __v__3.Variables.Add(dt);
                var __v__4 = new global::MetahWActionActivity(__ctx__ =>
                {
                    s.SetEx(__ctx__, "hello");
                }

                );
                __v__3.Activities.Add(__v__4);
                {
                    var __v__5 = new global::System.Activities.Statements.Parallel();
                    var __v__6 = new Activity2().Initialize(__activity__ =>
                    {
                        __activity__.Arg1 = new global::System.Activities.InArgument<int>(new global::MetahWFuncActivity<int>(__ctx__ => i.Get(__ctx__)));
                        __activity__.Arg2 = new global::System.Activities.InOutArgument<string>(new global::MetahWLocationActivity<string>(s));
                        __activity__.Result = new global::System.Activities.OutArgument<global::System.DateTime>(new global::MetahWLocationActivity<global::System.DateTime>(dt));
                    }

                    );
                    __v__5.Branches.Add(__v__6);
                    var __v__7 = new global::System.Activities.Statements.Delay();
                    __v__7.Duration = new global::System.Activities.InArgument<global::System.TimeSpan>(new global::MetahWFuncActivity<global::System.TimeSpan>(__ctx__ => TimeSpan.FromSeconds(2)));
                    __v__5.Branches.Add(__v__7);
                    __v__3.Activities.Add(__v__5);
                }

                var __v__8 = new global::MetahWActionActivity(__ctx__ =>
                {
                    Console.WriteLine("S: {0}, DT: {1}", s.Get(__ctx__), dt.Get(__ctx__));
                    i.SetEx(__ctx__, __val__ => ++__val__, true);
                }

                );
                __v__3.Activities.Add(__v__8);
                __v__2.Body = __v__3;
            }

            __v__0.Activities.Add(__v__2);
            __vroot__ = __v__0;
        }

        return __vroot__;
    }

    private global::System.Func<global::System.Activities.Activity> __implementation__;
    protected override global::System.Func<global::System.Activities.Activity> Implementation
    {
        get
        {
            return __implementation__ ?? (__implementation__ = __GetImplementation__);
        }

        set
        {
            throw new global::System.NotSupportedException();
        }
    }
}

class Activity2 : global::System.Activities.Activity<DateTime>
{
    public global::System.Activities.InArgument<int> Arg1
    {
        get;
        set;
    }

    public global::System.Activities.InOutArgument<string> Arg2
    {
        get;
        set;
    }

    private global::System.Activities.Activity __GetImplementation__()
    {
        global::System.Activities.Activity __vroot__;
        var __v__0 = new global::MetahWActionActivity(__ctx__ =>
        {
            Console.WriteLine("Arg1: {0}, Arg2: {1}", Arg1.Get(__ctx__), Arg2.Get(__ctx__));
            Arg2.SetEx(__ctx__, Arg2.Get(__ctx__) + " world");
            Result.SetEx(__ctx__, DateTime.Now);
        }

        );
        __vroot__ = __v__0;
        return __vroot__;
    }

    private global::System.Func<global::System.Activities.Activity> __implementation__;
    protected override global::System.Func<global::System.Activities.Activity> Implementation
    {
        get
        {
            return __implementation__ ?? (__implementation__ = __GetImplementation__);
        }

        set
        {
            throw new global::System.NotSupportedException();
        }
    }
}
//helper code omitted
Sep 18, 2014 at 2:55 PM
Edited Nov 2, 2014 at 8:38 AM
KathleenDollard wrote:
Are you working directly between your expanded syntax and Roslyn, or have you created an abstraction layer for the language? I am in no way suggesting that is the appropriate approach. But if you have, I want to ask you about it.
I use Roslyn API directly. Roslyn API is really good despite a little inconvenience. I have no idea about an abstraction layer(RoslynDOM).
Sep 18, 2014 at 4:38 PM
You say:

"In meta-thinking, an activity is a function; for implementation, an activity is a C# class."

I am struggling with my "elevator speech" (which is a short explanation of what I'm trying to do). I did not have "meta-thinking/implementation" in my wording. Thank you for this great set of words.

(Perhaps your thinking deliberately about English is good for all of us).

Kathleen
Sep 18, 2014 at 4:45 PM
I grew frustrated with my direct use of the Roslyn API and built an abstraction layer because the "little inconveniences" overwhelmed me.

I believe your rules ("//MW variable declarator cannot have initializer(eg: int i = 0)") are a good idea, but for what I wanted, almost all valid C# code should be available, and the "little inconveniences" become seemed to multiply.

The abstraction layer has consumed my time, dragging me away from my intent - a better template language. I am glad you are getting your full system operational.

Can you describe how you anticipate someone debugging their activities?
Sep 18, 2014 at 5:06 PM
Hi Kathleen,
Metah is just a metaprogramming experiment, no intellisense, no debugging support, no...
I think you should first concentrate on a concrete problem domain(eg: OData v4 metaprogramming, Entity framework metaprogramming...) instead of inventing a fancy abstraction layer.
Sorry, English is hard for me :(