This project is read-only.

Is an expression-bodied function both an Expression<Func<..>> and Func<..>?

Topics: C# Language Design
Dec 30, 2014 at 9:15 AM
Hello everyone!

While using Entity Framework, in particular its LINQ-to-SQL conversion, I had a thought that it would be really cool if we could write one member function on an entity and then use it both in DB queries and in-memory.

A dumb example, User.IsOnline property, which refers to some other field:
public bool IsOnline => this.ActiveSessionId != null;
Or, maybe a trickier one, function HasRight:
public bool HasRight(RightsEnum right) => this.Roles.Any(r => r.HasRight(right));
and then you define in the role:
public bool HasRight(RightsEnum right) => this.Rights.Contains(right);
I guess this one hints at problems with potential recursion.

Basically, can the expression-bodied function above be used inside an Expression<Func<User, bool>> without being placed there as just a method ref, but rather its internals somehow "inlined" (or at least "inspectable"?..), thus making it simultaneously and transparently translatable to DB queries and usable in in-memory tests?

Is this possible? Is it even doable? Does it make any sense? :)
Dec 30, 2014 at 11:44 AM
This is a question better suitable for http://entityframework.codeplex.com/discussions

Unless you have a concrete suggestion/problem related to the language/compiler.
Dec 30, 2014 at 1:19 PM
Maybe you should read the question again? ;)

It is definitely about a language feature, I don't know how you missed it.
Dec 30, 2014 at 1:28 PM
This does fall under the compiler's purview because it is the compiler that generates the expression trees; EF only interprets them.

So, what you're basically asking is that for the following simple POCO:
public class Person {
    public byte Age { get; set; }
    public bool IsMinor => Age < 18;
}
That when the compiler is emitting an expression that it can recompile and inline the IsMinor member as a fragment of the expression tree rather than just referencing the member, which is what it does today:
var query = from person in db.People
    where person.IsMinor
    select person;

/*
Produces:
.Call System.Linq.Queryable.Where(
    .Constant<System.Linq.EnumerableQuery`1[Person]>(Person[]),
    '(.Lambda #Lambda1<System.Func`2[Person,System.Boolean]>))
.Lambda #Lambda1<System.Func`2[Person,System.Boolean]>(Person $person) {
    $person.IsMinor
}

But you'd want:
.Call System.Linq.Queryable.Where(
    .Constant<System.Linq.EnumerableQuery`1[Person]>(Person[]),
    '(.Lambda #Lambda1<System.Func`2[Person,System.Boolean]>))
.Lambda #Lambda1<System.Func`2[Person,System.Boolean]>(Person $person) {
    $person.Age < 18
}
*/
My question is how would the compiler know to do this? These members would effectively need to be compiled twice, once to IL and again to an expression tree fragment. In any web application on which I've worked and whereEF or LINQ to SQL was employed the POCOs were kept in separate assemblies and isolated from the repository/controller code. As such the compiler wouldn't be capable of recompiling the members on-demand within a single assembly. Even if that were an expected limitation it would make the solution brittle to refactoring, but you wouldn't know about it until runtime when the expressions failed to be converted.

Also, even if such a feature were to be implemented you'd still not reach behavior parity between EF and in-memory. For example, take your HasRight example method. If either Roles or Rights were null it would throw an ArgumentNullException. However, in EF it would run just fine and return false.
Dec 30, 2014 at 1:43 PM
If an Invoke method was added to LambdaExpression that cached the delegate (BCL), it could be used to invoke the expression.

EF (EF) could, instead, expand the body of the expression.

None of which requires language changes.

Or are you asking for C-like #defines or C++-like templates?
Dec 30, 2014 at 2:01 PM
Edited Dec 30, 2014 at 2:16 PM
Halo_Four wrote:
That when the compiler is emitting an expression that it can recompile and inline the IsMinor member as a fragment of the expression tree rather than just referencing the member, which is what it does today:
Yup.
My question is how would the compiler know to do this?
Well, I was thinking maybe just the expression-bodied properties and methods could work this way? Taking the purely cosmetic similarity between them and lambdas a bit deeper.
... it would make the solution brittle to refactoring, but you wouldn't know about it until runtime when the expressions failed to be converted.
This is a good point! A lambda using an expression-bodied function is not different to one using an ordinary function. If the expression-bodied function changes to an ordinary one, suddenly lambdas in other projects become different. However, maybe this risk could be tolerated? Same risk exists when using any Expression declared publicly in another assembly...
Also, even if such a feature were to be implemented you'd still not reach behavior parity between EF and in-memory. For example, take your HasRight example method. If either Roles or Rights were null it would throw an ArgumentNullException. However, in EF it would run just fine and return false.
This is pretty EF specific :) Anyway, this problem exists regardless of this feature. Many expressions can work just fine in DB, and throw null ref exceptions in-memory. E.g. expression that traverse references can cross any reference they like when compiled to SQL, SQL will not throw a null reference exception. In-memory, particularly if you don't use lazy loading (and let's face it, no one should use it :D), these things easily throw. Ultimately, SQL has tri-state logic, it is impossible (or impractical) to have full parity.
Dec 30, 2014 at 2:16 PM
Edited Dec 30, 2014 at 2:18 PM
(at) PauloMorgado - If you had a public property, of type Expression, then yes, you could easily Invoke() it and use it where an Expression<> is needed. And just caching the compiled version would make this efficient. However, to construct a new lambda which uses that expression, e.g. which composes two or more expressions into one, would be hard and ugly to look at.

What I mean is having expression-bodied functions and properties (one of the new features in C# 6) behaving like they are both an ordinary function and an Expression<>, transparently! So that if we have two of them, on some Person class (which has Age and Name properties), and they're defined like:
public bool IsMinor = Age < 18;
public bool NameBeginsWIthA = Name.StartsWith("A");
and I use them like:
Expression<Func<Person, bool>> test = p => p.IsMinor && !p.NameBeginsWithA && p.Country = "Iceland";
...that whoever is visiting the compiled expression "test" can also visit the IsMinor and NameBeginsWithA properties. "Expanding" them in place is not really necessary, the bare minimum is that these method/property calls are somehow special in that they can be regarded as simple method calls by whoever is visiting the expression, or the visitor can traverse inside them too. Then a library like EF could take advantage of this to traverse them and see that it can translate IsMinor and NameBeginsWithA into SQL, and avoid loading every person from Iceland into memory (if it would even be able to figure out the Iceland bit).

And of course, if the expression is compiled to IL and invoked, then the IsMinor and NameBeginsWithA are simple method calls in the generated IL.


BTW, why is it not possible to use the (at) character, aka monkey, in text here? Kinda annoying.
Dec 30, 2014 at 9:26 PM
This is an interesting feature request and I'm often facing the problem you are trying to solve. I wouldn't exactly inline the referenced lambda-member, instead I'd emit factory methods that would build the appropriate expression for the invoked member. This would then work across assembly boundaries. Like so:
public bool IsMinor => this.Age < 18;
would become:
[UseFactoryMethodForExpressions(nameof(IsMinor_Factory))]
public bool IsMinor
{
    get { return this.Age < 18; }
}
public static Expression IsMinor_Factory(Expression self)
{
    return Expression.LessThan(Expression.Property(self, nameof(Person.Age)), Expression.Constant(18));
}
Then at the call site instead of having an Expression.Property(somePerson, "IsMinor") you can have an IsMinor_Factory(somePerson) which generates the appropriate expression from the argument. This also works with methods, you just add more arguments.

All of this you can do without changing compiler:
  1. decorate your extensibility points with a custom attribute, then wrap your queries with a custom IQueryProvider that walks the expression tree and replaces all MemberExpression instances where the member is marked by the hypothetical UseFactoryMethodForExpressionsAttribute by the expression returned by invoking the referenced factory method with the member call arguments (adding the instance argument to the front if translating an instance member). EF7 is coming with new hooks that allow you to inject your custom expression-processing before query execution, but even with a different vendor/technology wrapping an IQueryable with a custom IQueryProvider is definitely possible. I'v been using this technique in production for quite some time.
  2. write the expression factory methods by hand. You can reuse the original lambda with a custom expression visitor that replaces lambda parameters with the call arguments. This way it looks similar to the original member. You could do a pre-build step that parses your code, looks for the custom attribute (or whatever marker you decide on) and generates the code for the factory methods from the original methods for you to automate this. You can already do this with Roslyn and t4, not sure it's worth to mess with the compiler for this one either.
Dec 31, 2014 at 1:40 AM
I like the idea that compiler could generate Func<> and Expression<Func<>> for expression-bodied members.

But I really like the idea of using attribute meta-programming capabilities that was already proposed here that could provide more general solution for this scenario. The second example is what we are talking about here. I don't know what is the compiler team stance on this.