Java-like virtual extension methods/default interface implementations, aka traits or (ruby-like) mixins

Topics: C# Language Design
Jan 18, 2015 at 9:47 AM
I've just posted a new language feature request to UserVoide about adding default method implementations to interfaces, also known as "virtual extension methods", similar to traits or mixins in other languages. When writing frameworks, such a feature would often make much more sense than to provide an abstract base class with the default implementations because there is only one base class while interfaces can be combined with much more flexibility.

However, I'm wondering if this hasn't been discussed before, since it's a somewhat obvious feature now that Java 8 has it. I haven't found anything via Google or CodePlex, am I missing something?
Jan 18, 2015 at 3:04 PM
What would the benefit of these "virtual extension methods" be over the other extension methods that are in C# since version 3.0?
Jan 18, 2015 at 7:16 PM
I have to agree. In Java they were useful since there is no concept of extension methods and they were the only way to enable expanding upon existing interfaces without breaking a ton of existing code. But since default implementations need to be defined as a part of the interface itself they are significantly more limited than extension methods since nobody can provide additional methods from outside of the interface. Extension methods avoid having to muck with the runtime and avoid the entire diamond/MI issue which is now a real problem in Java.
Jan 19, 2015 at 7:06 AM
Sure, Java added them as an alternative to extension methods, but they really have a very different feature set, with different purposes.

Extension methods:
  • Allow methods to be added to existing interfaces from the outside.
  • Are great for adding APIs with default implementations that feel like instance methods to existing interfaces.
  • Are really static, so they do not provide a way to override behavior added by extension methods in an object-oriented sense.
  • Cause conflicts when a class has the same APIs both via an (interface-based) extension method and in the implementation. The compiler chooses to prefer the instance variant, so a class instance behaves differently depending on the static type of the variable on which the API is invoked.
Virtual extension methods:
  • Allow interfaces to bring a default implementation (similar to an abstract base class, but without the restriction on linear inheritance).
  • Are great for providing "right in 90% of the cases" APIs, while still allowing object-oriented extensibility.
  • Must be defined on the interface, so they do not provide a way to add behavior by a third-party component.
  • Cause conflicts when two interfaces bring default imlementations. The compiler could warn and choose one (like in Ruby; e.g., the last one in the interface list) or it could raise an error and have the developer implement the method (like in Java 8).
The classic way to provide default implementations while supporting extensibility in C# is to provide an abstract base class. This has a lot of restrictions because base classes can only follow one dimension (i.e., only one base class per class), so the classic second option is to provide the default implementation as a separate class and delegating to it. This second option, however, can be quite cumbersome if it's about more than just one interface member, and it's "mechanical" code that the compiler could automatically generate.

We actually had this situation recently about a feature of a framework a colleague of mine is implementing. That framework defines a base class hierarchy according to a certain "is a" dimension. Framework clients derive from those base classes. The framework additionally wants to offer other features to be mixed into those classes. Implementing this with normal extension methods is only possible as long as the other features never need to be adapted ("overridden"). In the sense of object-orientation, this is crippling - it doesn't allow us to be open for extension by subclasses. So composition and delegation was chosen instead, forcing (many) framework clients to write mechanical delegation code...

While there are workarounds (the extension method could perform a dynamic type check or reflection on its target and decide whether to delegate back to the target class), they are exactly that: workarounds. Languages like Ruby, Python, and, now, Java have mixin mechanisms for such scenarios, which solve the problem much more elegantly.

Virtual extension methods can, BTW, be a pure C# feature, they don't need runtime extensions (unless you want to be able to add interface members while retaining compatibility of compiled code). When compiling an interface with a default member, the C# compiler could move it to a separate (compiler-generated) class. When compiling a class implementing such an interface but omitting the default member, the C# compiler could auto-implement it to call into the compiler-generated class. A custom attribute on the interface member could be used as metadata describing the relationship between the interface member and its default implementation.
Jan 19, 2015 at 7:35 AM
FabianSchmied wrote:
Virtual extension methods can, BTW, be a pure C# feature, they don't need runtime extensions (unless you want to be able to add interface members while retaining compatibility of compiled code). When compiling an interface with a default member, the C# compiler could move it to a separate (compiler-generated) class. When compiling a class implementing such an interface but omitting the default member, the C# compiler could auto-implement it to call into the compiler-generated class. A custom attribute on the interface member could be used as metadata describing the relationship between the interface member and its default implementation.
How would that work with reflection for emitting types at run time?
Jan 19, 2015 at 8:52 AM
Reflection would work the same way. If you wanted to emit an interface with a default implementation, you could simply emit the default implementation into a static method of some generated class, then link it to the interface member by putting a custom attribute on that member. If you wanted to emit a class implementing an interface member with a default implementation, you could detect the presence of the custom attribute and generate IL calling the referenced static method.
Jan 19, 2015 at 12:42 PM
FabianSchmied wrote:
Reflection would work the same way. If you wanted to emit an interface with a default implementation, you could simply emit the default implementation into a static method of some generated class, then link it to the interface member by putting a custom attribute on that member. If you wanted to emit a class implementing an interface member with a default implementation, you could detect the presence of the custom attribute and generate IL calling the referenced static method.
That doesn't seem to be consistent with what you wrote before. Weren't you talking about a base class being generated?
Jan 19, 2015 at 1:52 PM
Edited Jan 19, 2015 at 1:53 PM
PauloMorgado wrote:
FabianSchmied wrote:
Reflection would work the same way. If you wanted to emit an interface with a default implementation, you could simply emit the default implementation into a static method of some generated class, then link it to the interface member by putting a custom attribute on that member. If you wanted to emit a class implementing an interface member with a default implementation, you could detect the presence of the custom attribute and generate IL calling the referenced static method.
That doesn't seem to be consistent with what you wrote before. Weren't you talking about a base class being generated?
No, I wrote this (shortened and added emphasis):
Virtual extension methods can, BTW, be a pure C# feature, they don't need runtime extensions [...]. When compiling an interface with a default member, the C# compiler could move it to a separate (compiler-generated) class. When compiling a class implementing such an interface but omitting the default member, the C# compiler could auto-implement it to call into the compiler-generated class.
So, to clarifywith an example, consider the following code:
interface ISomeInterface
{
  string Property { get; }

  default string Format()
  {
    return string.Format ("{0} ({1})", GetType().Name, Property);
  }
}

class SomeClass : ISomeInterface
{
  public string Property { get; set; }
}
This could be translated as if it had been written as follows:
interface ISomeInterface
{
  string Property { get; }

  [DefaultImplementation (typeof (ISomeInterface_DefaultMembers), Method = "...")]
  string Format();
}

class ISomeInterface_DefaultMembers
{
  public static string Format (ISomeInterface obj)
  {
    return string.Format ("{0} ({1})", obj.GetType().Name, obj.Property);
  }
}

class SomeClass : ISomeInterface
{
  public string Property { get; set; }

  public string Format()
  {
    return ISomeInterface_DefaultMembers.Format (this);
  }
}
Jan 20, 2015 at 3:50 PM
Such a feature could be extremely useful if it could allow an interface to add new members with default implementations, while being compatible with old implementations. Imagine, for example, how much more efficient many operations on collections could be if IEnumerator<T> could add:
int Move(int distance);

static int Move_Impl(ref IEnumerator<T> it, int distance)
{
  while(distance > 0 && it.MoveNext())
    distance--;
  return distance;
}
If IEnumerable<T>.Append() were used to combine a collection whose enumerator has an efficient alternate implementation for the above and one which does not, the collection returned by it would be able to advance quickly past the parts which support rapid skipping, and advance slowly over the parts which don't. Thus, for example, if code were to use Append() to add a 4-item iterator to a 999,997-item List<T>, and then wanted to read the millionth item of the collection, it could use one call to Move to skip over the 999,997 items of the List<T>, and a second to move three items ahead in the iterator. The latter call would then perform three MoveNext calls on the iterator. All in all a much more efficient sequence than making a total of a million MoveNext() calls.

Note that while it would be possible to define a new interface, and have a Move extension method attempt to invoke it using Reflection or try-casting, such an approach would likely be far less efficient than having the type loader auto-generate:
int IEnumerator<T>.Move(int distance) { IEnumerator<T>.Move_Impl(ref this, distance); }
for any class which claimed to implement IEnumerator<T> but did not implement the indicated method.

That would generally be a run-time rather than a language issue, however. The main language-related issue I would see would be how to handle the situation where a class claims to implement IEnumerator<T> and doesn't have any method marked in its metadata as implementing int IEnumerator<T>.Move(int), but does include a public method with that signature that isn't marked as an implementation. If C# had required that methods that implement interfaces be marked as doing so (as is the case in VB.NET) there would be no ambiguity, but the (IMHO unfortunate) design of implicit interface implementation could mean that recompilation of a valid program following an upgrade to one of the interfaces it uses could alter its semantics.
Jan 21, 2015 at 2:38 PM
I will say that I do like the idea to clean up the amount of useless boilerplate required to implement some interfaces like IEnumerable<T> and IEnumerator<T>. It would be nice if the non-generic members could be automatically explicitly implemented to call their generic counterparts and members like Reset (throw NotSupportedException) and Dispose (do nothing). If that also opened the gates towards supporting more common operations at that level where the implementation can provide optimized versions then that would be good as well.

The one way that default members are a definite improvement over extension methods is that by being properly virtual they would still be called even with a variable of a base class/interface. We'd still have to worry about the entire diamond/MI problem but I think since C# is more flexible about interface implementation than Java that we have more options that could be employed.

I'm personally of the opinion that changes like this belong in the run-time proper rather than strictly as a form of syntactic candy bouncing off of static methods.
Jan 21, 2015 at 10:58 PM
Halo_Four wrote:
The one way that default members are a definite improvement over extension methods is that by being properly virtual they would still be called even with a variable of a base class/interface. We'd still have to worry about the entire diamond/MI problem but I think since C# is more flexible about interface implementation than Java that we have more options that could be employed.
The diamond/MI problem could only arise in cases where an interface member is implemented somewhere other than the implementing class or a class associated with the original interface. I would suggest that requiring that default implementations must be defined in the interface itself would avoid the diamond/MI problem entirely; it would mean as a slight annoyance that one would either have to have implementations of IEnumerator<T> IEnumerable<T>.GetEnumerator chain to the non-generic form (rather than vice versa) or else have the default implementation of IEnumerator IEnumerable.GetEnumerator use Reflection to find and chain to a generic version if one exists. I've proposed some other slightly-more-sophisticated binding approaches elsewhere which would be a little more complexity in the run-time, but facilitate this extremely common case [and many others besides].

As noted, though, this is mainly a run-time issue rather than a language one, though the interaction of implicit interface implementations with new interface members should perhaps be considered.
Jan 25, 2015 at 2:29 PM
Edited Jan 25, 2015 at 2:29 PM
I've created a copy of this discussion at GitHub, now that Roslyn has moved: https://github.com/dotnet/roslyn/issues/73.

I'd like to add that while this feature is more powerful as a runtime feature, it also has its benefits as a pure language feature, which is much, much more realistic than a runtime change. The main difference is whether adding members with a default implementation to an interface is seen as binary compatible or only source compatible (requiring recompilation of implementers). Since my main focus is that of having "overridable default interface implementations", I'd be very happy with the pure language/syntactic sugar version.