This project is read-only.

F#'s Type Providers in C#?

Topics: C# Language Design, VB Language Design
May 18, 2014 at 8:27 PM
Historically, the previous versions of C# tackled huge problems with simple and elegant language-based solutions: generics, linq, dynamic types, async/await, etc.

I was hoping that The Next Big Thing to tackle at the language level was to somehow bring F#'s type providers into the mainstream .NET languages. Here is something huge which is ripe for solving at the language level. How nice would it be to simply hook up your code with a remote library in one line of code, and for its world of types to be immediately available in IntelliSense without any code-gen magic?

Are there any plans for this, or if it's been intentionally shelved for the time being, why?
May 21, 2014 at 8:06 AM
gzak wrote:
Historically, the previous versions of C# tackled huge problems with simple and elegant language-based solutions: generics, linq, dynamic types, async/await, etc.

I was hoping that The Next Big Thing to tackle at the language level was to somehow bring F#'s type providers into the mainstream .NET languages. Here is something huge which is ripe for solving at the language level. How nice would it be to simply hook up your code with a remote library in one line of code, and for its world of types to be immediately available in IntelliSense without any code-gen magic?
This was prototyped in C-omega about 10 years ago.

The primary block as I see it is that the strategy for producing a data class varies by data source and use case both: serialization, live-ness of member writes, bindability, etc.

Given an interface description (even an interface type), and a strategy, implementing types at is surprisingly not that hard, but you still need to specify. This leads us back to a suggestion raised by myself and a few others after PDC2005: syntax to specify a default implementation of an interface, using a strategy. For example:
var x = new IFoo(); // simple strategy of autoprops and NotImplemented methods
var y = new IFoo(TStrategy); // use strategy to construct the implementation
The latter example could be demonstrated:
var z = new IFoo(DependencyObjectStrategy); // create something useful for WPF
var i = new IFoo(INotifyPropertyChangedStrategy); // implements with INPC support
Then we get into the possibility of composition:
class Foo: IFoo(DependencyObjectStrategy) { ... }
What would be needed for your case is an appropriate strategy that fits the actual need of the application being written, to be applied to the schema the endpoint exposes.
May 24, 2014 at 5:00 AM
One thing I've advocated for awhile would be the ability for interfaces or their members to specify an affinity for one or more static types, with a few notable features:

-1- Members of the static types could be used as though they were members of the interface (avoiding the need to have interface helpers like like IEnumerator<T>.Empty or Comparer<T>.Default specify a class name distinct from the interface name.

-2- Have the class loader accept classes or structures which claim to implement interfaces but do not implement all the members, provided that the class loader can produce for every member that isn't implemented an implementation of the form
int ISomeInterface.someMethod(int someParam) { return designatedStaticClass.someMethod(this, someParam); } // For classes
or
int ISomeInterface.someMethod(int someParam) { return designatedStaticClass.someMethod(ref this, someParam); } // For structs
for some static class designated in an interface or member attribute [using the first class containing an acceptable overload]

Double-diamond issues could be avoided by specifying that interface members' static helper classes must be listed within interfaces that actually define them, rather than those that inherit them, but allowing for the possibility that e.g. a non-generic IEnumerable could designate a class with a helper method
IEnumerator IEnumeratorHelper.GetEnumerator<T>(IEnumerable<T> it) { return it.GetEnumerator(); }
which would allow the non-generic IEnumerable.GetEnumerator() to be auto-implemented by types which implement IEnumerable<T>. Even though the static method would only be usable by implementations of IEnumerable<T>, it would be defined within IEnumerator.

Allowing interfaces to specify default implementations for members that could be auto-generated at runtime would help solve one of the major limitations of interfaces--the inability to have future versions extend their abilities without breaking existing implementations. If IEnumerable and IEnumerator could include default implementations for members, then it would be possible to add things like Count, Snapshot, Abilities [a property saying if and how well various abilities are supported], int Move(int) [an IEnumerator<T> method which would be equivalent to calling MoveNext a specified number of times, but could be orders of magnitude faster) etc, Not only would this eliminate the need for Reflection when using extension methods like Count, but more importantly it would allow composite enumerables (such as one returned from the Concat extension method) to behave much more efficiently. For example, if one uses Concat to join a million-item List to a five item iterator, calling Count on the result of such concatenation should require enumeration of the five-item iterator, but should not require enumeration of the million-item list. Likewise, calling SnapShot on a concatenation of two collections should call SnapShot on each; if each collection's SnapShot methods returns itself, the union's SnapShot should return the existing union; otherwise it should return the union of the snapshots [so calling SnapShot on a concatenation of a million-item immutable list and a five-item mutable list would only require creating a new five-item immutable list along with a new wrapper; calling SnapShot on that should simply return itself).

Many of the methods in Linq could be efficiently implemented if IEnumerable<T> and IEnumerator<T> which were capable of a few operations beyond enumeration could expose their abilities. Unfortunately, there are no composable abilities other than enumeration, thus causing many operations to be orders of magnitude slower than they would otherwise need to be.
May 24, 2014 at 1:02 PM
While I think that default interface members are interesting I don't find them all that particularly useful. I think it was something that Java did in order to wade into the world of LINQ while trying to avoid actually implementing extension methods. The big problem I see with them, though, is that they can only be added by the author of the original interface, so for all of these additional features you'd be at the mercy of the BCL team. And the default implementation is limited to whatever public members are exposed by the interface anyway so they wouldn't be able to carry any real intelligence.

As for allowing an instance to define their capabilities above and beyond the base implementation of an interface I like having that instance also implement additional interfaces for each of those abilities. An extension method then could negotiate the implemented interfaces and use any helper members. The Count extension method already does this if the enumerable implements either ICollection<T> or ICollection. That doesn't solve your issue with concatenated enumerable because nobody defined an interface to represent a series of concatenated enumerable which Count could then also negotiate. There's nothing stopping the addition of that through ConcatEx and CountEx extension methods, although of course to "fix" LINQ proper would also require the BCL team.
May 24, 2014 at 5:39 PM
Halo_Four wrote:
While I think that default interface members are interesting I don't find them all that particularly useful. I think it was something that Java did in order to wade into the world of LINQ while trying to avoid actually implementing extension methods. The big problem I see with them, though, is that they can only be added by the author of the original interface, so for all of these additional features you'd be at the mercy of the BCL team. And the default implementation is limited to whatever public members are exposed by the interface anyway so they wouldn't be able to carry any real intelligence.
Even if limited to the author of the interface, moving IEnumerable extension methods into the interface could greatly improve their performance. The default implementations couldn't offer a huge amount of intelligence (though they could offer some type-specific intelligence, as with the method illustrated method which would add a default IEnumreable.GetEnumerator() only to types that implement IEnumerable<T> for a single type T), but if IEnumerable<T> included more methods, then default implementations could use those methods and behave intelligently when used in scenarios involving composition.
As for allowing an instance to define their capabilities above and beyond the base implementation of an interface I like having that instance also implement additional interfaces for each of those abilities. An extension method then could negotiate the implemented interfaces and use any helper members. The Count extension method already does this if the enumerable implements either ICollection<T> or ICollection. That doesn't solve your issue with concatenated enumerable because nobody defined an interface to represent a series of concatenated enumerable which Count could then also negotiate. There's nothing stopping the addition of that through ConcatEx and CountEx extension methods, although of course to "fix" LINQ proper would also require the BCL team.
The problem with that approach is that there's no nice way for something like Concat to expose any special abilities of the collections being concatenated. If one concatenates two collections that are efficiently addressable by index, and the length of the first is immutable, the resulting collection should be efficiently addressable by index, but if the only types which are supposed to implement an interfaced getter are those which promise to implement it efficiently, the only way the concatenated collection could offer the ability would be if the Concat method determined--before it returned--that it should offer that ability. Once the Concat method returns an object which doesn't implement an indexed getter, that would forever foreclose the possibility of an indexed get. By contrast, if there was a property that returned an EnumeratorAbilities object that could be queried for IsKnownFixedSize and IsEfficientlyReadableByIndex, those properties could be lazily evaluated if and when client code needed them.
May 24, 2014 at 7:40 PM
supercat wrote:
Halo_Four wrote:
While I think that default interface members are interesting I don't find them all that particularly useful. I think it was something that Java did in order to wade into the world of LINQ while trying to avoid actually implementing extension methods. The big problem I see with them, though, is that they can only be added by the author of the original interface, so for all of these additional features you'd be at the mercy of the BCL team. And the default implementation is limited to whatever public members are exposed by the interface anyway so they wouldn't be able to carry any real intelligence.
Even if limited to the author of the interface, moving IEnumerable extension methods into the interface could greatly improve their performance. The default implementations couldn't offer a huge amount of intelligence (though they could offer some type-specific intelligence, as with the method illustrated method which would add a default IEnumreable.GetEnumerator() only to types that implement IEnumerable<T> for a single type T), but if IEnumerable<T> included more methods, then default implementations could use those methods and behave intelligently when used in scenarios involving composition.
As for allowing an instance to define their capabilities above and beyond the base implementation of an interface I like having that instance also implement additional interfaces for each of those abilities. An extension method then could negotiate the implemented interfaces and use any helper members. The Count extension method already does this if the enumerable implements either ICollection<T> or ICollection. That doesn't solve your issue with concatenated enumerable because nobody defined an interface to represent a series of concatenated enumerable which Count could then also negotiate. There's nothing stopping the addition of that through ConcatEx and CountEx extension methods, although of course to "fix" LINQ proper would also require the BCL team.
The problem with that approach is that there's no nice way for something like Concat to expose any special abilities of the collections being concatenated. If one concatenates two collections that are efficiently addressable by index, and the length of the first is immutable, the resulting collection should be efficiently addressable by index, but if the only types which are supposed to implement an interfaced getter are those which promise to implement it efficiently, the only way the concatenated collection could offer the ability would be if the Concat method determined--before it returned--that it should offer that ability. Once the Concat method returns an object which doesn't implement an indexed getter, that would forever foreclose the possibility of an indexed get. By contrast, if there was a property that returned an EnumeratorAbilities object that could be queried for IsKnownFixedSize and IsEfficientlyReadableByIndex, those properties could be lazily evaluated if and when client code needed them.
Sure, I just don't see why default implementations solve that any better than a combination of extension methods and "ability" interfaces do. The real problem isn't what you can declare and support via interfaces (or default implementations), it's that the composed result from any extension method like Concat is a completely different iterator which wasn't written to return a type that can also declare it's capabilities or automatically implement that default member, so even with default interface implementation you'd still be executing the default implementation, which would still follow the non-efficient code path. Every single composable iterator would have to knowingly interrogate the capabilities and compose a new object which also supported those capabilities. Nothing stops Concat from accepting two IEnumerable<T>, determining that both are actually IList<T> and creating a new enumerable which also implements IList<T>. If Concat still returned a new IEnumerable<T> which did not provide it's own optimized implementation of the default member you're still back in the same exact boat.

However, I stand by that the much more flexible manner of handling this is via interface interrogation and writing intelligent extension methods as you aren't locked into whatever the BCL team has decided would be appropriate for IEnumerable<T> to declare as capabilities, which would certainly move at a much slower pace than any outside developer could. I pity the folks using Java when they realize that there are probably some more functionality that they would like to see in Iterable<?> knowing how long the JCP process will take in order to consider it.
May 24, 2014 at 9:38 PM
Halo_Four wrote:
If Concat still returned a new IEnumerable<T> which did not provide it's own optimized implementation of the default member you're still back in the same exact boat.
If Concat returns an implementation of IEnumerable<T> which doesn't provide specialized implementations of methods like Count, then such methods could not be performed efficiently. On the other hand, I see no reason to expect that if methods like Count and Snapshot were added to IEnumerable<T>, the Concat method in the framework wouldn't be modified to return an object which could implement such methods in such fashion as to take advantage of specialized abilities of the underlying types.

Actually, the first change I'd make would be to IEnumerator--a method int Move(int n) whose default implementation would be:
public static int Move<T>(T it, int n) where T:IEnumerator // One of four methods--for generic and non-generic classes and structures
{
  while(n > 0)
  {
    if (!it.MoveNext())
      break; // Return count of moves *not* processed
    n--;
  }
  return n;
}
Adding that primitive alone would allow many implementations of IEnumerable to behave much more efficiently with many kinds of Linq operations. Something like Count could try requesting a 2.1-billion-item move and subtracting from 2.1 billion the number of items remaining to move. A "capabilities" property would be helpful to indicate whether "tail" could be more efficiently implemented by creating one enumerator, skipping to the end (to get get count), creating a second, skipping to the right point, and reading the remainder, or whether it would be more efficient to iterate through once into a circular buffer.
However, I stand by that the much more flexible manner of handling this is via interface interrogation and writing intelligent extension methods as you aren't locked into whatever the BCL team has decided would be appropriate for IEnumerable<T> to declare as capabilities, which would certainly move at a much slower pace than any outside developer could. I pity the folks using Java when they realize that there are probably some more functionality that they would like to see in Iterable<?> knowing how long the JCP process will take in order to consider it.
Conditionally casting to an interface which a reference may or may not implement is slower than unconditionally invoking a method which the interface is known to implement. Your concern about whether the BCL team would add things as fast as would be ideal is certainly notable, but on the flip side, I think it would be better to have IEnumerable<T>/IEnumerator<T> add a few methods, and have programmers who write implementations start supporting them, than to have everyone under the sun define their own interface which includes int Move(int), with zero interoperability between them.

Further, if one doesn't add a "capabilities" property to IEnumerable<T> or IEnumerator<T>, by what means could an infinite collection indicate that something like ToArray() should fail immediately, without trying to run the system out of memory first? If there were a standard way for querying an IEnumerable<T> whether it knew itself to be infinite, knew itself to have a fixed finite count, knew itself to be completely enumerable (but not necessarily with an immutable count), or didn't know anything about its count, then a method like ToArray() could behave appropriately once infinite IEnumerable<T> implementations started implementing it. Without such an ability, I can't think of any reasonable way for code to know whether any particular type "supports" IEnumerable<T>.ToArray().