VB LDM - interface ambiguity

Topics: VB Language Design
Coordinator
Oct 31, 2014 at 7:10 AM
Edited Oct 31, 2014 at 6:24 PM

Back-compat break with interface implementations

High-level summary

  • On the declaration side, C# users always used to be able to declare overloads of ambiguous interface methods. We are adding this ability also to VB for VS2015.
  • On the consumption side, C# language is more "hide-by-name" while VB supports both "hide-by-name" and "hide-by-sig".
  • In such cases VB used to try to emulate the C# behavior when meta-importing such declarations. Now it will follow the more VB behavior.
  • This will cause some small breaking changes. The breaking cases are limited to where...
  • (1) Library code was authored in C#
  • (2) In that library an interface ID inherits from two others IA,IB and declares a more general overload of a method in each of them
  • (3) VB consumers of that library used to pick the more general overload (copying C#'s hide-by-name behavior). But now they will pick the more specific overload (using VB's hide-by-sig behavior)
  • (4) If there were ambiguous specific overloads then it will be a break for VB consumers of the C# library.

Background

Interface A
  Sub M(x As Integer)
End Interface

Interface B
  Sub M(x As String)
End Interface

Interface C : Inherits A,B
  Overloads Sub M(x As Char)
End Interface

Interface D : Inherits A, B
End Interface
DECLARATION: Declaration C" had been illegal in VB up to VS2013; you could only declare M in C with 'Shadows, or meta-import it from C#. But VB Roslyn made a change... it now lets you declare such things.

CONSUMPTION: We never came up with a policy around CONSUMPTION of such things.
  • VS2013: In the type "C" if you try to access "M", the compiler doesn't see "M" from A and B. It only sees it from C. Note: you can meta-import C, but can't declare it.
  • VS2013: In the type "D" if you try to access "M", the compiler gives a name-lookup ambiguity error prior to even attempting overload resolution.
  • Roslyn as of May 2014: In both "C" and "D" if you try to access "M", the compiler gives name-lookup ambiguity error prior to even considering overload resolution.

Request

We'd like to be able to invoke "M" in both "C" and "D". But we'd also like to preserve backwards compatibility. It's a conundrum.


Note that there's a similar issue with Modules:
Module Module1
  Function Sin(d As Double) As Double

Modle Module2
  Function Sin(s As Single) As Single

' this callsite is in a place that imports both modules' containing namespaces
Dim x = Sin(1.0) ' VB currently makes this an ambiguity. 
VB makes this invocation of Sin an ambiguity. But when you do the equivalent in C# using "static using", it gathers together all candidates from all usings, and then performs overload resolution on them. This case seems very minor. Not worth worrying about.
Coordinator
Oct 31, 2014 at 6:22 PM

VB LDM 2014.04.16

PROPOSAL1: revert change that enabled declaration in C. (but doesn't help consumption of meta-imported C, nor consumption of D).

PROPOSAL2: silently treat it as shadowing at the callsite, consistent with native compiler. (but doesn't help consumption of D).

PROPOSAL3: Allow at lookup, i.e. merge them all, and leave it to overload resolution. (This is the proposal that we picked). This is a small breaking change. It will break code which used to meta-import "C", and invoked "M" in it, and always got the version of "M" in "C". Now with this change, it might get the version of "M" from "A" or "B". We think this is a minor breaking change, because (1) it only applied to meta-import situations, (2) it will now behave as the author intended. Note that this requires a change to VB spec 4.3.2.

PROPOSAL4: combination of 1 and 2, i.e. completely duplicate behavior of native compiler.
Coordinator
Oct 31, 2014 at 6:22 PM
Edited Oct 31, 2014 at 7:22 PM

VB LDM 2014.10.08

We revisited the previous conclusion in the light of implementation. The related bug numbers are:
https://roslyn.codeplex.com/workitem/34
Internal vstfdevdiv bug 527642

The issue is that the C# language has hide-by-name lookup rules, and because of this it doesn't matter whether it meta-exports its signatures as "hide-by-name" or "hide-by-sig"; that difference is ignored. It happens to export them as "hide-by-sig".


Here's a representative example of the kind of breaks we'll now get under PROPOSAL3. There is a library that was authored in C#:
interface IA {void m(int x);}
interface IB {void m(int x);}
interface ID : IA,IB {void m(object x);}
C# users of the library are able to use it fine:
ID d = null;
d.m(1);  // picks the "object" overload
However VB users of the library will now experience a break
Dim x As ID = Nothing
x.m(1) ' VS2013 used to pick 'Object' but under Proposal3 will now give an ambiguity error
x.m(CObj(1)) ' this is the workaround to use in this case
The breaking cases are limited to where C# had ambiguous methods in unrelated interfaces, and overrode them with a more general overload in the derived interface ID. We imagine this breaking case will be uncommon.



There's a more common related scenario which WILL CONTINUE to work just fine. Let's walk through it:
interface IJson {string GetName();}
interface IComponent {string GetName();}
interface IPatient : IJson,IComponent {}
I am a C# author. I am taking in two libraries that provide IJson and IComponent. They are unrelated by they happen to have a method which shares a signature and has more or less the same behavior. I write it as above. However, within my C# use of my library, I discover a problem straight away: IPatient y = ...; y.GetName(); This code will give a compile-time ambiguity error because it doesn't know which unrelated overload of GetName to use. So I fix up my C# library as follows, to silence up the compiler:
interface IPatient : IJson,IComponent {new string GetName();}
With this change, the same C# code IPatient y = ...; y.GetName(); will work just fine. The reason it works in C# is because C# language uses hide-by-name lookup rules, and finds IPatient.GetName, and is happy with that.
Dim y As IPatient = Nothing
y.GetName() ' correctly picks IPatient.GetName
Fortunately the VB consumers of the library will also work great. In VS2013 it worked because VB faked up hide-by-name semantics on meta-import and so got the derived class. Under Proposal3 it will work because VB honestly uses hide-by-sig semantics and in this case (unlike ID.m(object) above) the hide-by-sig semantics pick the correct method.


How did VS2013 VB fake up "hide-by-name" semantics? Well, upon meta-import, it did a "shadowing trick": if it encountered an interface like C, it knew that such an interface could not have been declared by VB, and therefore it must have been declared in C#, and C# language had hide-by-name (shadowing) semantics, and so VB ignored the "hide-by-sig" attribute on the metadata and imported it as hide-by-name.


We went back on our resolution. We decided to forbid declaration of interfaces like "C". By forbidding it, we open the door to pulling the same "shadowing" trick as VS2013 did on meta-import. Actually the Roslyn compilers have a cleaner design and can't pull the trick on meta-import; however they can pull it on name lookup.

In other words, we resolved to go for PROPOSAL4.
Coordinator
Oct 31, 2014 at 6:22 PM
Edited Oct 31, 2014 at 7:25 PM

VB LDM 2014.10.08

Gosh this is difficult. In trying to replicate the behavior of VS2013, Aleksey discovered some pretty weird bugs in the way VS2013 applied its shadowing heuristics.
' BUG NUMBER 1 IN VS2013

Interface IA(Of T)
    Sub M1(x As T)
End Interface

Interface IB
    Inherits IA(Of Integer), IA(Of Short)
    Overloads Sub M1(x As String) ' VS2013 doesn't report an error
End Interface

Interface IC1
    Sub M1(x As Integer)
End Interface

Interface IC2
    Sub M1(x As Short)
End Interface

Interface ID
    Inherits IC1, IC2
    Overloads Sub M1(x As String) ' VS2013 reports error BC31410 "Overloading methods declared in multiple base interfaces is not valid"
End Interface

Module Module2
    Sub Test(x As IB)
        x.M1(1) ' VS2013 reports error BC30685: 'M1' is ambiguous across the inherited interfaces 'IA(Of Integer)' and 'IA(Of Short)'
    End Sub

    Sub Test(x As ID)
        x.M1(1) 'ID
    End Sub
End Module  
What's strange in this bug number 1 is that you expect the declaration error BC31410 to be reported in both cases IB and ID. However VS2013 only reports a declaration error for ID. Instead it reports a consumption error for IB.
' BUG NUMBER 2 IN VS2013

Interface IA
    Sub M1()
End Interface

Interface IB
    Inherits IA
    Overloads Sub M1(x As Integer)
End Interface

Interface IC1
    Inherits IA, IB
    Overloads Sub M1(x As String) ' VS2013 reports error BC31410 "Overloading methods declared in multiple base interfaces is not valid."
End Interface

Interface IC2
    Inherits IB, IA
    Overloads Sub M1(x As String) ' VS2013 reports error BC31410 "Overloading methods declared in multiple base interfaces is not valid."
End Interface

Interface IC3
    Inherits IB
    Overloads Sub M1(x As String) ' VS2013 reports no error
End Interface
What's strange about this is that interfaces IC1, IC2 and IC3 are semantically equivalent: whether or not you explicitly declare that you inherit from IA is irrelevant, since it's implied. So the compiler shouldn't be reporting a declaration error in IC1/IC2, just as it doesn't report a declaration error in IC3.

This means that we have to refine our idea of Proposal 4:

PROPOSAL4a: disallow declaration C by reimplementing all the weird quirks of VS2013, and use implicit shadowing upon name lookup. We rejected this as too difficult: Roslyn has very different code paths, so there's a high risk that we'd get different behavior from VS2013.

PROPOSAL4b: disallow declaration C in a clean way without any of the weird quirks of VS2013.

The question is: out of Proposal3 and Proposal4b, which will be the most painful break?

Proposal4b will hurt people who declare things that used to be okay under the old quirky rules but are no longer okay. We haven't discovered many cases yet.

Proposal3, as we said before, will hurt people who invoked x.m(1) in the case identified above. Their workaround is to change the callsite with an explicit cast. There is precedent for this: the overload resolution rules between VB and C# are different, and VB people have to put explicit casts in other callsites.

COM is an interesting case where you generally have IFoo1 and IFoo2 where IFoo2 duplicates (doesn't inherit) the methods of IFoo but also adds a few others. However we don't see this as a problem. It's very nontypical to declare a third interface IBar which inherits both IFoo1 and IFoo2 in COM scenarios.

In the light of this new evidence, we prefer Proposal3. It will be a better cleaner future for the VB language to declare things like this. And it puts the pain on callsites to ambiguous interfaces, rather than producers of them; it's generally easier to fix callsites. As for the breaking changes, some of them will be silently picking more appropriate overloads, which is generally the right thing to do and more intuitive. Only in rare and contrived cases (where C# author overrides two ambiguous specific methods with a less specific methods) will there be a compile-time failure.

Note that the VS2015 CTP4 (already shipped by the time we had this LDM) lets you declare things like "C" but there's not even any overload resolution at the callsite. That will give us lots of opportunity to hear from customers whether they're suffering from the fact that we no longer do implicit shadowing. If we hear pain, then we can re-visit.