Can late-bound access with Option Strict On finally be resolved?

Topics: VB Language Design
Apr 21, 2014 at 5:55 PM
Can this please get addressed once and for all? VB was clearly ahead of C# in late-bound/dynamic handling until being leap-frogged with C#'s support of the dynamic keyword/DLR integration.

Current undesired scenario:
Option Strict Off 'evil

Module Stuff
    Public Sub Groverina(x)
        With x
            .DoThis
            .DoThat
            .IsAMonkey = True
        End With
    End Sub

    Public Sub DoStuffWithADynamicObject()
        Dim x = GetDynamicObject()

        With x
            .Quack
        End With
    End Sub
End Module
Proposed desired scenario:
Option Strict On

Module Stuff
    Public Sub Groverina(Dynamic x)
        With x
            .DoThis
            .DoThat
            .IsAMonkey = True
        End With
    End Sub

    Public Sub DoStuffWithADynamicObject()
        Dynamic Dim x = GetDynamicObject()

        With x
            .Quack
        End With
    End Sub
End Module
"Dynamic Dim" indicates run-time type checking would occur for the particular variable. I like this approach because it is an explicit opt-in. 99% of the time things are left early bound but for those scenarios that need it I don't have to play games using partial classes with Option Strict Off to isolate the evil.

At present, this is an unresolved feature parity issue between the two languages.
Apr 22, 2014 at 12:48 AM
Like the idea of having a "strongly typed" representation for a dynamic object, rather than the current implementation of using object.
Sub Groverina ( myArg As Dynamic )

End Sub

Dim x As Dynamic = GetDynamicObject()
Or have a new keyword Dym which defines a dynamic object.
Dym obj = GetDynamicObject
Dynamic :> Object is a lifted kind of object, whose behaviour and properties are dynamic. Where as Object are fixed in behaviour.

Every Object can be cast to Dynamic
Apr 23, 2014 at 5:44 AM
I thought about "myArg as Dynamic" as well - the only concern I had about it was that Dynamic is in the System namespace so there could be some ambiguity? I'd happily accept either approach into the language and am hopeful it will get resolved.
Coordinator
Apr 28, 2014 at 6:48 AM
I prototyped exactly this, but I used the (reserved) keyword "Variant" instead of "Dynamic". I figured they mean roughly the same thing... The rule was: Variant means exactly the same as Object. However, any warnings or error messages about late-bound-access are suppressed if the operand has type Variant.

It would also be possible to use Dynamic, and we'd avoid clashes the same way that C# avoids clashes with the "var" type. That is: if the identifier "Dynamic" binds to a type, then use that type, otherwise use this new pseudo-type.


Interesting to note that C# is moving in the opposite direction...
  • Currently C# "dynamic" uses IDynamicMetaObjectProvider for all its dynamic invocations, i.e. it requires Ref.Emit. If the operand isn't a DLR object, then it falls back to using reflection to dispatch to the right method. The impact is that there are heavy system requirements (Ref.Emit is dang hard to support, much harder than reflection). Also, C# has higher up-front cost, that pays off if you execute the same dynamic call in a loop 50 or more times.
  • Currently VB late-binding first attempts the dynamic call using Reflection. Reflection is still a heavyweight burden for a platform to support, but not as heavy-weight as Ref.Emit.
  • With things like ProjectN coming along for some scenarios, it's very hard to support ref.emit, and moderately hard to support reflection. C# has navigated itself into a pit where the best way out is the proposed C# syntax x.$foo, an exact copy of the x!foo syntax in VB.
Anyway, all this is to say that what you describe is possible, and I think it would be great, but it's doesn't fit with any other agendas currently going on at Microsoft. If you want us to work on it, please post a suggestion on UserVoice and encourage people to vote on it, and if there are enough votes then we'll be motivated to do it.
Marked as answer by lwischik on 4/27/2014 at 11:49 PM
Apr 28, 2014 at 4:04 PM
Good to hear from you on this topic, Lucian.

I reviewed every one of the 139 VB requests on UserVoice. There are more votes for resurrecting VB6 than all other issues combined so I'm not holding out much hope of getting any traction over there :)

That being said, Dim x as Variant really would be a nice win for the language.

Regarding .NET Native, I spoke to one of their engineers at //Build and I don't think they are planning to support dynamic invocation via Reflection or Ref.Emit at all (certainly not in 1.0). All types must be explicitly made known to the compiler via some sort of manifest. Makes sense, since how could you know something you don't know before you know you don't know it? (meta is fun). I think a natural tradeoff of late binding is that you must forego native compilation. However, common dynamic scenarios - scripting, REPL, traversing arbitrary XML/JSON, etc. - aren't really native-compilation scenarios to begin with, so to me it's sort of a non-issue and an understandable constraint.

Getting into the nitty-gritty of DLR binding and call-site caching is head-spinning. But the good thing is that it's already written. From a VB implementation perspective, I thought that dogfooding a Variant into the DLR would be time well spent. I imagine that deciding whether to wrap a call in an emitted delegate or whether to one-off it using MethodInfo.Invoke would be best handled by compiler intelligence. As you said, the timings of the two can vary dramatically. For instance, if a Variant invocation falls under a loop or a foreach, then emitting a delegate in advance and referencing that delegate could be one such optimization.

Regarding the above paragraph, doesn't the DLR sort of handle all of this already? Or is your point that the DLR is unsuitable and that there have been a lot of problems encountered by the C# team? If so, then couldn't there be a common layer that both compilers reference at a certain point when late-bound invocations are involved - to minimize duplication of the gnarly abstractions required to support bindings in a performant way?

Without getting into the whole static vs. dynamic debate (which changes fortnightly), my main point is this: if VB/C# is going to support dynamic invocation (which they do), then that support should be top-notch, and, frankly, the current use case that forces Option Strict Off and the elimination of all type checking is rather unfortunately bad.

Craig.
Apr 29, 2014 at 7:31 PM
IMHO, one of the biggest problems with VB.NET (and to a lesser extent C#) is the lack of a distinction between "Reference to heap object derived from System.Object" and "Anything". Even if an "Anything" [or Variant, or Dynamic, or whatever it's called] is stored internally as a reference to a System.Object, that doesn't mean there shouldn't be a distinction between them. It's not possible to retroactively apply the distinction everyplace it should be applied (e.g. auto-boxing value types when converting to Anything but not Object), but I'd like to see an "Option Quirks All/None/{SpecificChoices}" which would eliminate from System.Object some of the quirks which derive from its use as a substitute for Variant [though bizarrely enough, the VB6 variant couldn't be used this way anyhow!]. For example:
Dim P As Object = New Point(1,2)
Debug.Print("{0}", Object.ReferenceEquals(P,P)) ' Prints False, but should only do so with quirk enabled
It might make sense to have a type which encapsulates boxed structures' values rather than their identities, but a references of type System.Object should exhibit reference semantics regardless of the type of object identified thereby.
Coordinator
May 2, 2014 at 12:21 AM
Hey lonewolfcj,

I'll say this is definitely a topic we know the community is passionate about and we do consider it often when discussing language design. As Lucian stated it's certainly something we could do, it's just a matter of when and whether we should leapfrog both VB late-binding and C# dynamic with a "dynamic interface" type of feature.

One thing you mentioned though that I'd like to clear up is that VB does have the same DLR integration as C# has with dynamic today. When you do late-binding against "Object" in VB we perform the binding depending on the target. For COM objects we use the COM binder VB has always used and for plain ol' CLR objects we use the existing VB late-binder. For DLR objects (any object that implements IDynamicMetaObjectProvider) we use our own DLR binder the same way C# does. The languages have complete parity here (and if you find places where they don't, please file a bug).

Also, you stated that Option Strict Off eliminates all type checking and that's not accurate. Option Strict Off doesn't have any less type checking than Option Strict On. You will still get type-checking errors with it Off. The only thing On does is disallow all late-binding and disallow implicitly typing things as Object (by omitting As clauses and not using type inference) which would cause more late-binding and disallow a certain class of type conversions to happen implicitly. It doesn't disallow those conversions - you can still perform them explicitly. It doesn't make the conversions any safer - they can still fail. It doesn't make them any faster - the emitted IL is the same no matter. And it doesn't really make anything less possible - even with Option Strict Off it's impossible to convert a Button to a TextBox and it's still impossible to treat a Button like a TextBox at runtime so it's always an error. Also, even when late-binding is enabled the compiler only late-binds when it can't early bind (if you call a method that takes an object it's still an early bound call, if you call a method with only one type it's an implicit conversion, if you call a method with overloaded parameter types then it's late-bound).

It's worth noting that all of these patterns can be restricted in more granular ways in the project properties:

Image

If you only wish to enable late-binding and nothing else you can do this at the project-level. I personally usually run with late-binding enabled, implicit conversions enabled, and implicit-typing as Object as an error because in practice I never accidentally late-bind to things typed as Object (I'm much more likely to accidentally type something as Object). The unfortunate thing about this approach is that it still doesn't give you the granularity of enabling late-binding at a method level (or variable-by-variable basis).

Anyway, because adding a dynamic type doesn't actually enable any new scenarios we always have to weigh the value of it against lighting up something new and more generally useful (like Async).

Regards,

Anthony D. Green, Program Manager, Visual Basic & C# Languages Team
May 2, 2014 at 3:52 PM
ADGreen wrote:
Anyway, because adding a dynamic type doesn't actually enable any new scenarios we always have to weigh the value of it against lighting up something new and more generally useful (like Async).
How about the ability to have a type which can hold a reference to any heap object, but could only be assigned by copying a reference to an existing object, as distinct from a type whose assignment may create a new object and store a reference to that?

There are times when it is useful to have a method (like e.g. String.Format) which can accept anything. There are other times (e.g. with ReferenceEquals or the constructor for WeakReference) where reference semantics are crucial, and the use of anything other than reference semantics is guaranteed to produce erroneous results.
Coordinator
May 2, 2014 at 7:53 PM
supercat,

I'm soo glad you brought that up. That's actually one of the reasons I'm most interested in having an option to use a Dynamic type. Today because Object is VB's dynamic type there are all kinds of subtleties to its behavior to work in both the static and dynamic worlds. The copying semantic VB exhibits today is a side-effect of that. As is the inability to use extension methods on Object. I've been bitten by these a few times myself and dynamic interfaces wouldn't help that. However, this issue (while annoying) is still not more important that features like Async. But it is just one argument in support of doing it when we have time.

BTW: Why are you using ReferenceEquals instead of the VB Is operator? It should perform faster than the method call to ReferenceEquals.

-ADG
May 2, 2014 at 8:44 PM
ADGreen wrote:
BTW: Why are you using ReferenceEquals instead of the VB Is operator? It should perform faster than the method call to ReferenceEquals.
My goal was to illustrate the difficulty of calling a method which takes parameters of type Object and requires that they behave as a reference type. I used ReferenceEquals because it is such a method. Perhaps a better example would have been the constructor for WeakReference; that's actually where I discovered the issue.

Incidentally, I wonder if there's any production code which would break if the weird copying behavior were simply eliminated? Nothing in VB6 worked that way.
May 2, 2014 at 9:43 PM
interjecting @Anthony, yes, I never knew why we couldn't in VB write an <Extension> to System.Object :)

Our team definitely appreciates game-changing features like direct language support for Async/Await. It's in active use every day so thank you.

Thanks also for clarifying the details about the internal implementation. If I could summarize what you are saying (feel free to correct):

1) Adding first-class-citizen support for an explicit Dynamic type in VB would be valuable (acknowledged by both language team and community for quite some time).
2) Although support now is not ideal, there are workarounds that can improve some scenarios (thanks for the screenshot).
3) As there are more important items on the backlog, this request is sort of in a holding pattern - not relegated to a feature black hole but not on the radar for the foreseeable future.

Strictly as an example - say I serialize an anonymous type (the result of a LINQ query, for example), pass it to a service, then deserialize it in on the service side and act on it via a script which I also send with the object (or interactively via a Roslyn-enabled scripting session). I want the processor to kick out a script that would fail an Option Strict On test while realizing that the object being acted on is a dynamic type (it would be declared using the new Dynamic syntax).

As for syntax, maybe even simplify things further by enabling a new Option Dynamic On mode and then just leave off the As clause. AHHHH that would be fantastic.
Option Dynamic On

Function Q(format)
    Dim x 'would be a dynamic object
    Dim y as Object
    Dim z as String

    z.Monkey = "Banana" 'generate error - you can't do that with String, silly

    x.Request = "Goo"
    AddSomethingInterestingToDynamic(x)
    AnotherPipelineRoutine(x)
    OneMoreInThePipeline(x)

    If format="JSON" Then
        Return x.AsJSON() 'invokes extension on dynamic object
    End If
End Sub

Function AddSomethingInterestingToDynamic(x)
    x.Logged = "ByMeAtThisTime"
    With x.Validated
        .By = "Me"
        .At = Date.Now
    End With
  Return x
End Function

...
May 3, 2014 at 10:35 PM
Are the parameters to those functions also Dynamic or Object?

Object would make it backward compatible, if it Dynamic then it breaks code.