Nullable<T> for reference types

Topics: C# Language Design
Nov 11, 2014 at 4:27 AM
Because Nullable<T> only supports value types, some teams write their own implementation of Nullable<T>, e.g. Option<T> or Maybe<T> for both value and reference types, but these come with various limitations:
  1. requires the introduction of an interface to support covariance, which can then be null...
  2. there is no way to have implicit type conversion to an interface
  3. no support for ?? and ?. operators
  4. no support for the T? notations
So why not let Nullable<T> accept reference types (and be covariant). Some of the benefits of doing so would be:
  • use of Nullable<T> where it really matters, reference types are the most common source of null reference exceptions.
  • LINQ support can be added and integrated with LINQ for IEnumerable
  • Common approach for value and reference types
Nov 11, 2014 at 10:32 AM
Edited Nov 11, 2014 at 10:34 AM
The big problem with this approach is that it doesn't really solve anything: "non-nullable" reference types can still be null, you're proposing adding distinction without a difference.

Also:
  • Classes and structs can't be covariant, only interfaces and delegates are. This is a limitation of the CLR, so even adding a built-in type wouldn't solve this. Or are you proposing to change the CLR for this?
  • LINQ is just a bunch of extension methods, with some syntactic sugar to call them. Unless you want to modify that syntax sugar, you can write those extension methods yourself.
Nov 11, 2014 at 11:38 AM
The Maybe<T> or Option<T> implementations ensure that they cannot contain null values, they are initialized to being empty when the reference passed in the constructor is null. I can assure you that Option<T>/Maybe<T> (even with the risks associated to using an interface) is extremely efficient at solving the null reference problem. The only issue I can see with Nullable<T> is the presence of an explicit type converter which will throw when the Nullable value is empty.

LINQ support for Nullable<T> is very limited as more complex LINQ queries generate anonymous classes (not struct) which are not compatible with Nullable<T>.

Yet another reason to add variance annotation support to struct/classes?
Nov 12, 2014 at 5:28 PM
svick wrote:
  • Classes and structs can't be covariant, only interfaces and delegates are. This is a limitation of the CLR, so even adding a built-in type wouldn't solve this. Or are you proposing to change the CLR for this?
The prohibition on covariant structures is not merely a CLR limitation, but a semantic one:
  1. A class with covariant parameters cannot be type-safe if it has any mutable fields of any type involving T.
  2. For every structure type other than Nullable<T> the run-time also defines a heap object type with class semantics, whose fields are all mutable.
Semantically there would be no problem with an aggregate type that contained a field of a covariant generic type, if the aggregate could never be boxed as its own type. Unfortunately, there's no way to specify that an aggregate should behave in such fashion.

The biggest semantic obstacle to making Nullable<T> accept arbitrary covariant type parameters would be formulating suitable boxing and unboxing behavior for a Nullable<T> whose inner content would either be a null reference or is itself an empty Nullable<T>. Were it not for the convention of comparing Nullable<T> objects to null to test if they are empty, it would have been possible to say that an empty Nullable<T> will box as a singleton instance of class type EmptyNullable<T> [a system-private sealed class with no fields derived from system-private EmptyNullable, which would compare equal to all instances of types derived from EmptyNullable]; a Nullable<Nullable<T>> with no value would box as EmptyNullable<Nullable<T>> while one whose outer layer contains an empty Nullable<T> would box as EmptyNullable<T>.

Despite the aforementioned difficulty, it would almost be possible to achieve workable semantics by having an empty Nullable<T> box as null while one that contained a null reference boxed as some non-empty-nullable-that-containings-null type. That would probably be workable but for one exception: the behavior of casting a Nullable<Object> to Object. Normally, casting a Nullable<T> to T should be equivalent to calling Value. Unfortunately, it would be ambiguous as to whether casting a Nullable<Object> to Object was supposed to box it (which should yield null if the container is empty, or a non-empty-nullable-that-containings-null if it contained a null reference), or should yield Value (which should throw an exception if the container is empty, or return null if it contains a null reference).

Since the inability to use a covariant type with Nullable<T> precludes the possibility of having an interface with covariant type T include a TryGetValue method that returns a Nullable<T>, perhaps what's needed is a means via which a TryGetValue method could have a signature of the form: T TryGetValue(someParameters, out bool gotValue) but be marked with an attribute that would request the compiler to allow a friendlier caller-side syntax (e.g. if ok(thing=mySet.TryGetValue(myKey)). That would allow many of the advantages of a covariant Nullable<T> without requiring CLR changes (the biggest weakness would be the inability to use the new syntax with existing collections without making code unusable with versions of the runtime that only support the covariance-hostile bool TryGetValue(someParameters, out T value).
Nov 17, 2014 at 12:50 PM
Allowing Nullable to wrap a null value will lead to a lot of confusion and problems as mentioned above. In that regard I would follow Scala which will returns an empty Option when passed a null rather than F# which seems to happily wrap a null reference.

Similarly Nullable<Nullable<T>> should not be supported. I am currently exploring LINQ queries where both IEnumerable<Nullable<T>> and Nullable<IEnumerable<T>> are reduced to IEnumerable<T>. Although there might be some edge cases where one would be interested in counting holes, there is little point, in general, in enumerating emptiness or extracting an empty enumeration from Nullable. See here for more details.

It is worth giving a simple example to illustrate some of the advantages of Nullable/Option/Maybe<T>:
var r = from o in someObject.AsOption()
        from s in o.GetPropertyValue<string>("P")
        from i in s.ParseToInt()
        from v in i.Lookup(dict)
        select v;
which is not only better in terms of clarity, readability, elegance and conciseness but also much less error prone than the following:
string r = null;
if (someObject != null) {
    var value = someObject.GetPropertyValue<string>("P");
    int i;
    if (value != null && int.TryParse(value, out i)) {
        dict.TryGetValue(i, out r);
    }
}
The above LINQ query can also be combined with IEnumerable:
var rs = from someObject in manyObjects 
         select from o in someObject.AsOption()
                from s in o.GetPropertyValue<string>("P")
                from i in s.ParseToInt()
                from v in i.Lookup(dict)
                select v;
where only non-empty values will be included in the result.
Nov 17, 2014 at 3:43 PM
GabrielHorvath wrote:
Similarly Nullable<Nullable<T>> should not be supported.
Were it not for the unusual boxing behaviors, there would be nothing particularly odd about Nullable<Nullable<T>>; it would be a logical return type for a TryGetValue method to return when using a collection whose generic type parameter was a Nullable<T>; if the outer HasValue is false, that would imply that the item was not found in the collection; if the outer HasValue was true but the inner one was false, that would mean that a null-valued Nullable<T> was stored in the collection. A GenericCollection<T> could yield the same behavior for any type T whether it was nullable or not. It would be better if there were some way to efficiently allow for a covariant conditional return type. Because of array covariance, one could have a TryGetValue method return a T[] which could either return a zero- or one-element array, but that would require the creation of an extra heap object every time it was run and a value existed.
Nov 18, 2014 at 7:46 PM
Nullable<Nullable<Nullable<Nullable<... Russian dolls all the way down ;-)

So far I have not had any real world use for Nullable<IEnumerable<T>> or IEnumerable<Nullable<T>>, maybe it is because I don't like Emmental cheese. I would like to see examples where this would be the best approach.
Dec 2, 2014 at 1:30 PM
var r = someObject?.GetPropertyValue<string>("P")?.ParseToInt()?.Lookup(dict);
Granted, this is not identical to having a Maybe<T>/Optional<T>, but rarely are you going want to include a null reference as an "existing" value. Maybe<T>/Optional<T> are very useful with the existing C# language, but much less so in C# 6 with "null propagation".

I'd much rather see a NonNullable<T> added ;).
Dec 2, 2014 at 4:02 PM
GabrielHorvath wrote:
Nullable<Nullable<Nullable<Nullable<... Russian dolls all the way down ;-)

So far I have not had any real world use for Nullable<IEnumerable<T>> or IEnumerable<Nullable<T>>, maybe it is because I don't like Emmental cheese. I would like to see examples where this would be the best approach.
If the Framework allowed MaybeValid<out T> for arbitrary T (basically like Nullable<T>, but with a name that would be more descriptive of common usage scenarios), then "try-get" patterns could be written much more nicely as MaybeValid<TValue> TryGetValue(TKey key) rather than the non-covariant bool TryGetValue(TKey key, out TValue value);. I'd call that a pretty big win. Further, if that were a common "try-get" pattern, then it would be worthwhile to have language support for a construct to create a new scoped alias for the value [the existing pattern requires:
TheResultType result;
if (myDict.TryGetValue(myKey, out result)
{ ... }
even though result will likely only be useful within the if clause]. Of course, things don't work that way, and the existing boxing behaviors wouldn't be suitable for a universal-type MaybeValid<T>, but that doesn't mean that being able to call TryGetValue on a Dictionary<SomeType, MaybeValid<SomeOtherType>> and get a result of type MaybeValid<MaybeValid<SomeOtherType>> wouldn't be useful.
Dec 2, 2014 at 5:29 PM
if (myDict.TryGetValue(myKey, out TheResultType result)) { ...  }
For most people, that would be good enough for this case. In fact, it may be considered by many imperative programmers as "better".

However, I don't want to sound completely negative towards the adoption of a Maybe<T>. The maybe monad is extremely useful in functional programming, and C# is a pseudo-functional language now. The more functional concepts we adopt, the better, IMHO. I'm mostly playing devils advocate. Not all C# developers are comfortable with functional programming concepts, and the most common use cases for a maybe monad are addressed in a different way with newer language features. Maybe<T> is a nice to have, now, while in .NET 4.5 I would have classified it more as a should have.

I will say, though, that if you want to "fix the null mistake" you'd be better off with a NonNullable<T>. Borrowing some Spec# syntax for it would be nice as well: string! notNullString. If we were designing a new language, NonNullable should be the default, but we can't do that with C#. At least let me transition to specifying the intent, though.