`Compilation` mismatch with custom BCL

Topics: APIs
Apr 4, 2014 at 6:42 PM
Edited Apr 4, 2014 at 6:42 PM
I have a custom BCL and am rewriting syntax trees and regenerating compilations. Upon regenerating the compilation, some of the symbols in the 2nd compilation end up with references to symbols in the 1st compilation. This causes failures since the symbols no longer match up.

Consider a custom System.Type and System.String. If I ask my 2nd Compilation for System.Type and look up the method GetField(string), I find that the IMethodSymbol that is returned has for the string parameter a type that is attached to the 1st compilation. (I determine this by getting the ContainingAssembly of the parameter type, and inspect the non-public fields in the debugger -- one of which is the Compilation.

The ultimate problem is that symbols contained in the 1st compilation are not equal (.Equals returns false) to the equivalent symbol in the 2nd compilation. To generate the 2nd compliation I have tried both Compilation.Clone and Compilation.ReplaceSyntaxTree.

Is it wrong to assume that the types contained in one compilation should never have references to types contained in another compilation? (I should note that in the previous version of Roslyn, this situation never arose)
Developer
Apr 4, 2014 at 7:42 PM
Could you share a snippet of code that demonstrates how do you create these compilations? Specifically, how do you reference your custom BCL libraries?
Apr 4, 2014 at 8:22 PM
Edited Apr 4, 2014 at 8:24 PM
In reference to your second question, the problem actually arises when compiling the BCL project itself. I do not encounter any issues when compiling projects dependent on the custom BCL. I create the original compilation via:
var compilation = await project.GetCompilationAsync();
I then walk the compilation units and use a CSharpSyntaxRewriter to replace each syntax tree:
compilation = compilation.ReplaceSyntaxTree(syntaxTree, SyntaxFactory.SyntaxTree(compilationUnit, syntaxTree.FilePath));
It's these secondary compilations that end up having symbols that reference symbols in the original compilation. The project is open source, so you're welcome to view all the code (or run the WootzJs.Compiler project yourself -- just make sure not to try to build any of the other projects since they currently fail) The lines I've quoted above come from my Compiler class.

I would volunteer to create a smaller more self-contained example, but as you know, a minimal BCL project still requires a fair number of types to be provided by the project.

Let me know if there is any more info I can provide.
Apr 4, 2014 at 8:42 PM
One clue is I notice that this problem seems to stem from predefined types like string, int, bool, etc. In my above example, I have the method GetField defined in my System.Type:
public FieldInfo GetField(string name) { ... }
If I change the signature of this method to:
public FieldInfo GetField(String name) { ... }
Or in other words, I use System.String rather than the compiler alias string, the problem goes away (for that one case, and rears its head the next time I use a predefined type).
Developer
Apr 5, 2014 at 1:13 AM
Kirk, the way Roslyn compiler determines which assembly is "the core library", is:
  1. The assembly must declare a public class System.Object
  2. The assembly must have no references
Make sure both conditions are satisfied by your custom corlib. Also, it helps if you dump compilation.GetDiagnostics() to see what the compiler is not happy about. There might be another error and what you're seeing is a result of that.
Apr 5, 2014 at 5:57 PM
Kirill, thanks for getting back to me. My core library does declare a class System.Object and has no references to any other project. I've created a SSCCE (well, as short as it can be with a minimal BCL) and made it available on my Google Drive here. (it's 70 mb since I included the Roslyn binaries) I have a class called Compiler now that illustrates well the probem I am facing.

In paritcular note below how I declare each symbol twice, first by obtaining them from the initial compilation (compilation1) and then again by obtaining them from the second compilation (compilation2). You'll notice that the second compilation is obtained from the first via the Clone() method. Pay particular attention to the last five Console.WriteLine statements, as they illustrate precisely what the problem is. That is, I have symbols that represent the same type but are not considered equal.
public class Compiler
{
    public static void Main(string[] args)
    {
        var project = MSBuildWorkspace.Create().OpenProjectAsync(args[0]).Result;

        var compilation1 = project.GetCompilationAsync().Result;
        INamedTypeSymbol typeType1;
        INamedTypeSymbol typeString1;
        IMethodSymbol getField1;
        INamedTypeSymbol getFieldParameter1;
        GetSymbols(compilation1, out typeType1, out typeString1, out getField1, out getFieldParameter1);

        Console.WriteLine("This will print true, since the parameter is a string");
        Console.WriteLine("typeString1.Equals(getFieldParameter1): " + typeString1.Equals(getFieldParameter1));
        Console.WriteLine();

        var compilation2 = compilation1.Clone();
        INamedTypeSymbol typeType2;
        INamedTypeSymbol typeString2;
        IMethodSymbol getField2;
        INamedTypeSymbol getFieldParameter2;
        GetSymbols(compilation2, out typeType2, out typeString2, out getField2, out getFieldParameter2);

        Console.WriteLine("This will print false.  I expect it to print true");
        Console.WriteLine("typeString2.Equals(getFieldParameter2): " + typeString2.Equals(getFieldParameter2));
        Console.WriteLine();

        Console.WriteLine("This will print true!  This shows that a type obtained in compilation2 has a reference to a symbol in compilation1");
        Console.WriteLine("getFieldParameter2.Equals(typeString1): " + typeString1.Equals(getFieldParameter2));

        Console.ReadLine();
    }

    private static void GetSymbols(Compilation compilation, out INamedTypeSymbol typeType, out INamedTypeSymbol typeString, out IMethodSymbol getField, out INamedTypeSymbol getFieldParameter)
    {
        typeType = compilation.GetTypeByMetadataName("System.Type");
        typeString = compilation.GetTypeByMetadataName("System.String");
        getField = typeType.GetMembers("GetField").OfType<IMethodSymbol>().Single(x => x.Parameters.Count() == 1);
        getFieldParameter = (INamedTypeSymbol)getField.Parameters.Single().Type;            
    }
Apr 5, 2014 at 7:18 PM
Also, for compilation1, there are no alarming warnings in the diagnostics. However, for compilation2 (which, remember, is nothing but a clone) I get the errors:
C:\dev\RoslynIssue\WootzJs.Runtime\Object.cs(40,21): warning CS0108: 'System.Object.GetType()' hides inherited member 'object.GetType()'. Use the new keyword if hiding was intended.
C:\dev\RoslynIssue\WootzJs.Runtime\Object.cs(49,31): warning CS0114: 'System.Object.ToString()' hides inherited member 'object.ToString()'. To make the current member override that implementation, add the override keyword. Otherwise add the new keyword.
This seems to indicate that after cloning the compilation, Roslyn "forgets" that it's a BCL project.
Developer
Apr 9, 2014 at 12:12 AM
This is a bug in Compilation.Clone(). Thanks for the repro!

As a workaround you can use compilation.WithReferences() instead of Clone().
Apr 9, 2014 at 12:24 AM
Hi Tomas, thanks for the reply. Just wanted to make sure you know this is the ticket: https://roslyn.codeplex.com/workitem/43

Also, my project was a minimal test that demonstrates the problem. I am not actually using compilation.Compile() in my code. The bug also manifests itself if you use ReplaceSyntaxTree, which is what I'm doing in my real project. Replacing the compilation1.Clone() line with this:
    var compilation2 = compilation1.ReplaceSyntaxTree(compilation1.SyntaxTrees.First(), compilation1.SyntaxTrees.First());
Will also reproduce the problem.
Marked as answer by TomasMatousek on 4/9/2014 at 4:34 PM