Using modified Roslyn bits against Visual Studio Previews

Topics: APIs, General
Developer
Nov 12, 2014 at 8:00 PM
I'm creating this thread to create a more discoverable discussion around how to try out your modified bits against specific VS previews, and the work involved in maintaining the scripts that support it.

(For the backstory to get up to speed on the previous discussion, please see https://roslyn.codeplex.com/discussions/569086.)

The current status is that we are working on an updated script to support the VS Preview bits that came out today; KevinR will loop back with updates.

--Matt--*
Developer
Nov 13, 2014 at 2:29 AM
Edited Nov 13, 2014 at 2:31 AM
Nov 20, 2014 at 8:16 AM
Edited Nov 20, 2014 at 10:11 AM
@KevinRansom Thank you for these great improvements. I've found an error in the "Debugging unit test failures" section of your document: it does not specify that one must pass the "noshadow" flag to xUnit. Without this, xUnit always fails with:
System.IO.FileLoadException: Could not load file or assembly 'Microsoft.CodeAnalysis, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one
of its dependencies. Strong name signature could not be verified. The assembly may have been tampered with, or it was delay signed but not fully signed with the correct private key. (Exception from HRESULT: 0x80131045)
So I'd suggest changing step 5
from:
Enter the unit test assembly name in the "Command Line Arguments" text box
to:
Enter the full path of the unit test assembly, followed by " -noshadow" to the "Command Line Arguments" text box
Developer
Nov 21, 2014 at 3:18 PM
Thanks, Omer. We will take a look at this.

--Matt--*
Developer
Nov 21, 2014 at 5:59 PM
Thanks for reporting this, I have made the change you proposed. Let us know if there is anything else that should be amended.

Enjoy

Kevin
Nov 23, 2014 at 10:26 AM
Thank you. Just to point out, the change does not yet seem to appear at the URL referenced from this discussion - https://roslyn.codeplex.com/wikipage?title=Building%2c%20Testing%20and%20Debugging%20Roslyn
Developer
Nov 24, 2014 at 2:57 PM
Thanks -- should be there now.

--Matt--*
Developer
Nov 24, 2014 at 5:39 PM
@Omer,

I apologize I should have made things clearer, I updated the page for the official instructions: https://roslyn.codeplex.com/wikipage?title=Building%2c%20Testing%20and%20Debugging&referringTitle=Documentation that are linked from the Documentation Page: https://roslyn.codeplex.com/documentation

The page you referred to was the "preview" page for the documentation so that I could get feedback on the contents prior to checking in the fix. To avoid confusion moving forward I will delete the page: https://roslyn.codeplex.com/wikipage?title=Building%2c%20Testing%20and%20Debugging%20Roslyn

Thanks

Kevin
Dec 4, 2014 at 9:42 AM
Edited Dec 4, 2014 at 9:44 AM
Hi Kevin,

I’ve been trying to create a Pull Request with OmerRaviv using the latest version of the “Build and Test” document.
The “How To Contribute” document states that Pull Requests must be done on ‘master’, but I cannot get all the unit tests to pass even though I tried to follow your instructions to perfection, and even tried it on a clean VM.
Is it possible that Microsoft’s internal build servers use some sort of “special” customized environment which can account for these failures?

Here’s what we did:
  1. Created a squeaky-clean Virtual Machine on Azure (selected “Visual Studio 2015 Preview” Image from the Gallery)
  2. Cloned Roslyn Rep
  3. Src\.nuget\nuget restore Src\Roslyn.sln
  4. msbuild /m BuildAndTest.proj
  5. Many tests fail due to with a NullReferenceException in CompilerServerUnitTests.AllCompilerFiles, originating from a mismatch in versions of System.Collections.Immutable.dll:
    It looks like the failure is in CompilerServerUnitTests.ResolveAssemblyPath: In our environment, since the tests compare the currentlyloaded version of System.Collections.Immutable.dll (1.1.32.0) with the one in the msbuild folder (1.0.0.0), it returns null.
    To fix this, I changed the reference to System.Collections.Immutable.dll in CompilerServerUnitTests.csproj to “Copy Local = true”.
    Microsoft.CodeAnalysis.BuildTasks.UnitTests.CompilerServerUnitTests.AnalyzerChangesOnDisk
    Microsoft.CodeAnalysis.BuildTasks.UnitTests.CompilerServerUnitTests.AssemblyIdentityComparer1
    Microsoft.CodeAnalysis.BuildTasks.UnitTests.CompilerServerUnitTests.CompileErrorsCS
  6. Hundreds of tests fail with FileLoadException : Could not load file or assembly
    and lots of other Tests failures due to nonexistent Debug\MyAnalyzer.dll
    All, directly or indirectly, originating from strong name validation failures for numerous assemblies including Roslyn.Compilers.BuildTasks and Microsoft.CodeAnalysis.Test.Resources.Proprietary
    To fix this I executed sn -Vr *,31bf3856ad364e35
    AssemblyAttributeTests.AlgorithmIdAttribute
    AssemblyAttributeTests.AssemblyAttributesFromNetModule
    AssemblyAttributeTests.AssemblyAttributesFromNetModuleAddMulti
    ...
    Microsoft.CodeAnalysis.BuildTasks.UnitTests.CompilerServerUnitTests.AnalyzerChangesOnDisk
    and many many others
  7. Again ranmsbuild /m BuildAndTest.proj but still there were test failures:
    Microsoft.CodeAnalysis.CSharp.UnitTests.Semantics.NameCollisionTests.Bug879811_1
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_AssemblyNameIsPath
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_AssemblyNameIsPath2
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_UpdateExistingReferences
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_WithReferencedProject_LoadMetadata_ExistingMetadata_Succeeds
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_WithUnrecognizedProjectReferenceFileExtension_WithMetadata_SkipFalse_SucceedsByLoadingMetadata
    Microsoft.CodeAnalysis.UnitTests.MSBuildWorkspaceTests.TestOpenProject_WithUnrecognizedProjectReferenceFileExtension_WithMetadata_SkipTrue_SucceedsByLoadingMetadata
    Microsoft.CodeAnalysis.VisualBasic.UnitTests.PDB.PDBTests.LocalCapturedAndHoisted
    xunit output file can be found here
    prior to these changes the xunit looked like this - heavy 18mb!
  8. Most of these tests are related to MSBuildWorkspace. We validated that we are using msbuild14 so we suspect these failures occur because you might be using a more up to date
    version of msbuild 14 internally than what was shipped with the VS2015 Preview.
Please advise.
Developer
Dec 4, 2014 at 9:50 PM
Nitzan, thanks for the report -- we will huddle on this and get back soon.

--Matt--*
Developer
Dec 5, 2014 at 4:10 PM
NitzanZada,

hi, I'm looking at this right now. How did your changes fare when you ran them against the preview branch? Did everything build and run clean?

Thanks

Kevin
Dec 7, 2014 at 11:51 AM
Edited Dec 7, 2014 at 12:02 PM
Hi Kevin, About the preview branch, I'm getting the same results minus one failing test:
Microsoft.CodeAnalysis.VisualBasic.UnitTests.PDB.PDBTests.LocalCapturedAndHoisted
xunit output file can be found here

Thanks in advance.
Developer
Dec 9, 2014 at 3:47 PM
Thanks, Nitzan. Kevin & co. are actively working on fixes for this right now and we will loop back.

--Matt--*
Developer
Dec 9, 2014 at 9:21 PM
Update: the fixes are somewhat complex, so we've got a plan to get you unblocked quickly; Kevin will elaborate shortly.
Developer
Dec 9, 2014 at 10:47 PM
Edited Dec 10, 2014 at 2:34 AM
@NitzanZada

Hey,

I have submitted a PR to the preview branch that disables a few test and fakesigns some assemblies. Mostly these are workarounds to unblock the branch, the issues will be triaged and fixed in the normal way.

The PR is at: https://roslyn.codeplex.com/SourceControl/network/forks/KevinRansom/roslyn/contribution/7808

You can clone my branch and then using a Visual Studio 2015 command window:
git clone https://git01.codeplex.com/forks/kevinransom/roslyn
cd roslyn
msbuild buildandtest.proj /p:DeployExtensions=false
Please let me know if it works for you:

The following is the list of issues that need to be resolved as a result of this:

Codeplex issue 450 ---- Ambiguous reference between Static<T1>.Blah and Static<T2>.Blah produces non-deterministic
Codeplex issue 451 ---- TestOpenProject_WithReferencedProject_LoadMetadata_ExistingMetadata_Succeeds
Codeplex issue 452 ---- Testcase failure: TestOpenProject_AssemblyNameIsPath
Codeplex issue 453 ---- Testcase TestOpenProject_AssemblyNameIsPath2 fails in OSS preview build
Codeplex issue 454 ---- Testcase TestOpenProject_UpdateExistingReferences fails in OSS preview build
Codeplex issue 455 ---- Testcase TestOpenProject_WithUnrecognizedProjectReferenceFileExtension_WithMetadata_SkipFalse_SucceedsByLoadingMetadata fails in OSS preview build
Codeplex issue 456 ---- Roslyn.Diagnostics.Analyzers in OSS preview branch need to be fake signed
Codeplex issue 457 ---- Testcase TestOpenProject_WithUnrecognizedProjectReferenceFileExtension_WithMetadata_SkipTrue_SucceedsByLoadingMetadata fails in OSS preview build
Codeplex issue 458 ---- Nuget package for Microsoft.CodeAnalysis.Test.Resources.Proprietary.0.blah has a delay signed signed .dll
Codeplex issue 460 ---- Testcase AnalyzerChangesOnDisk fails on OSS Preview branch
Codeplex issue 461 ---- Testcase CompilerServerUnitTests.SimpleMSBuild fails on OSS Preview branch
Dec 11, 2014 at 3:52 PM
Sweet, thanks Kevin!

Your fork compiles well and all the unit tests pass, I'm working on rebasing our changes on top of your fork and testing them out to see that we're not breaking anything.
Are we expected to submit a PR to the 'devpreview' branch when we're done?
Weren't the original instructions to do PR directly on top of the 'master' branch?
If the answer’s yes, should we wait until your Pull Request is accepted before submitting ours?
Developer
Dec 11, 2014 at 5:05 PM
Yes the process will be to submit PR's to the Latest Preview branch. The instructions were written way back in the day, when we were still shipping as an addin to VS 2013, even then we knew VS 2015 would be trickier but we never really got the chance to focus on the process and write it up. I modified the instructions following your report. Hopefully this is a more reliable workflow.

As for Pull requests, we will take care of ensuring they work well in master, also there is no need to wait on my PR being accepted the coordinator will ensure that mine is accepted before yours.

Please continue to let us know when our process is difficult to work with, as @MattGertz often says in his posts "We are new at this open source workflow" but we intend being very good at it.

Glad to have been able to help.

Kevin
Dec 17, 2014 at 12:57 AM
Edited Dec 17, 2014 at 12:58 AM
Visual Studio 2015 Preview, Roslyn runtime error.! (“OpenSourceDebug” project)

I am newbie in Roslyn.
  1. clean install 'Visual Studio 2015 Preview' and 'Visual Studio 2015 Preview SDK'
  2. git clone https://git01.codeplex.com/roslyn
  3. git checkout remotes/origin/releases/Dev14Preview
  4. select “OpenSourceDebug” project and choose “Set as Startup Project”
  5. build Solution (build success)
  6. Start Debugging (F5)
  7. second VS2015 instance executed
  8. open c# file or create c# project in executed second VS2015 instance
C:\roslyn\Src\Workspaces\Core\Portable\Workspace\Host\Mef\MefLanguageServices.cs (line 54)
return (TLanguageService)service.Value; ==> An exception of type 'System.Reflection.TargetParameterCountException' occurred in mscorlib.dll but was not handled in user code


F5 (ignore and continue)

C:\roslyn\Src\Workspaces\Core\Portable\Workspace\Host\Mef\MefLanguageServices.cs (line 54)
return (TLanguageService)service.Value; ==> An exception of type 'System.InvalidCastException' occurred in mscorlib.dll but was not handled in user code


crash


How to fix it ?
Developer
Dec 17, 2014 at 7:48 PM
Hi,

as far as we know this workflow should have worked for you. I am afraid I won't be able to take a look until tomorrow or perhaps Friday.

However, it would be helpful if you could re-create the Roslyn hive so that the We can be sure there was no Junk left over from a previous install of Visual Studio 2015.

From a Visual Studio Command Window type the following command:
  • "%VSINSTALLDIR%\VSSDK\VisualStudioIntegration\Tools\Bin\CreateExpInstance.exe" /Reset /VSInstance=14.0 /RootSuffix=Roslyn
  • after it has completed, clean and rebuild roslyn
  • then debug using F5
Also my Microsoft email alias is kevinr, could you send me the install logfile from:
  • "%USERPROFILE%\AppData\Roaming\Microsoft\VisualStudio\14.0Roslyn\ActivityLog.xml"
And I will take a look

Thanks

Kevin
Dec 18, 2014 at 4:53 AM
Edited Dec 18, 2014 at 4:54 AM
"%VSINSTALLDIR%\VSSDK\VisualStudioIntegration\Tools\Bin\CreateExpInstance.exe" /Reset /VSInstance=14.0 /RootSuffix=Roslyn
==> : Access to the path 'AssemblyInfo.fs' is denied ???

I manually deleted "Roslyn hive registry" and "%USERPROFILE%\AppData\Roaming\Microsoft\VisualStudio\14.0Roslyn" directoy

rebuild. now, successfully loads

Thanks Kevin!
Developer
Dec 18, 2014 at 5:16 AM
That's great news, if you wouldn't mind can you add an issue to the fsharp site that it interferes with CreateExpInstance.

Thank you for your participation,

Kevin