Roslyn and dynamic assemblies

Topics: APIs, General
Jun 19, 2014 at 10:13 AM
Hi,

According to the following link*, the possibility to emit dynamic assemblies from Roslyn has been removed since the CTP. This was to us one of the corner stones of Roslyn wrt dynamic code generation and domain specific languages.

Are there any plan to add this functionality back? Or is there any way around it?

Cheers,
Tanguy
Developer
Jun 29, 2014 at 7:42 AM
As the SO answer says it is indeed possible to compile source into a byte[] and then load it and run it via Assembly.Load(byte[]).

Is that not an option for your scenario? Do you need to compile and execute a lot of code snippets on the fly and you can't afford to create an assembly for each one of them? Or do you have a requirement to collect the generated code after it's not used?
Jun 30, 2014 at 9:54 AM
Hi Tomas,

The requirement is that the generated assemblies are collectible.

We have a domain specific language that generates intermediate C#. When the user redefines a function, the old generated assembly is discarded. In our use cases, collectible assemblies are a nice-to-have in production. But they are essential for unit testing, where thousands of transient assemblies are generated, tested and then discarded.

Cheers,
Tanguy
Jun 30, 2014 at 11:29 PM
Hi Tanguy,

We are doing something similar right now in writing an integration test engine that uses Roslyn to compile and run user-programmed integration tests from a web UI. As for you, a requirement for us was to be able to unload and collect the Assembly objects.

The way I am handling that right now is by always running the assemblies in a separate AppDomain. This does mean that you'll have to wrap your code in something that is MarsharlByRefType so you can get results out of it, but it is certainly possible. After my code in the temporary assembly is done, I tear down the AppDomain, which unloads the assembly and eventually will be GC'd itself.

There is some overhead here, which in my mind is not so easily avoided if you want to truly get rid of that code when you're done with it. However, in testing so far we have found the performance difference in spooling up an AppDomain to be fairly negligible.

-Matt
Developer
Jul 1, 2014 at 12:26 AM
There is certainly cost associated with creating that many AppDomains. I think the best solution would be to batch. E.g. create a new AppDomain for every 100 tests and then unload it. You can tweak the number of assemblies per AppDomain and see what works best.

Would that address your concerns?
Jul 1, 2014 at 12:16 PM
Marked as answer by TomasMatousek on 7/12/2014 at 9:52 PM
Jul 1, 2014 at 1:23 PM
Hi Matt/Tomas,

It is my understanding that to isolate a generated assemblies in a separate AppDomain, you need a narrow API (between the generated code and the main application) that you can wrap with marshaling.

In our case, the API is wide and highly flexible, and performance is critical. The complexity of our product/code base makes having a separate app domain impossible in practice (we've already investigated this before Roslyn arrived). In fact, one of the reason we've introduced so many unit tests is that we could refactor this massive code base.

I'm a bit confused why the support for collectible assemblies in Roslyn has been removed, given that collectible assemblies are already supported in .NET 4.0 but extremely unfriendly to use without a tool like Roslyn. Our experience with the last Roslyn CTP and collectible assemblies is highly convincing (we even have a unit test to ensure that the generated assembly is collected and that we don't forget to clear a state/collection).

Stéphane> you've got my votes.

-Tanguy
Developer
Jul 1, 2014 at 5:22 PM
We have removed the support for collectible assemblies since the underlying implementation in the CLR doesn't provide us the capabilities we need to build a fully functional C#/VB code gen on top of it. The implementation we had was mostly experimental with a lot of workarounds and problems.
Jul 2, 2014 at 12:01 AM
Edited Jul 2, 2014 at 12:03 AM
Tomas,

My intention was not to imply that the cost was zero, in fact there is certainly a cost. But in the realm of parsing the code, compiling it, generating an assembly, loading an assembly, and running the code, the cost of the AppDomain was pretty swamped in our profiling.

Tanguy,

Is the performance as critical in the development arena as it is in a production scenario? Perhaps this path of execution can be limited to a development only situation. Certainly our scripting engine is a little different in the API constraints. We do catch exceptions across the domains, but the rest we are only capturing console output and returning an IEnumerable<string> back to the script engine host.

However, there isn't a limit to what you can return with marshaling, and the CLR primitives and much of the BCL will marshal just fine. If there are other DTOs that need marshaling, it is certainly possible to add that capability.

Really the question is how small the code slices that need loading are, and if the goal is many unit tests, those are usually grouped well in their classes. Can the whole test group be loaded? I'm struggling to envision a situation in which the desired behavior is to dynamically compile method level code AND performance is critical. Usually those two goals will not be very compatible as a whole, AppDomain overhead aside. If performance is key, then cacheing is equally key, and you're going to have to spend memory to get the speed you want.

-Matt
Jul 2, 2014 at 10:21 AM
Hi Tomas,

Thanks for the answer.

I do find it a bit worrying that .NET 4 exposes an entire API for emitting code (including collectible assemblies) in System.Reflection.Emit, yet Roslyn fails to make use of it due to .NET/CLR limitations. Looks like a good opportunity for dogfooding and improving the platform as a whole.

Having a limited collectible assemblies support in Roslyn would be also acceptable (given that the CTP was giving us a lot already) if these limitations are well documented and easily avoidable for most practical use cases.

-Tanguy