This is the fourth and final part of a series about using Roslyn with dependency injection to create a flexible and powerful plug-in framework. Here I review the parts of the solution that deal with the Roslyn runtime compilation of plug-ins. Check out the working example on GitHub.
Let’s look at some of the main classes used to compile plug-in code at runtime.
The PluginSnippetCompiler.Compile() method takes a string (for instance, the contents of an uploaded raw C# file) and converts it into an in-memory assembly with the same assembly references as the main project.
The Roslyn compiler is still in beta, and the Microsoft team have recently removed some syntactic sugar which made the code in the Compile() routine look cleaner. Hopefully they will include something similar soon. The code below works with version 0.7.0.0.
publicclassPluginSnippetCompiler{publicPluginSnippetCompiler(IAssemblyReferenceCollectorassemblyReferenceCollector){if(assemblyReferenceCollector==null)thrownewArgumentNullException("assemblyReferenceCollector");_AssemblyReferenceCollector=assemblyReferenceCollector;}privatereadonlyIAssemblyReferenceCollector_AssemblyReferenceCollector;privateIEnumerable<Diagnostic>_Diagnostics=Enumerable.Empty<Diagnostic>();publicIEnumerable<Diagnostic>Errors{get{return_Diagnostics.Where(d=>d.Severity==DiagnosticSeverity.Error);}}publicIEnumerable<Diagnostic>Warnings{get{return_Diagnostics.Where(d=>d.Severity==DiagnosticSeverity.Warning);}}privatestringGetOutputAssemblyName(stringname){returnString.Format("RoslynPlugins.Snippets.{0}",name);}/// <summary>/// Compiles source code at runtime into an assembly. The assembly will automatically include all/// the same assembly references as the main RoslynPlugins assembly, so you can call any function which is/// available from within the deployed RoslynPlugins. Compilation errors and warnings can be obtained from /// the Errors and Warnings properties./// </summary>/// <param name="name">The name of the class, e.g., HelloWorldGenerator</param>/// <param name="script">Source code such as the contents of HelloWorldGenerator.cs</param>/// <returns>The compiled assembly in memory. If there were errors, it will return null.</returns>publicAssemblyCompile(stringname,stringscript){if(name==null)thrownewArgumentNullException("name");if(script==null)thrownewArgumentNullException("script");stringoutputAssemblyName=GetOutputAssemblyName(name);vardefaultImplementationAssembly=typeof(HelloWorldGenerator).Assembly;varassemblyReferences=_AssemblyReferenceCollector.CollectMetadataReferences(defaultImplementationAssembly);// Parse the script to a SyntaxTreevarsyntaxTree=CSharpSyntaxTree.ParseText(script);// Compile the SyntaxTree to an in memory assemblyvarcompilation=CSharpCompilation.Create(outputAssemblyName,new[]{syntaxTree},assemblyReferences,newCSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));using(varoutputStream=newMemoryStream()){using(varpdbStream=newMemoryStream()){// Emit assembly to streams. Throw an exception if there are any compilation errorsvarresult=compilation.Emit(outputStream,pdbStream:pdbStream);// Populate the _diagnostics property in order to read Errors and Warnings_Diagnostics=result.Diagnostics;if(result.Success){returnAssembly.Load(outputStream.ToArray(),pdbStream.ToArray());}else{returnnull;}}}}}
In this demo, I have not included any user feedback about compilation errors, but they are easily obtainable from the Errors and Warnings properties. At present, if there is an error, the plug-in will be ignored and the original implementation will be used.
The class above depends on an AssemblyReferenceCollector which is responsible for enumerating the references to add to the runtime-generated plug-in assembly. We want exactly the same assembly references as the assembly which contains the original implementation so that we can reference any dependencies within those references.
1234567891011121314151617181920
publicclassAssemblyReferenceCollector:IAssemblyReferenceCollector{publicIEnumerable<MetadataReference>CollectMetadataReferences(Assemblyassembly){varreferencedAssemblyNames=assembly.GetReferencedAssemblies();varreferences=newList<MetadataReference>();foreach(AssemblyNameassemblyNameinreferencedAssemblyNames){varloadedAssembly=Assembly.Load(assemblyName);references.Add(newMetadataFileReference(loadedAssembly.Location));}references.Add(newMetadataFileReference(assembly.Location));// add a reference to 'self', i.e., NetMWCreturnreferences;}}
Connecting the pieces
We need the PluginLocator class to connect the Ninject resolution root to the runtime-generated assembly (if one exists). It just looks for classes with the correct interface IGenerator within the PluginAssemblyCache.
publicclassPluginLocator{publicPluginLocator(PluginAssemblyCachepluginAssemblyCache){if(pluginAssemblyCache==null)thrownewArgumentNullException("pluginAssemblyCache");_PluginAssemblyCache=pluginAssemblyCache;}privatereadonlyPluginAssemblyCache_PluginAssemblyCache;publicTypeLocate<T>(){returnLocate(new[]{typeof(T)});}protectedTypeLocate(IEnumerable<Type>serviceTypes){varimplementingClasses=AssemblyExplorer.GetImplementingClasses(_PluginAssemblyCache.GetAssemblies(),serviceTypes);if(implementingClasses.Any()){if(implementingClasses.Count()>1)thrownewException("More than one plugin class found which implements "+String.Join(" + ",serviceTypes.Select(t=>t.ToString())));elsereturnimplementingClasses.Single();}returnnull;}}
The PluginAssemblyCache avoids having to run the Compile() routine more than once by maintaining a dictionary of previously compiled plug-ins. It has the following dependencies:
an IPluginSnippetProvider which (in this case) reads the existing snippets from the database (not shown here)
a PluginLoader which uses the above PluginSnippetCompiler to convert a snippet into a runtime assembly.
/// <summary>/// This class maintains a list of runtime-compiled in memory assemblies loaded from the plugins/// available via the provider. It is a singleton class./// </summary>publicclassPluginAssemblyCache{publicPluginAssemblyCache(IPluginSnippetProviderpluginSnippetProvider,PluginLoaderpluginLoader){if(pluginSnippetProvider==null)thrownewArgumentNullException("pluginSnippetProvider");_PluginSnippetProvider=pluginSnippetProvider;if(pluginLoader==null)thrownewArgumentNullException("pluginLoader");_PluginLoader=pluginLoader;}privateclassCacheEntry{publicstringName{get;set;}publicVersionVersion{get;set;}publicAssemblyAssembly{get;set;}}privatereadonlyIPluginSnippetProvider_PluginSnippetProvider;privatereadonlyPluginLoader_PluginLoader;privateList<CacheEntry>_Cache=newList<CacheEntry>();privatevoidAdd(stringname,stringversion,Assemblyassembly){varcacheEntry=newCacheEntry(){Name=name,Version=newVersion(version),Assembly=assembly};_Cache.Add(cacheEntry);}privatevoidRefreshCache(){varpluginScriptContainers=_PluginSnippetProvider.GetPlugins();// Add a new assembly for any new or updated pluginforeach(varpluginScriptContainerinpluginScriptContainers){varname=pluginScriptContainer.Name;varversion=pluginScriptContainer.Version;if(!_Cache.Any(a=>a.Name==name&&a.Version==newVersion(version))){varassembly=_PluginLoader.Load(pluginScriptContainer);Add(name,version,assembly);}}// Remove any assemblies which we no longer have a plugin for._Cache.RemoveAll(cacheEntry=>!pluginScriptContainers.Select(plugin=>plugin.Name).Contains(cacheEntry.Name));}publicIEnumerable<Assembly>GetAssemblies(){RefreshCache();// Return only the assemblies with the highest version numbersreturn_Cache.GroupBy(d=>d.Name).Select(g=>g.OrderByDescending(d=>d.Version).First().Assembly);}}
So whenever the SomeGenerator class is resolved by Ninject, it will now
Check whether there are any new plug-ins and compile them into runtime assemblies and add them to the PluginAssemblyCache.
Then the PluginLocator will search these assemblies for a newer version of SomeGenerator.
If it finds one, it will be resolved along with any constructor dependencies, otherwise it will use the original SomeGenerator.
Version numbers
The version number of the plug-in is a key part of our solution. Let’s say you have version 1.0 in production. Then you fix some bugs in staging (version 1.1). You create a plug-in from this staging code and upload it into production. Then much later, you decide to upgrade production to 1.2. Then, with the query in GetAssemblies(), the 1.1 plug-in will automatically be ignored and be superseded by whatever was shipped with 1.2 since that is newer code. So we do not have to remember to remove obsolete plug-ins after an upgrade - they will automatically be ignored because of the version number.
Security
Obviously, security is a chief concern and you may have to secure the plug-ins. In this demo project, I just created a simple view for the IPlugin object, but in our production environment we handle the creation of plug-ins differently. We use a combination of role-based security (to control who has permission to upload plugins) and encryption with checksumming. No user can directly enter arbitrary code - instead, we send the user a zip file which contains the code (encrypted), the version number and a checksum and our application verifies the checksum and builds the IPlugin object from the contents of the zip. A Powershell script running on our build server is responsible for creating the checksummed plug-in directly from the source code used in our staging environment.
Conclusions - the ultimate plug-in framework?
The strength of the Roslyn approach is that it is easy to maintain while being extremely versatile. In our case, it provides us with the ability to restrict the number of major releases approximately one per annum while catering for the inevitable little fixes to output formats and reports.
In the example we replaced an existing class, but it would be straightforward to add the concept of discovery and use the same Roslyn features to make any new plug-in classes available to your application. Ninject, makes it easy to instantiate, say, every implementor of IGenerator, so you could enumerate all available plug-ins instead of replacing a single one.
So here’s a basic plug-in framework which is very flexible and very powerful without many of the versioning headaches of MEF or MAF. It’s also easy to maintain, since the plug-in code is identical to the ‘normal’ code in staging (just packaged, delivered and compiled in a different way to production).