This is the third part of a series about migrating a large application from XAF 12.1 to XAF 15.1. In this part I will compare the results of a simple stress test between the versions.
I have described in previous posts how to stress test XAF applications. One of our most basic tests is to simulate 25 users cycling through all the navigation tabs for an hour. I’m happy to report there is a considerable improvement under load in version 15.1.
(Note that we purposefully stress test against a single web application so that we can compare apples with apples. In production we have multiple instances load-balanced.)
Here is an interactive summary of the 15.1 results:
Here is version 15.1. There were zero errors and 382 completed scripts.
By comparison, the same test against DevExpress 12.1 yielded only 258 completions. So 15.1 shows a 48% performance improvement over 12.1.
This is the second part of a series about migrating a large application from XAF 12.1 to XAF 15.1.
In the 13.1 release, DevExpress made a change to the way XAF Validator class is used. It now requires an IObjectSpace parameter corresponding to the object. It is needed to correctly evaluate any rules which are descendants of the RuleSearchObjectProperties. These include:
RuleCombinationOfPropertiesIsUnique
RuleIsReferenced
RuleObjectExists
RuleUniqueValue
A lot of our code has been around for years now and the older parts rely heavily on Session and UnitOfWork instead of IObjectSpace For the most part our application used IObjectSpace only within ViewControllers.
But there were several situations where we need the validator where we don’t have an IObjectSpace. For instance we sometimes need to validate from within method actions (decorated with the ActionAttribute). For performance reasons, we pass criteria to our middleware and it uses a UnitOfWork to run the method on each object. So in this case, there was no IObjectSpace to pass to the XAF Validator.
Here I had a refactoring dilemma to solve. Either I need to rewrite all of the affected rules so that they no longer make use of the IObjectSpace. For instance, I could use RuleFromBoolProperty instead. In our application, this would mean rewriting about 50 rules. Or alternatively, I could go through the entire code base looking for new UnitOfWork() and new Session() and try to use an IObjectSpace instead. When writing code I often find myself having to decide between the ‘quick’ fix and the ‘right’ fix. Here, moving to IObjectSpace throughout is clearly the right fix and although it would take more time to implement, the system will be more in-line with best XAF practices throughout.
Eventually, the refactoring was complete and all unit tests are passing. I was eager to run a multi-user load stress test against the 15.1 version to compare performance under load. I have described in previous posts how to stress test XAF applications. I’ll be sharing the results in the next post.
I am the principal software architect for a treasury application in use by over 100 large multinational corporates. Upgrades are generally met with reluctance in the enterprise world and so we’ve been somewhat stuck on an old version of the expressAppFramework.
Recently I’ve been pushing to move to the newer version and I have spent about three weeks migrating the substantial code base to XAF 15.1 and .NET 4.6.
The steps are:
Run the project converter tool.
Try to compile. Identify the errors which are easily fixable (check with the ‘breaking changes’ documents from DevExpress.)
When in doubt, compare the libraries with a decompiler like .NET Reflector.
Refactor where necessary (ensure you have unit tests for the behaviour you are changing).
On this last point, my whole approach to refactoring has been shaped by Michael C. Feather’s book Working Effectively with Legacy Code. Highly recommended for anyone maintaining complex applications regardless of whether the code is legacy or not…
I was pleasantly surprised that I was very quickly able to get everything to compile and even run. The layout was not correct, but a lot of things worked straight away.
I then had to spend some time restoring the customisations we’d made to the default ASP.NET layout. In XAF 12.1 these were applied directly to default.aspx.cs and dialog.aspx.cs, but in 15.1 these no longer exist. Instead, you can customise layout by providing your own alternate templates. I was expecting this to be much harder, but by following the instructions in the documentation I managed to restore our layout quite easily.
I still had a lot of failing unit tests. One seemingly minor change to XAF validation proved to be a lot of work to fix in our code.
Since 13.1, the XAF Validator class now requires an IObjectSpace parameter in the constructor. This was by far the biggest problem to fix and is the subject of the next blog post.
How can I force a full garbage collection easily within an ASP.NET application? The method here is for XAF web applications but the same approach should work with any ASP.NET app.
First up: Never mess with the .NET garbage collector.
I sometimes mess with the garbage collector in .NET when I’m trying to pin down some memory problem. Also, after a load test, I prefer to force the garbage collector to collect everything it can so that I can check that the memory drops as expected.
Garbage collection in .NET is complex and it is hard to be sure you’ve done it correctly. This ancient article by Tess Ferrandez pointed me in the following direction.
123456789
voidForceGarbageCollection(){/// This will garbage collect all generations (including large object), GC.Collect(3);/// then execute any finalizersGC.WaitForPendingFinalizers();/// and then garbage collect again to take care of all the objects that had finalizers. GC.Collect(3);}
How can we easily trigger this routine in an XAF web application? First, add the following to the Global.asax.cs file:
When I recently upgraded to Visual Studio 2015, everything seemed to go very smoothly except that whenever I debugged my main application I got a dialog window with the following strange error:
The procedure entry point could not be located in the dynamic link library C:\...\bin\Debug\netutils.dll.
After pressing OK everything seemed to work as normal.
After a considerable number of dead ends, I finally worked out that changing the name of the NetUtils.dll assembly fixes the problem. It seems that Visual Studio 2015 gets confused with a Windows system assembly with the same name. I don’t know why it was never a problem with Visual Studio 2013, but I renamed the assembly and now everything is working fine.
For a while I’ve noticed an annoying slowness when debugging ASP.NET applications from Visual Studio. Just after every page load it takes about a second before the buttons become clickable. I noticed mostly when debugging XAF applications, perhaps because the pages are quite complex.
Turns out the culprit is something called Browser Link which was introduced in Visual Studio 2013. It’s enabled by default.
To turn it off you can turn it off from the menu:
Or you can add the following to your web.config file.
Here’s a quick hack when your CSV file has a different separator than Excel is expecting.
Add this on the first line of the CSV file:
sep=;
or
sep=,
This will override system setting for list separator character and Excel will open the file correctly.
Note: Excel expects the separator to match the one defined in the Control Panel/Region/Formats/Additional settings/List Separator. On a French system, it expects a semi-colon.
This post is an overview of the brand new version XAF 14.2. The truly outstanding new feature is the ASP.NET report writer which is now available in all XAF applications.
A few months ago, we lost a potential sale because the customer wanted the ability to create custom reports from within the browser. We told them it was impossible to provide a fully-fledged report designer within our web application - but the DevExpress guys have done it! And how!
The web-based report designer
Let’s fire up the MainDemo application and navigate to the reports view. The first thing to notice is that there is a new action Show Report Designer.
The designer action is disabled because the selected report is predefined. Predefined reports are a feature of Reports v2 which were introduced in version 13.2 (see my previous review). So first, we clone the existing predefined report. I renamed the copy (via the edit button) so that we can tell them apart.
Now the Show Report Designer action is enabled. Let’s click it. Whoa! That’s one impressive user interface for a web application!
Let’s add a chart and a few controls. I thought (incorrectly) that the link to the domain model might be somewhat lacking because the report designer is not designed specifically for XAF (you can also use it with non-XAF ASP.NET or ASP.NET MVC applications) but navigating the available domain objects to select a property seemed very natural and simple.
I had a few little mouse issues while trying to resize or move controls, and there were a couple of places where the interface seemed slightly sluggish, but these were very minor issues. In general the designer is slick and easy to use. I also had a little difficulty finding the Save button, but here it is:
And here’s the live output after my modifications.
You can also start from scratch with a new blank report.
This report designer is an extremely impressive achievement. I played around with it for over an hour and it did not crash once. I managed to implement everything I tried including a chart, a bar code and a new data field.
There are some features missing from the web-based report designer compared to the Windows Forms version. Most significant is the ability to attach events and scripts to controls. Here is a full feature comparison table.
I had a quick look for the tools they used to implement it. It looks like it uses jQuery, jQuery.UI and knockout.js and you can automatically bundle the required libraries via a new setting in the web.config. There is some more information here.
On the whole I am utterly impressed. Hats off to the DevExpress team!
Other new features in XAF 14.2
The new 14.2 includes several other new features. These include the ability to store user settings in the data store as well as improvements to the speed of the grids. For a full list of the new features and improvements see here and here.
It was written by Atif Aziz who happens to be an old school-friend from the International School of Geneva.
XAF provides quite extensive error handling options out of the box, but I have found Elmah better suited to production environments because of the ability to remotely view the full error log.
Setting up
First, get the ELMAH package via NuGet into the MainDemo.Web project. ELMAH provides dozens of different methods of persisting the error log. For this example we’ll choose one of the simplest. Make sure you select the ELMAH on XML Log package.
NuGet makes several automatic modifications to the web.config. Unfortunately, these are not quite accurate enough for XAF. The changes you need to make are detailed below:
Add a <configSection> for ELMAH as alongside the existing devExpress one.
web.config
123456789
<configSections><sectionGroupname="devExpress">...</sectionGroup><!-- this should already exist--><sectionGroupname="elmah"><!-- this is new--><sectionname="security"requirePermission="false"type="Elmah.SecuritySectionHandler, Elmah"/><sectionname="errorLog"requirePermission="false"type="Elmah.ErrorLogSectionHandler, Elmah"/><sectionname="errorMail"requirePermission="false"type="Elmah.ErrorMailSectionHandler, Elmah"/><sectionname="errorFilter"requirePermission="false"type="Elmah.ErrorFilterSectionHandler, Elmah"/></sectionGroup></configSections>
Your <system.webServer> section should look like this:
web.config
12345678910
<system.webServer><handlers>...</handlers><!-- This is unchanged --><validationvalidateIntegratedModeConfiguration="false"/><modules><addname="ASPxHttpHandlerModule"type="DevExpress.Web.ASPxClasses.ASPxHttpHandlerModule, DevExpress.Web.v14.1, Version=14.1.7.0, Culture=neutral, PublicKeyToken=b88d1754d700e49a"/><addname="ErrorLog"type="Elmah.ErrorLogModule, Elmah"preCondition="managedHandler"/><addname="ErrorMail"type="Elmah.ErrorMailModule, Elmah"preCondition="managedHandler"/><addname="ErrorFilter"type="Elmah.ErrorFilterModule, Elmah"preCondition="managedHandler"/></modules></system.webServer>
Add a <location> for the path elmah.axd (alongside the existing <location> tags).
web.config
123456789101112131415161718192021
<locationpath="elmah.axd"inheritInChildApplications="false"><system.web><httpHandlers><addverb="POST,GET,HEAD"path="elmah.axd"type="Elmah.ErrorLogPageFactory, Elmah"/></httpHandlers><!-- See http://code.google.com/p/elmah/wiki/SecuringErrorLogPages for more information on using ASP.NET authorization securing ELMAH. <authorization> <allow roles="admin" /> <deny users="*" /> </authorization> --></system.web><system.webServer><handlers><addname="ELMAH"verb="POST,GET,HEAD"path="elmah.axd"type="Elmah.ErrorLogPageFactory, Elmah"preCondition="integratedMode"/></handlers></system.webServer></location>
Add a new <elmah> section. I put mine just before the final </configuration> tag.
web.config
12345678
<elmah><errorLogtype="Elmah.XmlFileErrorLog, Elmah"logPath="~/App_Data/Elmah.Errors"/><!-- See http://code.google.com/p/elmah/wiki/SecuringErrorLogPages for more information on remote access and securing ELMAH. --><securityallowRemoteAccess="false"/></elmah>
Now modify HttpModules.Web.Config to look like this:
And then modify Global.asax.cs to instantiate the new class
1234567
protectedvoidApplication_Start(objectsender,EventArgse){ErrorHandling.Instance=newElmahErrorHandling();// <---this line is newASPxWebControl.CallbackError+=newEventHandler(Application_Error);#if DEBUGTestScriptsManager.EasyTestEnabled=true;#endif}
The complete files are available with the source code.
Now run the application and trigger an unhandled exception. Change the URL to something that does not exist. Or open any detail view and modify the URL so that the Guid in the ShortcutObjectKey is invalid (replace a digit with an ‘X’). Then the application error page appears.
Then return to the application and change the URL to Elmah.axd. You are looking at the log of all unhandled exceptions.
And for each exception, you can view the full details of any logged exception including coloured stack trace and full server variables.
ELMAH options
By default, ELMAH is configured to disallow remote access to the error logs - only a local user can get to elmah.axd. If you take care of the security implications it can be very useful to enable remote access and monitor the logs on your production servers.
We chose to use an XML file for each error but ELMAH is entirely pluggable. There are dozens of alternatives for persisting the error log including Sql Server, an RSS feeds, to Twitter, even to an iPhone app. There are even third party sites such as elmah.io who will host your error logs for you.
One of the advantages of using XML files is that the files can be copied to another machine. If you look in MainDemo.Web\App_Data\Elmah.Errors, you will find the resulting xml files.
You can just copy these files to another installation’s Elmah.Errors folder and the log will show up when you visit Elmah.axd.
One final note. ELMAH was developed for ASP.NET applications and web services, but it is possible to get it to work with other types of applications such as Windows Forms, Windows Service or console applications. Check out this StackOverflow question.
This is the fourth and final part of a series about using Roslyn with dependency injection to create a flexible and powerful plug-in framework. Here I review the parts of the solution that deal with the Roslyn runtime compilation of plug-ins. Check out the working example on GitHub.
Let’s look at some of the main classes used to compile plug-in code at runtime.
The PluginSnippetCompiler.Compile() method takes a string (for instance, the contents of an uploaded raw C# file) and converts it into an in-memory assembly with the same assembly references as the main project.
The Roslyn compiler is still in beta, and the Microsoft team have recently removed some syntactic sugar which made the code in the Compile() routine look cleaner. Hopefully they will include something similar soon. The code below works with version 0.7.0.0.
publicclassPluginSnippetCompiler{publicPluginSnippetCompiler(IAssemblyReferenceCollectorassemblyReferenceCollector){if(assemblyReferenceCollector==null)thrownewArgumentNullException("assemblyReferenceCollector");_AssemblyReferenceCollector=assemblyReferenceCollector;}privatereadonlyIAssemblyReferenceCollector_AssemblyReferenceCollector;privateIEnumerable<Diagnostic>_Diagnostics=Enumerable.Empty<Diagnostic>();publicIEnumerable<Diagnostic>Errors{get{return_Diagnostics.Where(d=>d.Severity==DiagnosticSeverity.Error);}}publicIEnumerable<Diagnostic>Warnings{get{return_Diagnostics.Where(d=>d.Severity==DiagnosticSeverity.Warning);}}privatestringGetOutputAssemblyName(stringname){returnString.Format("RoslynPlugins.Snippets.{0}",name);}/// <summary>/// Compiles source code at runtime into an assembly. The assembly will automatically include all/// the same assembly references as the main RoslynPlugins assembly, so you can call any function which is/// available from within the deployed RoslynPlugins. Compilation errors and warnings can be obtained from /// the Errors and Warnings properties./// </summary>/// <param name="name">The name of the class, e.g., HelloWorldGenerator</param>/// <param name="script">Source code such as the contents of HelloWorldGenerator.cs</param>/// <returns>The compiled assembly in memory. If there were errors, it will return null.</returns>publicAssemblyCompile(stringname,stringscript){if(name==null)thrownewArgumentNullException("name");if(script==null)thrownewArgumentNullException("script");stringoutputAssemblyName=GetOutputAssemblyName(name);vardefaultImplementationAssembly=typeof(HelloWorldGenerator).Assembly;varassemblyReferences=_AssemblyReferenceCollector.CollectMetadataReferences(defaultImplementationAssembly);// Parse the script to a SyntaxTreevarsyntaxTree=CSharpSyntaxTree.ParseText(script);// Compile the SyntaxTree to an in memory assemblyvarcompilation=CSharpCompilation.Create(outputAssemblyName,new[]{syntaxTree},assemblyReferences,newCSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary));using(varoutputStream=newMemoryStream()){using(varpdbStream=newMemoryStream()){// Emit assembly to streams. Throw an exception if there are any compilation errorsvarresult=compilation.Emit(outputStream,pdbStream:pdbStream);// Populate the _diagnostics property in order to read Errors and Warnings_Diagnostics=result.Diagnostics;if(result.Success){returnAssembly.Load(outputStream.ToArray(),pdbStream.ToArray());}else{returnnull;}}}}}
In this demo, I have not included any user feedback about compilation errors, but they are easily obtainable from the Errors and Warnings properties. At present, if there is an error, the plug-in will be ignored and the original implementation will be used.
The class above depends on an AssemblyReferenceCollector which is responsible for enumerating the references to add to the runtime-generated plug-in assembly. We want exactly the same assembly references as the assembly which contains the original implementation so that we can reference any dependencies within those references.
1234567891011121314151617181920
publicclassAssemblyReferenceCollector:IAssemblyReferenceCollector{publicIEnumerable<MetadataReference>CollectMetadataReferences(Assemblyassembly){varreferencedAssemblyNames=assembly.GetReferencedAssemblies();varreferences=newList<MetadataReference>();foreach(AssemblyNameassemblyNameinreferencedAssemblyNames){varloadedAssembly=Assembly.Load(assemblyName);references.Add(newMetadataFileReference(loadedAssembly.Location));}references.Add(newMetadataFileReference(assembly.Location));// add a reference to 'self', i.e., NetMWCreturnreferences;}}
Connecting the pieces
We need the PluginLocator class to connect the Ninject resolution root to the runtime-generated assembly (if one exists). It just looks for classes with the correct interface IGenerator within the PluginAssemblyCache.
publicclassPluginLocator{publicPluginLocator(PluginAssemblyCachepluginAssemblyCache){if(pluginAssemblyCache==null)thrownewArgumentNullException("pluginAssemblyCache");_PluginAssemblyCache=pluginAssemblyCache;}privatereadonlyPluginAssemblyCache_PluginAssemblyCache;publicTypeLocate<T>(){returnLocate(new[]{typeof(T)});}protectedTypeLocate(IEnumerable<Type>serviceTypes){varimplementingClasses=AssemblyExplorer.GetImplementingClasses(_PluginAssemblyCache.GetAssemblies(),serviceTypes);if(implementingClasses.Any()){if(implementingClasses.Count()>1)thrownewException("More than one plugin class found which implements "+String.Join(" + ",serviceTypes.Select(t=>t.ToString())));elsereturnimplementingClasses.Single();}returnnull;}}
The PluginAssemblyCache avoids having to run the Compile() routine more than once by maintaining a dictionary of previously compiled plug-ins. It has the following dependencies:
an IPluginSnippetProvider which (in this case) reads the existing snippets from the database (not shown here)
a PluginLoader which uses the above PluginSnippetCompiler to convert a snippet into a runtime assembly.
/// <summary>/// This class maintains a list of runtime-compiled in memory assemblies loaded from the plugins/// available via the provider. It is a singleton class./// </summary>publicclassPluginAssemblyCache{publicPluginAssemblyCache(IPluginSnippetProviderpluginSnippetProvider,PluginLoaderpluginLoader){if(pluginSnippetProvider==null)thrownewArgumentNullException("pluginSnippetProvider");_PluginSnippetProvider=pluginSnippetProvider;if(pluginLoader==null)thrownewArgumentNullException("pluginLoader");_PluginLoader=pluginLoader;}privateclassCacheEntry{publicstringName{get;set;}publicVersionVersion{get;set;}publicAssemblyAssembly{get;set;}}privatereadonlyIPluginSnippetProvider_PluginSnippetProvider;privatereadonlyPluginLoader_PluginLoader;privateList<CacheEntry>_Cache=newList<CacheEntry>();privatevoidAdd(stringname,stringversion,Assemblyassembly){varcacheEntry=newCacheEntry(){Name=name,Version=newVersion(version),Assembly=assembly};_Cache.Add(cacheEntry);}privatevoidRefreshCache(){varpluginScriptContainers=_PluginSnippetProvider.GetPlugins();// Add a new assembly for any new or updated pluginforeach(varpluginScriptContainerinpluginScriptContainers){varname=pluginScriptContainer.Name;varversion=pluginScriptContainer.Version;if(!_Cache.Any(a=>a.Name==name&&a.Version==newVersion(version))){varassembly=_PluginLoader.Load(pluginScriptContainer);Add(name,version,assembly);}}// Remove any assemblies which we no longer have a plugin for._Cache.RemoveAll(cacheEntry=>!pluginScriptContainers.Select(plugin=>plugin.Name).Contains(cacheEntry.Name));}publicIEnumerable<Assembly>GetAssemblies(){RefreshCache();// Return only the assemblies with the highest version numbersreturn_Cache.GroupBy(d=>d.Name).Select(g=>g.OrderByDescending(d=>d.Version).First().Assembly);}}
So whenever the SomeGenerator class is resolved by Ninject, it will now
Check whether there are any new plug-ins and compile them into runtime assemblies and add them to the PluginAssemblyCache.
Then the PluginLocator will search these assemblies for a newer version of SomeGenerator.
If it finds one, it will be resolved along with any constructor dependencies, otherwise it will use the original SomeGenerator.
Version numbers
The version number of the plug-in is a key part of our solution. Let’s say you have version 1.0 in production. Then you fix some bugs in staging (version 1.1). You create a plug-in from this staging code and upload it into production. Then much later, you decide to upgrade production to 1.2. Then, with the query in GetAssemblies(), the 1.1 plug-in will automatically be ignored and be superseded by whatever was shipped with 1.2 since that is newer code. So we do not have to remember to remove obsolete plug-ins after an upgrade - they will automatically be ignored because of the version number.
Security
Obviously, security is a chief concern and you may have to secure the plug-ins. In this demo project, I just created a simple view for the IPlugin object, but in our production environment we handle the creation of plug-ins differently. We use a combination of role-based security (to control who has permission to upload plugins) and encryption with checksumming. No user can directly enter arbitrary code - instead, we send the user a zip file which contains the code (encrypted), the version number and a checksum and our application verifies the checksum and builds the IPlugin object from the contents of the zip. A Powershell script running on our build server is responsible for creating the checksummed plug-in directly from the source code used in our staging environment.
Conclusions - the ultimate plug-in framework?
The strength of the Roslyn approach is that it is easy to maintain while being extremely versatile. In our case, it provides us with the ability to restrict the number of major releases approximately one per annum while catering for the inevitable little fixes to output formats and reports.
In the example we replaced an existing class, but it would be straightforward to add the concept of discovery and use the same Roslyn features to make any new plug-in classes available to your application. Ninject, makes it easy to instantiate, say, every implementor of IGenerator, so you could enumerate all available plug-ins instead of replacing a single one.
So here’s a basic plug-in framework which is very flexible and very powerful without many of the versioning headaches of MEF or MAF. It’s also easy to maintain, since the plug-in code is identical to the ‘normal’ code in staging (just packaged, delivered and compiled in a different way to production).