A month ago

I’ve never liked laptops. The screen seems so small. A powerful laptop is too heavy; a light one doesn’t cut the mustard. I can’t even find the on/off switch.

Then there are those fancy new Macbook Pros with retina displays. They seem light and powerful and the screen looks great. I don’t know much about Macs, but I’d like to learn.

But I need Microsoft Windows and Visual Studio. I know theoretically I can run a virtual machine or setup a dual boot, but I’ll bet it’s all a bit of a hassle and rather clunky…

Three hours after my Macbook Pro arrived

Windows 8 is installed and looks absolutely beautiful! The font is small, but the screen is so sharp that it is quite readable. Visual Studio looks great with the font size in the editor Window set to 14 or 15.

The installation was much simpler than most other Windows installations I’ve done. I followed this tutorial on iClarified to set up a dual boot. You use a tool called the Bootcamp Advisor which is already installed on your Mac to create an bootable USB for installing Windows. Since all Macs are the same, the Bootcamp Advisor sets up all the drivers correctly for you.

Three weeks later

This is by far the most pleasant laptop I’ve ever used for development.

The screen is wonderful - the machine seems very fast and it is no weight to carry around.

A tiny little gripe

There is a little annoyance with the Bootcamp Control Panel which won’t start properly, but Thomas Jespersen found a workaround, which I’ve put in a batch file. You need this to make the F1-12 keys work as they would in Windows.

Some help with the Mac keyboard and trackpad

I’m very picky about keyboards and I need to be able to find all my Visual Studio shortcuts in the same places. All in all I find the keyboard fine to work with.

The delete key works like a PC backspace so you have to do fn+delete to delete forwards. Here’s a list of other shortcuts.

  There are some keys which you will still not find, for instance, NumLock, PrtScn, Break.

If I need these, I use the On-Screen keyboard (type ‘osk’ into the search bar). If you cannot see the NumLock, the options allow you to show the numeric keypad. Alternatively, you could always plug in a USB keyboard.

The trackpad works great including the two-finger click for right-click and two-finger swipe for scroll.

In this post I’ll demonstrate how to use the Google Page Speed tools to analyse the performance of the DevExpress XAF MainDemo.

The easiest way to run Google Page Speed is as a Chrome or Firefox plugin. Both are available here. I use Chrome.

Now open the DevExpress MainDemo from Visual Studio. By default it is installed to:

C:\Users\Public\Documents\DXperience 12.1 Demos\eXpressApp Framework\MainDemo\CS\MainDemo.sln

Set the MainDemo.Web as the startup project and change the connection string in web.config if necessary. Launch the application with Chrome and login as ‘Sam’ (password is blank). Then press F12 to bring up the developer tools. The last tab is the Page Speed Analysis and your browser should look like this:

Now click Start Analysis button. After a few seconds you should get something like the following results.

Results against the debug webserver

It’s as easy as that. We now have a list of suggested improvements. The same report can be generated for any page you visit with your browser.

I get an overall score of 72 out of 100. First thing to note is that the only ‘high priority’ recommendation is to ‘enable keep-alive’ which I suspect will not be necessary when running in IIS instead of the debug webserver.

Switch to use IIS

Actually to get the main demo to run in IIS is not altogether simple because of the security permissions required for logging in and creating or updating the schema. If you get an error message after login:

Login failed for user 'IIS APPPOOL\DefaultAppPool

you will need to add the IIS application pool identity to the SQL Server security.

• Launch SQL Server management studio and connect to the database.
• In the Security\Logins right click and select New Login....
• Type in IIS APPPOOL\DefaultAppPool (you won’t find it by searching) or IIS APPPOOL\ASP.NET v4.0 depending on the security context of the application pool you are using.
• Select Server Roles and check public and sysadmin to allow the MainDemo to create the database.

(All of this is assuming you are using a non-public instance of SQL Server for development.)

Results against IIS

The results are much better: an overall score of 93.

Other points of interest

Let’s experiment by turning off compression in the webconfig.

<compression 
  enableHtmlCompression="false" 
  enableCallbackCompression="false" 
  enableResourceCompression="false" 
  enableResourceMerging="false" />

The overall score drops to 62.

You can alternatively use IIS’s dynamic compression by setting enableResourceMerging="true" and the others false and adding a urlCompression setting as follows.

<system.webServer>
  <urlCompression doDynamicCompression="true" />
  ...
</system.webServer> 

(Note that you may need to install the dynamic compression module via Control Panel/Programs/Turn Windows Features On or Off.)

Then the analysis is back up to 93. The advantage of IIS dynamic compression in IIS 7 is that it turns itself off automatically when the CPU load is high. See Matt Perdeck’s series of articles about IIS Compression for more information.

One mysterious point: if you navigate to the online version of the MainDemo and run the analysis there you will notice that compression resource merging must be turned off for some reason and the overall score is only 75. Perhaps someone from DevExpress can explain…

The source code for the fast batch operations with DevExpress XPO is now available on on GitHub.

It includes unit tests that can be run against SQL Server or in memory. It runs against DevExpress 12.1.7, but it will work against 11.2 (and possibly earlier versions) with a couple of minor modifications which are documented in the code.

References

• Fast batch deletions
• Fast batch modifications

Last week I wrote about fast batch deletions. In this post I’ll show how to do the same for modifications.

Let’s assume we want to replace the ‘State’ property with ‘CA’ and CostCenter with 123 for all records where the ‘City’ is ‘San Francisco’. The recommended DevExpress approach would be something like the following:

1
2
3
4
5
6
7
8
9
10

using (UnitOfWork uow = new UnitOfWork())
{  
  var xpCollection = new XPCollection<MyObject>(uow, CriteriaOperator.Parse("City == 'San Francisco'"));
    foreach (MyObject myObject in xpCollection)
    {
        myObject.State = "CA";
        myObject.CostCenter = 123;
    }
    uow.CommitChanges();
}

The problem with the above code is that every record must be loaded and then an individual UPDATE command is generated for each modification. This is necessary for the business logic to be applied correctly (such as the code in methods such as OnSaving()). It is also necessary to handle record locking.

If you know that your objects do not require any of this processing, you can use use direct SQL as described in the XPO documentation. This however requires knowledge of the underlying database table and is not very versatile, (although the DevExpress.Data.Filtering.CriteriaToWhereClauseHelper() can help if you choose this route).

However, there is a method similar to the one described in the previous post which is equivalent to the direct SQL approach, but is much easier to use. The approach makes use of an extension method on the Session class.

Example

Since the extension method is somewhat more complicated than for the Delete case, I will start by showing an example of use before drilling into the supporting code.

The above example would now look like this:

1
2
3
4
5
6
7
8
9
10

using (UnitOfWork uow = new UnitOfWork())
{
  uow.Update<MyObject>(
        () => new MyObject(uow)
                  {
                     State = "CA",
                     CostCenter = 123
                  },
        CriteriaOperator.Parse("City == 'San Francisco'"));
}

The Update method takes an Expression<Func<T>> as the first parameter which allows us to pass in an anonymous type which serves as a template for the modification. This way we get strong typing for the property values.

The extensions method

Now for the guts of it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

public class PropertyValueStore : List<KeyValuePair<XPMemberInfo, Object>>
{
}

public static class SessionExtensions
{
    public static PropertyValueStore CreatePropertyValueStore(XPClassInfo classInfo, MemberInitExpression memberInitExpression)
    {
        PropertyValueStore propertyValueStore = new PropertyValueStore();

        /// Parse each expression binding within the anonymous class.  
        /// Each binding represents a property assignment within the IXPObject.
        /// Add a KeyValuePair for the corresponding MemberInfo and (invoked) value.
        foreach (var binding in memberInitExpression.Bindings)
        {
            var assignment = binding as MemberAssignment;
            if (binding == null)
            {
                throw new NotImplementedException("All bindings inside the MemberInitExpression are expected to be of type MemberAssignment.");
            }

            // Get the memberInfo corresponding to the property name.
            string memberName = binding.Member.Name;
            XPMemberInfo memberInfo = classInfo.GetMember(memberName);
            if (memberInfo == null)
                throw new ArgumentOutOfRangeException(memberName, String.Format("The member {0} of the {1} class could not be found.", memberName, classInfo.FullName));

            if (!memberInfo.IsPersistent)
                throw new ArgumentException(memberName, String.Format("The member {0} of the {1} class is not persistent.", memberName, classInfo.FullName));

            // Compile and invoke the assignment expression to obtain the contant value to add as a parameter.
            var constant = Expression.Lambda(assignment.Expression, null).Compile().DynamicInvoke();

            // Add the 
            propertyValueStore.Add(new KeyValuePair<XPMemberInfo, Object>(memberInfo, constant));
        }
        return propertyValueStore;
    }

    public static ModificationResult Update<T>(this Session session, Expression<Func<T>> evaluator, CriteriaOperator criteria) where T : IXPObject
    {
        if (ReferenceEquals(criteria, null))
            criteria = CriteriaOperator.Parse("True");

        XPClassInfo classInfo = session.GetClassInfo(typeof(T));
        var batchWideData = new BatchWideDataHolder4Modification(session);
        int recordsAffected = (int)session.Evaluate<T>(CriteriaOperator.Parse("Count()"), criteria);

        /// Parse the Expression.
        /// Expect to find a single MemberInitExpression.
        PropertyValueStore propertyValueStore = null;
        int memberInitCount = 1;
        evaluator.Visit<MemberInitExpression>(expression =>
            {
                if (memberInitCount > 1)
                {
                    throw new NotImplementedException("Only a single MemberInitExpression is allowed for the evaluator parameter.");
                }
                memberInitCount++;
                propertyValueStore = CreatePropertyValueStore(classInfo, expression);
                return expression;
            });

        MemberInfoCollection properties = new MemberInfoCollection(classInfo, propertyValueStore.Select(x => x.Key).ToArray());

        List<ModificationStatement> collection = UpdateQueryGenerator.GenerateUpdate(classInfo, properties, criteria, batchWideData);
        foreach (UpdateStatement updateStatement in collection.OfType<UpdateStatement>())
        {
            for (int i = 0; i < updateStatement.Parameters.Count; i++)
            {
                Object value = propertyValueStore[i].Value;
                if (value is IXPObject)
                    updateStatement.Parameters[i].Value = ((IXPObject)(value)).ClassInfo.GetId(value);
                else
                    updateStatement.Parameters[i].Value = value;
            }
            updateStatement.RecordsAffected = recordsAffected;
        }
        return session.DataLayer.ModifyData(collection.ToArray<ModificationStatement>());
    }
}

Limitations

There is currently no way to refer to another field within the assignment expressions - you can only set the value to an OperandValue. So you cannot do

1
2
3
4
5
6
7
8
9

   uow.Update<MyObject>(
        o => new MyObject(uow)
                  {
                     // Does not Compile !!!
                     Property1 = o.Property2,
                     // Neither does this !!!
                     Property3 = o.Property3 + 1
                  },
        null);

In order to fix this, the evaluator has to be of type Expression<Func<T, T>> instead of Expression<Func<T>>, and then you can use expression trees to get an assignment expression. But then there is no way to pass it to a DevExpress UpdateStatement.Parameter as an OperandValue.

Update: The source code is now available on GitHub.

References

The code was inspired by an old blog post Terry Aney in which he describes a similar approach for LINQ to SQL.

When deleting a collection of objects, DevExpress recommends using Session.Delete(ICollection objects). This has the same effect as calling the Delete() method for every object in the collection so that the business logic is applied correctly. The business logic in this context refers to code such as that in the OnDeleting(), OnDeleted() methods, but it also includes the clearing of references to the object by other objects. This approach is slow, but ensures the integrity of the data.

If you know that your objects do not require any of this processing, you can use use direct SQL as described in the XPO documentation. This however requires knowledge of the underlying database table and is not very versatile, (although the DevExpress.Data.Filtering.CriteriaToWhereClauseHelper() can help if you choose this route).

An alternative is to use the extension method below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

public static class SessionExtensions
{
    public static ModificationResult Delete<T>(this Session session, CriteriaOperator criteria = null) where T : IXPObject
    {
        if (ReferenceEquals(criteria, null))
            criteria = CriteriaOperator.Parse("True");

        XPClassInfo classInfo = session.GetClassInfo(typeof(T));
        var batchWideData = new BatchWideDataHolder4Modification(session);
        int recordsAffected = (int)session.Evaluate<T>(CriteriaOperator.Parse("Count()"), criteria);
        List<ModificationStatement> collection = DeleteQueryGenerator.GenerateDelete(classInfo, criteria, batchWideData);
        foreach (ModificationStatement item in collection)
        {
            item.RecordsAffected = recordsAffected;
        }

        ModificationStatement[] collectionToArray = collection.ToArray<ModificationStatement>();
        ModificationResult result = session.DataLayer.ModifyData(collectionToArray);
        return result;
    }
}

Here is an example of how to call the method:

1
2
3
4
5

using (UnitOfWork uow = new UnitOfWork())
{
    uow.Delete<MyObject>(CriteriaOperator.Parse("City != 'Chicago'"));
    uow.CommitChanges();
}

This achieves the same as result and similar performance to direct SQL, but with cleaner syntax and support for criteria. Also, since it uses a ModificationStatement[], it works with a remote IDataStore.

See the next post for a similar approach for fast batch modifications.

Update: The source code is now available on GitHub.

My new keyboarf keyboard has no labels on the keys!

On a normal keyboard, I can type at about 60 wpm, but I only use three or four fingers and I have to periodically glance down at the keyboard. I hope my new keyboard forces me to learn to touch type properly. Various coders have written on the subject - Jeff Atwood says we are typists first; Phil Haack opts for Dvorak.

The Das Keyboard is beautiful. It’s certainly heavier and noisier than other computer keyboards but with such nicely-weighted quality keys that it is a pleasure to type on. At the moment I am struggling, especially when it comes to brackets () [] {} and punctuation symbols. Looking down is of no use whatsoever. Having your hands positioned correctly (with forefingers on the tiny bumps on the f and j keys) helps a lot, although my ring finger and pinkie are not used to any action whatsoever.

I’ve been trying some typing games too. Suggestions welcome. My typing speed is currently about 20 wpm with the new keyboard, so I have some catching up to do…

On Saturday I attended an excellent discussion at the Tate modern which was part of a series of lectures on Topology. The first part was a screening of Au Bonheur des Maths, a film by Raymond Depardon and Claudine Nougaret which is a collection of portraits of famous mathematicians describing beauty in mathematics. The second part was a discussion on the same topic between Sir Michael Atiyah and Cédric Villani, both winners of the Fields Medal. You can read a review of the event at Tony Mann’s maths blog.

I particularly enjoyed witnessing world-class mathematicians in the flesh and learning a little about their day-to-day work. Sir Michael Atiyah is a jovial fellow in his 80s who speaks extremely quickly as if his mouth is trying to keep up with his thoughts. His passion for mathematics is infectious.

In stark comparison Cédric Villani is a flamboyant Frenchman who was dressed in a three-piece suit and a cravat. He looks more like a musketeer than a mathematician. He was wearing an ornate and sparkly jewel in the shape of a spider on his lapel (and apparently always does). He speaks very good English but with quite a thick French accent. He has a mischevous sense of humour and an endless enthusiasm. Check out his great TEDx talk.

Altogether the atmosphere was one of relaxed, humorous, intellectual entertainment. Both mathematicians drew on anecdotes from the history of mathematics. It was clear that both of them derive considerable pleasure from sharing their knowledge and experiences.

There were many aspects of their work that I felt I could relate to as a software developer: looking for answers sometimes feels like you are exploring a valley and sometimes like you are fighting a battle. Often the strict constraints of the system force you to be more creative to break away from them and towards new discovery. Being stuck is a common and natural occurrence.

One of the interesting digressions was regarding a famous question in topology - how to turn a sphere inside-out (which is known as eversion). Apparently, when Arnold Shapiro described his work on the problem to Bernard Morin who is blind, Morin was able to visualise the transformation, but Shapiro had the distinct impression that he was not taking any particular point of view. Perhaps this is similar to the when you think of holding a sphere in your hands rather than looking at it.

If you have a few minutes, you can learn how to turn a sphere inside-out in the following amazing video.

Sir Michael closed the discussion with the following quotation:

My work always tried to unite the truth with the beautiful, but when I had to choose one or the other, I usually chose the beautiful.

References

• Photo credits: Raymond Depardon/Magnum Photos
• Video credit: Outside In

This post is part of an ongoing series about robots, past and present. See Musical Interlude.

The Mars Opportunity Rover

Opportunity landed on Mars in January 2004. It is currently the only active robot on any planet other than Earth (until the scheduled arrival of Curiosity in August 2012). The mission was originally supposed to last 90 days.

The rover contains pieces of the fallen World Trade Center which were turned into shields to protect cabling.

This photo of the Endeavour Crater on Mars was taken by the Opportunity rover on March 9, 2012.

Hardware - if it ain’t broke, don’t fix it

NASA prefers hardware that has been used successfully on previous missions, even if it means compromising on performance.

The computer on board is a RAD6000. It is single board computer designed for the US Air Force by BAE Systems with a 32-bit processor based on the IBM RISC single chip CPU. It has 128 MB of RAM.It is radiation hardened so that it can withstand the harsh environment of space. The RAD6000 is reported to cost between USD 200,000 and USD 300,000.

It’s a venerable board. We’ve used it ever since the Mars Pathfinder Mission what — about 12 years ago, 14 years ago?

Lifecycle

The software on a Mars lander is expected take 3-4 years development. It’s designed according to a Waterfall model split roughly as follows:

• 9 months requirements definition
• 12-18 months in design phase
• 12-18 months test phase
• Launch!

Software

One of the things about the software, one of the unique aspects about working software on a spacecraft is it’s the only part of the system that you can change after launch.

The operating system is VxWorks. The software on board the spacecraft is entirely in C. The software on the rover is about 650,000 lines of C.

#include void main() { printf("Hello Mars\n"); }

No GitHub. Most space technology software is considered arms technology, so it cannot be made open source.

References

• Universe Today blog
• The Software Behind the Mars Phoenix Audio interview with Peter Gluck
• Image credits NASA/JPL-Caltech

The NuGet team just released NuGet 1.8. Unfortunately there is an issue when upgrading it via the NuGet package manager. The following message appears and the installation fails.

VSIXInstaller.SignatureMismatchException: The installed version of 'NuGet Package Manager' 
is signed, but the update version has an invalid signature. Therefore, Extension Manager 
cannot install the update. 

  at VSIXInstaller.Common.VerifyMatchingExtensionSignatures(IInstalledExtension installedExtension, 
    IInstallableExtension updateExtension) 
  at VSIXInstaller.InstallProgressPage.BeginInstallVSIX(SupportedVSSKU targetAppID)

As explained in the NuGet release notes, one solution is to uninstall NuGet from the VS Extension Gallery before reinstalling. Note: If Visual Studio won’t allow you to uninstall the extension (the Uninstall button is disabled), then you likely need to restart Visual Studio using “Run as Administrator.”

Another alternative (courtesy of Jeff Handley) is to install this Visual Studio hotfix and the next time you start Visual Studio, updating via the package manager will work.

One of the projects I work on uses the validation module of the eXpressApp Framework (XAF). Since the business logic is complex, there are many validation rules defined using the [RuleFromBoolProperty].

One of the recurring problems occurs when the signature of the associated property is incorrect. Consider the following:

1
2
3
4
5
6
7
8
9
10
11

[RuleFromBoolProperty("Invoice_IsAmountGreaterThanZero", 
  DefaultContexts.Save, 
  "Invoice amount must be greater than zero.", 
  UsedProperties = "Amount")]
public bool IsAmountGreaterThanZero
{
    get
    {
        return Amount > 0;
    }
}

Notice that the rule is declared public. This causes the getter to be executed when it is not required (see the note in the documentation). However another problem is that the default behaviour for public properties of XPObjects is to persist them to the datastore which means the application will attempt to create a new column called IsAmountGreaterThanZero.

Instead, either property should be declared protected or the property should also have the [NonPersistent] and [MemberDesignTimeVisibility(false)] attributes as well.

Consequently, I wrote the following unit test which will detect any properties which have the [RuleFromBoolProperty] attribute. This is not really a unit test, rather a sort of meta-test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

[TestFixture]
public class ValidationRuleDeclarationMetaTests
{
 [Test]
  public void Test_RuleFromBoolPropertyDeclarations_ShouldBeProtectedVisibility()
  {
      var assemblies = new Assembly[] { typeof(MyObjectAssembly).Assembly };

      var invalidProperties = assemblies.SelectMany(a => a.GetTypes())
                                        .SelectMany(t => t.GetProperties(BindingFlags.Public | BindingFlags.Instance))
                                        .Where(p => p.GetCustomAttributes(typeof(RuleFromBoolPropertyAttribute), true)
                                                     .Any())
                                        .Select(p => String.Format("{0}.{1}", p.DeclaringType, p.Name))
                                        .Distinct();
  
      Assert.IsFalse(invalidProperties.Any(),
                     "There are 'public' properties with the [RuleFromBoolProperty] attribute. " +
                     "These should be 'protected' instead. " +
                     "The invalid properties are: " + String.Join(", ", invalidProperties));
  }
}  

Now the build will fail whenever a validation property signature is incorrect.