Tomorrow could be an exciting moment in the history of maths. Sir Michael Atiyah is presenting a proof of the Riemann Hypothesis at the Heidelberg Laureate Forum which will be available on their youtube channel. There is a good angle by Ken Regan on the Gödel’s Lost Letter blog.

The Riemann Hypothesis is one of the most important unsolved problems in mathematics and the subject of one of my favourite books about maths: Prime Obsession.

A few years ago in Paris, I had the pleasure of seeing Sir Michael speak. He spoke so cheerfully about life and mathematics. There is a great interview with him on the Web of Stories YouTube channel.

This is part three of a series of posts about the software and tools I find invaluable. See Part 1: essential applications and Part 2: Visual Studio tools. In this post I’m covering the online applications and iPhone apps I find indispensible.

Gmail

Does the job.

Lastpass

Everyone needs a password manager.

Pinboard (and the PinSwift iPhone app)

The best bookmark tracking tool. Simple, fast, powerful. The best mobile app for it is PinSwift.

Bear

A very pretty MarkDown editor for Mac and iPhone. I use Bear for all my notes.

Google Drive File Stream

Since I use GSuite, it makes sense to try to put everything there. Google Drive has improved enormously since they moved to a file streaming approach.

Google Tasks (and the GoTasks app)

I need simple task manager with support for multiple lists, hierarchies and available everywhere. Google Tasks is often overlooked in this regard. Check out the Google Tasks canvas. There is also a fantastic free iPhone app for it GoTasks.

Feedly

Feedly is the best RSS news feed reader, in my opinion. I keep up with a few development blogs, a few science blogs and the blogs of my friends. The mobile app is great too.

Xero

I’ve been using Xero for eight years now. Business accounting software done right.

Kindle

I like real books but don’t always lug them about with me, so often I buy both the hard copy and the kindle version (there shoudl be some sort of clever discount for this…). The Kindle app is where I spend my time when I have no connectivity (the London tube, airplanes, etc.)

Chess.com and Lichess.com

I waste hours playing online blitz chess. I’ve even written a couple of Chrome extensions: Pretty print your games and Analyse any chess.com game with lichess.

Chess is a mascohistic, character-building pastime. Progress is elusive. You play a couple of good games and you think you’re improving and then you get thrashed repeatedly by a 10 year old.

This is part two of a series of posts about the software and tools I find invaluable. See part 1 and part 3.

CodeRush

I’ve been using DevExpress CodeRush since 2005. Check out this video tutorial for a lightening tour of a lot of the features, and look at the DevExpress youtube channel for a load of other tutorials.

NCrunch

NCrunch provides continuous testing for Visual Studio. When I make any change to my code which breaks a unit test, the NCrunch risk status goes red a few seconds later, even without recompiling. I get immediate feedback for any breaking change, so long as I have a test for it. Not only does it encourages me and my team to write good tests, but it allows us to make new changes and refactor with confidence.

Redgate .NET Reflector Developer Bundle

When the going gets tough, the ANTS Performance Profiler and ANTS Memory Profiler have helped me find some of the hardest bugs I’ve come across.

Also, while there are several good .NET decompilers, many of them free, I got used to .NET Reflector which comes part of this bundle.

T4

I use code generation to automatically generate templates for unit tests for validation rules and T4 and the T4 Toolbox fit the bill nicely.

For a more advanced use case, see my post about automatically converting DevExpress v1 report scripts to check for compilation errors and allow for unit testing of report scripts.

AWS Toolkit

I always install the Amazon Web Services Toolkit to make it easy to spin up test servers in the Amazon cloud.

Next up

My essential web applications and iPhone apps.

This is the first in a series of posts where I list the applications that I use and enjoy the most. See part 2) and part 3.

A bit of preamble, my most powerful machine is a Windows-only desktop with lots of RAM and an SSD drive. I use it almost exclusively for development.

I also have a Macbook Pro which is configured to dual-boot Windows and MacOS. The Windows machine is more or less a mirror of my main development machine. The Mac is where I do all my document editing, Word, blogging, etc. I also use it for my occasional forays in to iPhone development or other non-Windows experiments.

When I upgrade my development machine, I reconfigure the previous system for Linux, currently Ubuntu.

Chocolatey

In Windows, Chocolatey is a very convenient way to install programs. Just go:

C:> choco install nodejs

Boxstarter

Even better, create a powershell script with all the applications you want to install and run it with boxstarter. Repeatable and reboot-resilient environment installations using chocolatey under the hood. For several years now, I have maintained a script for installing my entire development environment (and the build server) from scratch. Works whether it’s a physical or a virtual machine.

BeyondCompare

BeyondCompare is the best file-comparison tool I’ve found. Cross-platform too, I have it installed on my Mac as well.

Synergy

Synergy allows me to share one mouse and keyboard across all my machines. It works just like dual monitors, except when the mouse moves onto the next screen, you’ve actually changed computer. I don’t have multiple monitors any more, I just switch the input on my giant curved 34” Dell monitor via a keyboard shortcut.

7Zip

Nothing to say.

VS Code

First class support for .NET and C#, Typescript in particular, but VS Code is also extremely good with Javascript, Powershell, Python, Markdown, etc. Multiple cursor support. Cross-platform. Extensions. I’m writing this blog post in VS Code.

Visual Studio

My daily workhorse. I’ve spent more screen time here than anywhere.

Zotero

Keep track of academic papers. Download them for offline reading. Handle citations and bibliographies effortlessly from within Word. Zotero.

Next up

More on Visual Studio in the next post where I go through my essential extensions and tools.

I finally worked out why my Macbook was randomly sleeping.

TL;DR - Skip to the end.

Clues

It only seemed to happen when I was working while sitting up on the bed with the computer on my lap. It seemed to be related to the position of the computer or possibly the lid. The computer would sleep. I’d wake it and login again. Continue. Sometimes it would happen once. Sometimes three times in ten minutes. Annoying. I assumed some hardware defect.

Pajamas

Then over several days I cracked it. I realised it never happened when I was wearing pajamas. Very strange.

Reflections

I started thinking about what normally triggers sleep mode. Closing the lid. How does a Macbook know the lid is closed. There’s not really a clasp or anything so it must be a magnet. Perhaps it’s getting confused by my belt buckle or something. My phone?

No, my phone case! The flip-case of my phone has a magnet to keep the flap closed. In my jeans pocket it’s near enough to make the laptop think the lid has been closed. Duh.

TL;DR

Take your mobile phone out of your pocket!

I was inspired by two events to jump back into serverless framework.

Firstly, I attended the second London serverless meetup yesterday evening which was excellent and showed just how much enthusiasm there is for serverless architectures. Check out their new logo on the left. It was significant that each of the three speakers announced that they are actively hiring serverless developers.

Secondly, Stolz has contributed improvements to my sample project for integrating PHP into the serverless framework. It’s the purpose of this blog post to cover the changes.

The trick to getting AWS lambda to support PHP is to bundle in a PHP binary so that nodejs can call it with child_process.spawn(). In my first implementation, I used an Ubuntu docker base image to compile and produce the php binary. Unfortunately, this is not identical to the container that AWS Lambda uses and so sometimes the logs would contain errors such as:

1
2
3
4
5

START RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7 Version: $LATEST
2017-03-01 18:11:10.455 (+00:00)    728dcddf-feaa-11e6-8346-2125e1c055d7    stderr: ./php: /usr/lib64/libcurl.so.4: no version information available (required by ./php)

END RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7
REPORT RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7    Duration: 6000.08 ms    Billed Duration: 6000 ms    Memory Size: 1024 MB    Max Memory Used: 23 MB  

In my experience, these errors were often not fatal, but the correct approach is to build the php binary from a base image which is closer to the one lambda uses. So instead of my docker file starting with FROM ubuntu, it now starts with FROM amazonlinux. Also, with this image, I can use yum to install other dependencies like libpng-devel. So the new docker build script for producing the php binary looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# Compile PHP with static linked dependencies
# to create a single running binary

FROM amazonlinux

ARG PHP_VERSION

RUN yum install \
    autoconf \
    automake \
    libtool \
    bison \
    re2c \
    libxml2-devel \
    openssl-devel \
    libpng-devel \
    libjpeg-devel \
    curl-devel -y

RUN curl -sL https://github.com/php/php-src/archive/$PHP_VERSION.tar.gz | tar -zxv

WORKDIR /php-src-$PHP_VERSION

RUN ./buildconf --force

RUN ./configure \
    --enable-static=yes \
    --enable-shared=no \
    --disable-all \
    --enable-json \
    --enable-libxml \
    --enable-mbstring \
    --enable-phar \
    --enable-soap \
    --enable-xml \
    --with-curl \
    --with-gd \
    --with-zlib \
    --with-openssl \
    --without-pear

RUN make -j 5

If you run this with

$ sh dockerfile.buildphp

It will use docker to overwrite the php binary which will get shipped when you deploy with sls deploy. And this time, there are no more libcurl errors. All the code is on Github.

In the last post I built a PHP capable sample project for the Serverless Framework. In this post, I’ll show a concrete use of it.

The service I’m building connects runs a PHP function for pretty-printing chess games from the lichess online chess server. James Clarke has written a PHP function to do this using fpdf17.

The lichess exporter takes the game id of any game that has been played on the lichess server and produced a PDF output. Take for example, Game 8 of the current World Championship which is here. When I open the resulting file, I see this:

In this blog post I’ll describe how I turned this into a serverless service. The goal is to create:

• Add an endpoint which takes the game id as a parameter
• Run the PHP function via an AWS lambda function
• Return the result as a stream

Prerequisites

First check everything we need is installed.

$ serverless --version
1.2.1
$ node --version
v7.1.0

Initial setup

$ mkdir serverless-lichess-to-pdf
$ cd serverless-lichess-to-pdf
$ sls install --url https://github.com/ZeroSharp/serverless-php

Next copy in the source from https://github.com/clarkerubber/lichessPDFExporter.

You can check it works by running the following.

$ php main.php COQChpzH > COQChpzH.pdf

What’s going on here? The php binary (from the serverless-php project) is running main.php (from the lichess-pdf-exporter project) with argument COQChpzH (which corresponds to a chess game on the lichess server. The main.php function downloads the game from the lichess API and passes it through the fpdf17 library to create a pdf stream which is written out to the COQChpzH.pdf file.

Lessons learned

I learned a few things while trying to get this project working. The basic plan is to modify handler.js so that it return the output of the call described above. Turns out there are quite a few gotchas along the way.

Lesson 1 - Defining a path parameter

I want my API to look like this:

http://.../serverless-lichess-to-pdf/export/{gameid}

I could not find an example in the serverless docs for getting a parameter that is passed in the URL.

Turns out your serverless.yml file should look like this:

1
2
3
4
5
6
7

functions:
  exportToPdf:
    handler: handler.exportToPdf
    events:
      - http:
          path: export/{gameid}
          method: get

Then, in your handler.js you can retrieve the parameter with:

1
2
3
4

module.exports.exportToPdf = (event, context, callback) => {
  var gameid = event.pathParameters.gameid;
  // etc...
}

Lesson 2 - API Gateway does not support binary data

I was hoping I could just do something like this:

1
2
3
4
5
6
7
8

// this does NOT work
const response = {
    statusCode: 200,
    body: outputFromPhpCall,
    content-type: "application/pdf"
};

return callback(null, response);

At present, you cannot return a binary file. Amazon have just (November 2016) released support for binary types in API Gateway but it’s currently an open issue in the Serverless Framework.

Lesson 3 - You can redirect the response to an S3 bucket

So instead of returning the binary output, I can write the output to an S3 bucket and return a 302 redirection to the S3 resource. Like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

// body contains the output from the PHP call
const params = {
    Bucket: bucket,
    Key: key,
    ACL: 'public-read-write',
    Body: body,
    ContentType: 'application/pdf'
};

// Save the pdf file to S3    
s3.putObject(params, function(err, data) {
if (err)
{
    return callback(new Error(`Failed to put s3 object: ${err}`));
}

// respond with a 302 redirect to the PDF file
const response = {
    statusCode: 302,
    headers: {
        location : `https://s3-eu-west-1.amazonaws.com/${bucket}/${key}`
    }
};

return callback(null, response);

Lesson 4 - You can automatically delete S3 objects after a number of days

Each S3 bucket has optional lifecycle rules where you can specify that files are automatically removed after a time period. I wanted to set this up within the serverless.yml resources section, but the syntax for the lifecycle rules were not very obvious and I could not find any examples online. The following seems to work:

1
2
3
4
5
6
7
8
9
10
11

resources:
  Resources:
    PackageStorage:
      Type: AWS::S3::Bucket
      Properties:
        AccessControl: PublicRead
        BucketName: ${self:custom.exportToPdfBucket}
        LifecycleConfiguration:
          Rules:
            - ExpirationInDays: 1
              Status: Enabled

It’s all working now

You can check it out by visiting this link.

The source code is on Github.

I also wrote a Chrome extension which injects the link into the lichess page.

The goal of this post is to explain how to call a PHP function from within an AWS lambda using the Serverless Framework.

Prerequisites

First check everything we need is installed.

$ serverless --version
1.1.0
$ node --version
v7.1.0

Install the sample PHP function

Install my sample Hello function from my github repository.

$ sls install --url https://github.com/ZeroSharp/serverless-php

1
2

Serverless: Downloading and installing "serverless-php"…
Serverless: Successfully installed "serverless-php".

The code

$ cd serverless-php

Let’s have a look at the serverless.yml file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

service: serverless-php

provider:
  name: aws
  runtime: nodejs4.3
  # region: eu-west-1

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: hello
          method: get

Now look at the php function index.php that we’d like our lambda to call.

1
2
3
4
5
6

<?php

# $argv will contain the event object. You can output its contents like this if you like
#var_export($argv, true);

printf('Go Serverless v1.0! Your PHP function executed successfully!');

And the handler.js for the hello function looks as follows. It defines a simple lambda which calls the PHP binary, logs any errors and returns the result.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

'use strict';

var child_process = require('child_process');

module.exports.hello = (event, context, callback) => {

  var strToReturn = '';

  var php = './php';

  // workaround to get 'sls invoke local' to work
  if (typeof process.env.PWD !== "undefined") {
    php = 'php';
  }

  var proc = child_process.spawn(php, [ "index.php", JSON.stringify(event), { stdio: 'inherit' } ]);

  proc.stdout.on('data', function (data) {
    var dataStr = data.toString()
    // console.log('stdout: ' + dataStr);
    strToReturn += dataStr
  });

  // this ensures any error messages raised by the PHP function end up in the logs
  proc.stderr.on('data', function (data) {
    console.log(`stderr: ${data}`);
  });

  proc.on('close', function(code) {
    if(code !== 0) {
      return callback(new Error(`Process exited with non-zero status code ${code}`));
    }

    const response = {
      statusCode: 200,
      body: JSON.stringify({
        message: strToReturn,
        //input: event,
      }),
    };

    callback(null, response);
  });
};

Included is the PHP binary to bundle with our serverless function.

(You may need to compile it yourself with different options. See below for help on how to do this.)

Check it works from your shell.

$ php index.php

1

Go Serverless v1.0! Your PHP function executed successfully!

Run it locally through the Serverless Framework.

$ sls invoke local --function hello

1
2
3
4
5
6

Serverless: Your function ran successfully.

{
    "statusCode": 200,
    "body": "{\"message\":\"Go Serverless v1.0! Your PHP function executed successfully!\"}"
}

Looks good. Let’s deploy.

$ sls deploy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

Serverless: Packaging service…
Serverless: Uploading CloudFormation file to S3…
Serverless: Uploading service .zip file to S3…
Serverless: Updating Stack…
Serverless: Checking Stack update progress…
..........
Serverless: Stack update finished…

Service Information
service: serverless-php
stage: dev
region: eu-west-1
api keys:
  None
endpoints:
  GET - https://c1w0hct166.execute-api.eu-west-1.amazonaws.com/dev/hello
functions:
  serverless-php-dev-hello: arn:aws:lambda:eu-west-1:962613113552:function:serverless-php-dev-hello

Run the remote function via Serverless.

$ sls invoke --function hello

1
2
3
4

{
    "statusCode": 200,
    "body": "{\"message\":\"Go Serverless v1.0! Your PHP function executed successfully!\",\"input\":{}}"
}

Visit the endpoint in your browser.

1
2
3

{
    "message": "Go Serverless v1.0! Your PHP function executed successfully!"
}

Nice. It’s all working.

Rebuilding the PHP binary

Depending on the PHP function you need to run, it may be necessary to rebuild the php binary with different flags and dependencies. You can do this best with docker.

$ docker --version
Docker version 1.12.3, build 6b644ec

Modify dockerfile.buildphp as necessary.

Then run:

$ sh buildphp.sh

This will build a new PHP binary and copy it to the project root. You can immediately deploy for testing with:

$ sls deploy

Thanks

Shout out to Danny Linden whose code got me started on this.

When an XAF list view has no selection-based actions available, the selection box still appears in the grid. Users get confused. In this post, we’ll look at a workaround.

The problem

In the XAF MainDemo, lets make Departments read-only for the User role.

1
2
3

userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Create, SecurityPermissionState.Deny);
userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Write, SecurityPermissionState.Deny);
userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Delete, SecurityPermissionState.Deny);

Then start the web application, login as John and navigate to the Departments list view. There is a column selection box, but it serves no purpose. There are no actions that depend on a grid selection.

Without the SelectionColumnVisibilityController

The fix

Here is a controller which calculates whether there are any available actions which require one or more rows to be selected. If there are none, the selection box will not appear.

Add the following controller to the MainDemo.Module.Web project. It hides the selection box if there are no actions which depend on a grid selection.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

using System;
using DevExpress.ExpressApp;
using DevExpress.ExpressApp.Actions;
using DevExpress.ExpressApp.Editors;
using DevExpress.ExpressApp.SystemModule;
using DevExpress.Web;
using System.Linq;

namespace MainDemo.Module.Web.Controllers
{
    public class SelectionColumnVisibilityController : ViewController
    {
        public SelectionColumnVisibilityController()
        {
            TargetViewType = ViewType.ListView;
        }

        private bool IsSelectionColumnVisible()
        {
            bool isSelectionColumnRequired = false;
            // remove checkbox if there are no available actions
            foreach (Controller controller in Frame.Controllers)
            {
                if (!controller.Active)
                    continue;

                if (controller.Actions.Count == 0)
                    continue;

                bool allowEdit = true;
                if ((Frame is NestedFrame) && (((NestedFrame)Frame).ViewItem is PropertyEditor))
                    allowEdit = (bool)((PropertyEditor)((NestedFrame)Frame).ViewItem).AllowEdit;

                foreach (ActionBase action in controller.Actions)
                {
                    if (action.SelectionDependencyType == SelectionDependencyType.RequireMultipleObjects)
                    {
                        if (action.Active || IsActionInactiveBySelectionContext(action))
                        {
                            if (action.Enabled || IsActionDisabledBySelectionContext(action))
                            {
                                isSelectionColumnRequired = true;
                                break;
                            }
                        }
                    }
                }
                if (isSelectionColumnRequired)
                    break;
            }
            return isSelectionColumnRequired;
        }

        private bool IsActionInactiveBySelectionContext(ActionBase action)
        {
            if (action.Active)
                return true;
            else
            {
                foreach (string item in action.Active.GetKeys())
                {
                    if (item == ActionBase.RequireMultipleObjectsContext || item == ActionBase.RequireSingleObjectContext)
                        continue;
                    if (!action.Active[item])
                        return false;
                }
                return true;
            }
        }

        private bool IsActionDisabledBySelectionContext(ActionBase action)
        {
            if (action.Enabled)
                return true;
            else
            {
                foreach (string item in action.Enabled.GetKeys())
                {
                    if (item == ActionBase.RequireMultipleObjectsContext ||
                        item == ActionBase.RequireSingleObjectContext ||
                        item == ActionsCriteriaViewController.EnabledByCriteriaKey)
                        continue;
                    if (!action.Enabled[item])
                        return false;
                }
                return true;
            }
        }

        protected override void OnViewControlsCreated()
        {
            base.OnViewControlsCreated();
            ASPxGridView grid = ((ListView)this.View).Editor.Control as ASPxGridView;
            if (grid != null)
            {
                grid.Load += grid_Load;
                grid.DataBound += grid_DataBound;
            }
        }

        protected override void OnDeactivated()
        {
            base.OnDeactivated();
            ASPxGridView grid = ((ListView)this.View).Editor.Control as ASPxGridView;
            if (grid != null)
            {
                grid.DataBound -= grid_DataBound;
                grid.Load -= grid_Load;
            }
        }

        void grid_Load(object sender, EventArgs e)
        {
            SetSelectionColumnVisibility(sender, e);
        }

        void grid_DataBound(object sender, EventArgs e)
        {
            SetSelectionColumnVisibility(sender, e);
        }

        private void SetSelectionColumnVisibility(object sender, EventArgs e)
        {
            bool isSelectionColumnVisible = IsSelectionColumnVisible();
            if (!isSelectionColumnVisible)
            {
                var grid = (ASPxGridView)sender;
                var selectionBoxColumn =
                    grid.Columns
                        .OfType<GridViewCommandColumn>()
                        .Where(x => x.ShowSelectCheckbox)
                        .FirstOrDefault();

                if (selectionBoxColumn != null)
                {
                    selectionBoxColumn.Visible = false;
                }
            }
        }
    }
}

Run the application again and see the difference. Now the grid looks like this. Notice, there is no longer a selection box on the row.

By the way, this is how it looks with old-style XAF web apps.

Without the SelectionColumnVisibilityController

With the SelectionColumnVisibilityController

Sometimes the deadline has arrived and you still have some failing tests. After a discussion with the dev team, you decide to deploy anyway and fix the bugs for the next release. You need to get the build server to ignore the tests.

One way is just to mark the test with the [Ignore] attribute.

1
2
3
4
5
6

[Test]
[Ignore] // TODO: Fix this test before the next release!
public void Test()
{
    // Some failing test code...
}

After the weekend, everyone forgets about the ignored tests and they never get fixed.

Instead, I like to do this.

1
2
3
4
5
6
7

[Test]
public void Test()
{
    if (DateTime.Now < new DateTime(2016, 10, 17))
        Assert.Ignore("Temporarily ignored until October 17.");
    // Some failing test code...
}

This is a fairly rare occurrence for my team, so the above approach is sufficient and works with all test frameworks. But if you want to go further Richard Slater shows how to create an NUnit attribute.