Sunday, 22 March 2020

Dual boot authentication with ASP.Net Core

This is a post about having two kinds of authentication working at the same time in ASP.Net Core. But choosing which authentication method to use dynamically at runtime; based upon the criteria of your choice.

Already this sounds complicated; let's fix that. Perhaps I should describe my situation to you. I've an app which has two classes of user. One class, let's call them "customers" (because... uh... they're customers). The customers access our application via a public facing website. Traffic rolls through Cloudflare and into our application. The public facing URL is something fancy like https://mega-app.com. That's one class of user.

The other class of user we'll call "our peeps"; because they are us. We use the app that we build. Traffic from "us" comes from a different hostname; only addressable on our network. So URLs from requests that we make are more along the lines of https://strictly4mypeeps.io.

So far, so uncontroversial. Now it starts to get interesting. Our customers log into our application using their super secret credentials. It's cookie based authentication. But for our peeps we do something different. Having to enter your credentials each time you use the app is friction. It gets in the way. So for us we have Azure AD in the mix. Azure AD is how we authenticate ourselves; and that means we don't spend 5% of each working day entering credentials.

Let us speak of the past

Now our delightful little application grew up in a simpler time. A time where you went to the marketplace, picked out some healthy looking servers, installed software upon them, got them attached to the internet, deployed an app onto them and said "hey presto, we're live!".

Way back when, we had some servers on the internet, that's how our customers got to our app. Our peeps, us, we went to other servers that lived on our network. So we had multiple instances of our app, deployed to different machines. The ones on the internet were configured to use cookie based auth, the ones on our internal network were Azure AD.

As I said, a simpler time.

A new hope

We've been going through the process of cloudifying our app. Bye, bye servers, hello Docker and Kubernetes. So exciting! As we change the way our app is built and deployed; we've been thinking about whether the choices we make still make sense.

When it came to authentication, my initial thoughts were to continue the same road we're travelling; just in containers and pods. So where we had "internal" servers, we'd have "internal" pods, and where we'd have "external" servers we'd have external pods. I had the good fortune to be working with the amazingly talented Robski. Robski knows far more about K8s and networking than I'm ever likely to. He'd regularly say things like "ingress" and "MTLS" whilst I stared blankly at him. He definitely knows stuff.

Robski challenged my plans. "We don't need it. Have one pod that does both sorts of auth. If you do that, your implementation is simpler and scaling is more straightforward. You'll only need half the pods because you won't need internal and external ones; one pod can handle both sets of traffic. You'll save money."

I loved the idea but I didn't think that ASP.Net Core supported it. "It's just not a thing Robski; ASP.Net Core doesn't suppport it." Robski didn't believe me. That turned out to a very good thing. There followed a period of much googling and experimentation. One day of hunting in, I was still convinced there was no way to do it that would allow me to look in the mirror without self loathing. Then Robski sent me this:

It was a link to the amazing David Fowler talking about some API I'd never heard of called SchemeSelector. It turned out that this was the starting point for exactly what we needed; a way to dynamically select an authentication scheme at runtime.

Show me the code

This API did end up landing in ASP.Net Core, but with the name ForwardDefaultSelector. Not the most descriptive of names and I've struggled to find any documentation on it at all. What I did discover was an answer on StackOverflow by the marvellous Barbara Post. I was able to take the approach Barbara laid out and use it to my own ends. I ended up with this snippet of code added to my Startup.ConfigureServices:

services
    .AddAuthentication(sharedOptions => {
        sharedOptions.DefaultScheme = "WhichAuthDoWeUse";
        sharedOptions.DefaultAuthenticateScheme = "WhichAuthDoWeUse";
        sharedOptions.DefaultChallengeScheme = "WhichAuthDoWeUse";
    })
    .AddPolicyScheme("WhichAuthDoWeUse", "Azure AD or Cookies", options => {
        options.ForwardDefaultSelector = context => {
            var (isExternalRequest, requestUrl) = context.Request.GetIsExternalRequestAndDomain();
            if (isExternalRequest) {
                _logger.LogInformation(
                    "Request ({RequestURL}) has come from external domain ({Domain}) so using Cookie Authentication",
                    requestUrl, ExternalBaseUrl);

                return CookieAuthenticationDefaults.AuthenticationScheme;
           }

           _logger.LogInformation(
               "Request ({RequestURL}) has not come from external domain ({Domain}) so using Azure AD Authentication",
               requestUrl, ExternalBaseUrl);

            return AzureADDefaults.AuthenticationScheme;
        };
    })
    .AddAzureAD(options => {
        Configuration.Bind("AzureAd", options);
    })
    .AddCookie(options => {
        options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
        options.Cookie.SameSite = SameSiteMode.Strict;
        options.Cookie.HttpOnly = true;
        options.Events.OnRedirectToAccessDenied = (context) => {
            context.Response.StatusCode = Microsoft.AspNetCore.Http.StatusCodes.Status401Unauthorized;
            return Task.CompletedTask;
        };

        options.Events.OnRedirectToLogin = (context) => {
            context.Response.StatusCode = Microsoft.AspNetCore.Http.StatusCodes.Status401Unauthorized;
            return Task.CompletedTask;
        };
    });

If you look at this code it's doing these things:

  1. Registering three types of authentication: Cookie, Azure AD and "WhichAuthDoWeUse"
  2. Registers the default Scheme to be "WhichAuthDoWeUse".

"WhichAuthDoWeUse" is effectively an if statement that says, "if this is an external Request use Cookies authentication, otherwise use Azure AD". Given that "WhichAuthDoWeUse" is the default scheme, this code runs for each request, to determine which authentication method to use.

Alongside this mechanism I added these extension methods:

using System;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Extensions;

namespace My.App.Auth {
    public static class AuthExtensions {
        public const string ExternalBaseUrl = "https://mega-app.com";
        public const string InternalBaseUrl = "https://strictly4mypeeps.io";

        /// <summary>
        /// Determines if a request is an "external" URL (eg begins "https://mega-app.com")
        /// or an "internal" URL (eg begins "https://strictly4mypeeps.io")
        /// </summary>
        public static (bool, string) GetIsExternalRequestAndDomain(this HttpRequest request) {
            var (requestUrl, domain) = GetRequestUrlAndDomain(request);

            var isExternalUrl = domain == ExternalBaseUrl;

            var isUnknownPath = domain == null; // This scenario is extremely unlikely but has been observed once during testing so we will cater for it

            var isExternalRequest = isExternalUrl || isUnknownPath; // If unknown we'll treat as "external" for a safe fallback

            return (isExternalRequest, requestUrl);
        }

        /// <summary>
        /// Determines if a request is an "external" URL (eg begins "https://mega-app.com")
        /// or an "internal" URL (eg begins "https://strictly4mypeeps.io")
        /// </summary>
        public static (bool, string) GetIsInternalRequestAndDomain(this HttpRequest request) {
            var (requestUrl, domain) = GetRequestUrlAndDomain(request);

            var isInternalRequest = domain == InternalBaseUrl;

            return (isInternalRequest, requestUrl);
        }

        private static (string, string) GetRequestUrlAndDomain(HttpRequest request) {
            string requestUrl = null;
            string domain = null;
            if (request.Host.HasValue) {
                requestUrl = request.GetEncodedUrl();
                domain = new Uri(requestUrl).GetLeftPart(UriPartial.Authority);
            }

            return (requestUrl, domain);
        }
    }
}

Finally, I updated the SpaController.cs (which serves initial requests to our Single Page Application) to cater for having two types of Auth in play:

        /// <summary>
        /// ASP.NET will try and load the index.html using the FileServer if we don't have a route
        /// here to match `/`. These attributes can't be on Index or the spa fallback doesn't work
        /// Note: this is almost perfect except that if someone actually calls /index.html they'll get
        /// the FileServer one, not the one from this file.
        /// </summary>
        [HttpGet("/")]
        [AllowAnonymous]
        public async Task<IActionResult> SpaFallback([FromQuery] string returnUrl) {
            var redirectUrlIfUserIsInternalAndNotAuthenticated = GetRedirectUrlIfUserIsInternalAndNotAuthenticated(returnUrl);

            if (redirectUrlIfUserIsInternalAndNotAuthenticated != null)
                return LocalRedirect(redirectUrlIfUserIsInternalAndNotAuthenticated);

            return await Index(); // Index just serves up our SPA index.html
        }

        /// <summary>
        /// SPA landing with authorisation - this endpoint will typically not be directly navigated to by a user; 
        /// rather it will be redirected to from the IndexWithoutAuthorisation and SpaFallback actions above
        /// in the case where a user is *not* authenticated but has come from an internal URL eg https://strictlyformypeeps.io
        /// </summary>
        [HttpGet("/login-with-azure-ad")]
        [Authorize]
        public async Task<IActionResult> IndexWithAuthorisation()
        {
            return await Index(); // Index just serves up our SPA index.html
        }

        /// <summary>
        /// This method returns a RedirectURL if a request is coming from an internal URL
        /// eg https://ix-web-int.prd.investec.cloud and is not authenticated.  In this case
        /// we likely want to trigger authentication by redirecting to an authorized endpoint
        /// </summary>
        string GetRedirectUrlIfUserIsInternalAndNotAuthenticated(string returnUrl)
        {
            // If a user is authenticated then we don't need to trigger authentication
            var isAuthenticated = User?.Identity?.Name != null;
            if (isAuthenticated)
                return null;

            // This scenario is extremely unlikely but has been observed once during testing so we will cater for it
            var (isInternalRequest, requestUrl) = Request.GetIsInternalRequestAndDomain();

            if (isInternalRequest) {
                var redirectUrl = $"/login-with-azure-ad{(string.IsNullOrEmpty(returnUrl) ? "" : "?returnUrl=" + WebUtility.UrlEncode(returnUrl))}";
                _logger.LogInformation(
                    "Request ({RequestURL}) has come from internal domain ({InternalDomain}) but is not authenticated; redirecting to {RedirectURL}",
                    requestUrl, AuthExtensions.InternalBaseUrl, redirectUrl);

                return redirectUrl;
            }

            return null;
        }

The code above allows anonymous requests to land in our app through the AllowAnonymous attribute. However, it checks the request when it comes in to see if:

  1. It's an internal request (i.e. the Request URL starts "https://strictly4mypeeps.io/")
  2. The current user is not authenticated.

In this case the user is redirected to the https://strictly4mypeeps.io/login-with-azure-ad route which is decorated with the Authorize attribute. This will trigger authentication for our unauthenticated internal users and drive them through the Azure AD login process.

The mystery of no documentation

I'm so surprised that this approach hasn't yet been better documented on the (generally superb) ASP.Net Core docs. It's such a potentially useful approach; and in our case, money saving too! I hope the official docs feature something on this in future. If they do, and I've just missed it (possible!) then please hit me up in the comments.

Friday, 21 February 2020

Web Workers, comlink, TypeScript and React

JavaScript is famously single threaded. However, if you're developing for the web, you may well know that this is not quite accurate. There are Web Workers:

A worker is an object created using a constructor (e.g. Worker()) that runs a named JavaScript file — this file contains the code that will run in the worker thread; workers run in another global context that is different from the current window.

Given that there is a way to use other threads for background processing, why doesn't this happen all the time? Well there's a number of reasons; not the least of which is the ceremony involved in interacting with Web Workers. Consider the following example that illustrates moving a calculation into a worker:

// main.js
function add2NumbersUsingWebWorker() {
    const myWorker = new Worker("worker.js");

    myWorker.postMessage([42, 7]);
    console.log('Message posted to worker');

    myWorker.onmessage = function(e) {
        console.log('Message received from worker', e.data);
    }
}

add2NumbersUsingWebWorker();

// worker.js
onmessage = function(e) {
  console.log('Worker: Message received from main script');
  const result = e.data[0] * e.data[1];
  if (isNaN(result)) {
    postMessage('Please write two numbers');
  } else {
    const workerResult = 'Result: ' + result;
    console.log('Worker: Posting message back to main script');
    postMessage(workerResult);
  }
}

This is not simple. It's hard to understand what's happening. Also, this approach only supports a single method call. I'd much rather write something that looked more like this:

// main.js
function add2NumbersUsingWebWorker() {
    const myWorker = new Worker("worker.js");

    const total = myWorker.add2Numbers([42, 7]);
    console.log('Message received from worker', total);
}

add2NumbersUsingWebWorker();

// worker.js
export function add2Numbers(firstNumber, secondNumber) {
  const result = firstNumber + secondNumber;
  return (isNaN(result))
    ? 'Please write two numbers'
    : 'Result: ' + result;
}

There's a way to do this using a library made by Google called comlink. This post will demonstrate how we can use this. We'll use TypeScript and webpack. We'll also examine how to integrate this approach into a React app.

A use case for a Web Worker

Let's make ourselves a TypeScript web app. We're going to use create-react-app for this:

npx create-react-app webworkers-comlink-typescript-react --template typescript

Create a takeALongTimeToDoSomething.ts file alongside index.tsx:

export function takeALongTimeToDoSomething() {
    console.log('Start our long running job...');
    const seconds = 5;
    const start = new Date().getTime();
    const delay = seconds * 1000;

    while (true) {
        if ((new Date().getTime() - start) > delay) {
            break;
        }
    }
    console.log('Finished our long running job');
}

To index.tsx add this code:

import { takeALongTimeToDoSomething } from './takeALongTimeToDoSomething';

// ...

console.log('Do something');
takeALongTimeToDoSomething();
console.log('Do another thing');

When our application runs we see this behaviour:

The app starts and logs Do something and Start our long running job... to the console. It then blocks the UI until the takeALongTimeToDoSomething function has completed running. During this time the screen is empty and unresponsive. This is a poor user experience.

Hello worker-plugin and comlink

To start using comlink we're going to need to eject our create-react-app application. The way create-react-app works is by giving you a setup that handles a high percentage of the needs for a typical web app. When you encounter an unsupported use case, you can run the yarn eject command to get direct access to the configuration of your setup.

Web Workers are not that commonly used in day to day development at present. Consequently there isn't yet a "plug'n'play" solution for workers supported by create-react-app. There's a number of potential ways to support this use case and you can track the various discussions happening against create-react-app that covers this. For now, let's eject with:

yarn eject

Then let's install the packages we're going to be using:

  • worker-plugin - this webpack plugin automatically compiles modules loaded in Web Workers
  • comlink - this library provides the RPC-like experience that we want from our workers
yarn add comlink worker-plugin

We now need to tweak our webpack.config.js to use the worker-plugin:

const WorkerPlugin = require('worker-plugin');

// ....

    plugins: [
      new WorkerPlugin(),

// ....

Do note that there's a number of plugins statements in the webpack.config.js. You want the top level one; look out for the new HtmlWebpackPlugin statement and place your new WorkerPlugin(), before that.

Workerize our slow process

Now we're ready to take our long running process and move it into a worker. Inside the src folder, create a new folder called my-first-worker. Our worker is going to live in here. Into this folder we're going to add a tsconfig.json file:

{
  "compilerOptions": {
    "strict": true,
    "target": "esnext",
    "module": "esnext",
    "lib": [
      "webworker",
      "esnext"
    ],
    "moduleResolution": "node",
    "noUnusedLocals": true,
    "sourceMap": true,
    "allowJs": false,
    "baseUrl": "."
  }
}

This file exists to tell TypeScript that this is a Web Worker. Do note the "lib": [ "webworker" usage which does exactly that.

Alongside the tsconfig.json file, let's create an index.ts file. This will be our worker:

import { expose } from 'comlink';
import { takeALongTimeToDoSomething } from '../takeALongTimeToDoSomething';

const exports = {
    takeALongTimeToDoSomething
};
export type MyFirstWorker = typeof exports;

expose(exports);

There's a number of things happening in our small worker file. Let's go through this statement by statement:

import { expose } from 'comlink';

Here we're importing the expose method from comlink. Comlink’s goal is to make exposed values from one thread available in the other. The expose method can be viewed as the comlink equivalent of export. It is used to export the RPC style signature of our worker. We'll see it's use later.

import { takeALongTimeToDoSomething } from '../takeALongTimeToDoSomething';

Here we're going to import our takeALongTimeToDoSomething function that we wrote previously, so we can use it in our worker.

const exports = {
    takeALongTimeToDoSomething
};

Here we're creating the public facing API that we're going to expose.

export type MyFirstWorker = typeof exports;

We're going to want our worker to be strongly typed. This line creates a type called MyFirstWorker which is derived from our exports object literal.

expose(exports);

Finally we expose the exports using comlink. We're done; that's our worker finished. Now let's consume it. Let's change our index.tsx file to use it. Replace our import of takeALongTimeToDoSomething:

import { takeALongTimeToDoSomething } from './takeALongTimeToDoSomething';

With an import of wrap from comlink that creates a local takeALongTimeToDoSomething function that wraps interacting with our worker:

import { wrap } from 'comlink';

function takeALongTimeToDoSomething() {
    const worker = new Worker('./my-first-worker', { name: 'my-first-worker', type: 'module' });
    const workerApi = wrap<import('./my-first-worker').MyFirstWorker>(worker);
    workerApi.takeALongTimeToDoSomething();    
}

Now we're ready to demo our application using our function offloaded into a Web Worker. It now behaves like this:

There's a number of exciting things to note here:

  1. The application is now non-blocking. Our long running function is now not preventing the UI from updating
  2. The functionality is lazily loaded via a my-first-worker.chunk.worker.js that has been created by the worker-plugin and comlink.

Using Web Workers in React

The example we've showed so far demostrates how you could use Web Workers and why you might want to. However, it's a far cry from a real world use case. Let's take the next step and plug our Web Worker usage into our React application. What would that look like? Let's find out.

We'll return index.tsx back to it's initial state. Then we'll make a simple adder function that takes some values and returns their total. To our takeALongTimeToDoSomething.ts module let's add:

export function takeALongTimeToAddTwoNumbers(number1: number, number2: number) {
    console.log('Start to add...');
    const seconds = 5;
    const start = new Date().getTime();
    const delay = seconds * 1000;
    while (true) {
        if ((new Date().getTime() - start) > delay) {
            break;
        }
    }
    const total = number1 + number2;
    console.log('Finished adding');
    return total;
}

Let's start using our long running calculator in a React component. We'll update our App.tsx to use this function and create a simple adder component:

import React, { useState } from "react";
import "./App.css";
import { takeALongTimeToAddTwoNumbers } from "./takeALongTimeToDoSomething";

const App: React.FC = () => {
  const [number1, setNumber1] = useState(1);
  const [number2, setNumber2] = useState(2);

  const total = takeALongTimeToAddTwoNumbers(number1, number2);

  return (
    <div className="App">
      <h1>Web Workers in action!</h1>

      <div>
        <label>Number to add: </label>
        <input
          type="number"
          onChange={e => setNumber1(parseInt(e.target.value))}
          value={number1}
        />
      </div>
      <div>
        <label>Number to add: </label>
        <input
          type="number"
          onChange={e => setNumber2(parseInt(e.target.value))}
          value={number2}
        />
      </div>
      <h2>Total: {total}</h2>
    </div>
  );
};

export default App;

When you try it out you'll notice that entering a single digit locks the UI for 5 seconds whilst it adds the numbers. From the moment the cursor stops blinking to the moment the screen updates the UI is non-responsive:

So far, so classic. Let's Web Workerify this!

We'll update our my-first-worker/index.ts to import this new function:

import { expose } from "comlink";
import {
  takeALongTimeToDoSomething,
  takeALongTimeToAddTwoNumbers
} from "../takeALongTimeToDoSomething";

const exports = {
  takeALongTimeToDoSomething,
  takeALongTimeToAddTwoNumbers
};
export type MyFirstWorker = typeof exports;

expose(exports);

Alongside our App.tsx file let's create an App.hooks.ts file.

import { wrap, releaseProxy } from "comlink";
import { useEffect, useState, useMemo } from "react";

/**
 * Our hook that performs the calculation on the worker
 */
export function useTakeALongTimeToAddTwoNumbers(
  number1: number,
  number2: number
) {
  // We'll want to expose a wrapping object so we know when a calculation is in progress
  const [data, setData] = useState({
    isCalculating: false,
    total: undefined as number | undefined
  });

  // acquire our worker
  const { workerApi } = useWorker();

  useEffect(() => {
    // We're starting the calculation here
    setData({ isCalculating: true, total: undefined });

    workerApi
      .takeALongTimeToAddTwoNumbers(number1, number2)
      .then(total => setData({ isCalculating: false, total })); // We receive the result here
  }, [workerApi, setData, number1, number2]);

  return data;
}

function useWorker() {
  // memoise a worker so it can be reused; create one worker up front
  // and then reuse it subsequently; no creating new workers each time
  const workerApiAndCleanup = useMemo(() => makeWorkerApiAndCleanup(), []);

  useEffect(() => {
    const { cleanup } = workerApiAndCleanup;

    // cleanup our worker when we're done with it
    return () => {
      cleanup();
    };
  }, [workerApiAndCleanup]);

  return workerApiAndCleanup;
}

/**
 * Creates a worker, a cleanup function and returns it
 */
function makeWorkerApiAndCleanup() {
  // Here we create our worker and wrap it with comlink so we can interact with it
  const worker = new Worker("./my-first-worker", {
    name: "my-first-worker",
    type: "module"
  });
  const workerApi = wrap<import("./my-first-worker").MyFirstWorker>(worker);

  // A cleanup function that releases the comlink proxy and terminates the worker
  const cleanup = () => {
    workerApi[releaseProxy]();
    worker.terminate();
  };

  const workerApiAndCleanup = { workerApi, cleanup };

  return workerApiAndCleanup;
}

The useWorker and makeWorkerApiAndCleanup functions make up the basis of a shareable worker hooks approach. It would take very little work to paramaterise them so this could be used elsewhere. That's outside the scope of this post but would be extremely straightforward to accomplish.

Time to test! We'll change our App.tsx to use the new useTakeALongTimeToAddTwoNumbers hook:

import React, { useState } from "react";
import "./App.css";
import { useTakeALongTimeToAddTwoNumbers } from "./App.hooks";

const App: React.FC = () => {
  const [number1, setNumber1] = useState(1);
  const [number2, setNumber2] = useState(2);

  const total = useTakeALongTimeToAddTwoNumbers(number1, number2);

  return (
    <div className="App">
      <h1>Web Workers in action!</h1>

      <div>
        <label>Number to add: </label>
        <input
          type="number"
          onChange={e => setNumber1(parseInt(e.target.value))}
          value={number1}
        />
      </div>
      <div>
        <label>Number to add: </label>
        <input
          type="number"
          onChange={e => setNumber2(parseInt(e.target.value))}
          value={number2}
        />
      </div>
      <h2>
        Total:{" "}
        {total.isCalculating ? (
          <em>Calculating...</em>
        ) : (
          <strong>{total.total}</strong>
        )}
      </h2>
    </div>
  );
};

export default App;

Now our calculation takes place off the main thread and the UI is no longer blocked!

This post was originally published on LogRocket.

The source code for this project can be found here.

Friday, 31 January 2020

From create-react-app to PWA

Progressive Web Apps are a (terribly named) wonderful idea. You can build an app once using web technologies which serves all devices and form factors. It can be accessible over the web, but also surface on the home screen of your Android / iOS device. That app can work offline, have a splash screen when it launches and have notifications too.

PWAs can be a money saver for your business. The alternative, should you want an app experience for your users, is building the same application using three different technologies (one for web, one for Android and one for iOS). When you take this path it's hard to avoid a multiplication of cost and complexity. It often leads to dividing up the team as each works on a different stack. It's common to lose a certain amount of focus as a consequence. PWAs can help here; they are a compelling alternative, not just from a developers standpoint, but from a resourcing one too.

However, the downside of PWAs is that they are more complicated than normal web apps. Writing one from scratch is just less straightforward than a classic web app. There are easy onramps to building a PWA that help you fall into the pit of success. This post will highlight one of these. How you can travel from zero to a PWA of your very own using React and TypeScript.

This post presumes knowledge of:

  • React
  • TypeScript
  • Node

From console to web app

To create our PWA we're going to use create-react-app. This excellent project has long had inbuilt support for making PWAs. In recent months that support has matured to a very satisfactory level. To create ourselves a TypeScript React app using create-react-app enter this npx command at the console:

npx create-react-app pwa-react-typescript --template typescript

This builds you a react web app built with TypeScript; it can be tested locally with:

cd pwa-react-typescript
yarn start

From web app to PWA

From web app to PWA is incredibly simple; it’s just a question of opting in to offline behaviour. If you open up the index.tsx file in your newly created project you'll find this code:

// If you want your app to work offline and load faster, you can change
// unregister() to register() below. Note this comes with some pitfalls.
// Learn more about service workers: https://bit.ly/CRA-PWA
serviceWorker.unregister();

As the hint suggests, swap serviceWorker.unregister() for serviceWorker.register() and you now have a PWA. Amazing! What does this mean? Well to quote the docs:

  • All static site assets are cached so that your page loads fast on subsequent visits, regardless of network connectivity (such as 2G or 3G). Updates are downloaded in the background.
  • Your app will work regardless of network state, even if offline. This means your users will be able to use your app at 10,000 feet and on the subway.

... it will take care of generating a service worker file that will automatically precache all of your local assets and keep them up to date as you deploy updates. The service worker will use a cache-first strategy for handling all requests for local assets, including navigation requests for your HTML, ensuring that your web app is consistently fast, even on a slow or unreliable network.

Under the bonnet, create-react-app is achieving this through the use of technology called "Workbox". Workbox describes itself as:

a set of libraries and Node modules that make it easy to cache assets and take full advantage of features used to build Progressive Web Apps.

The good folks of Google are aware that writing your own PWA can be tricky. There's much new behaviour to configure and be aware of; it's easy to make mistakes. Workbox is there to help ease the way forward by implementing default strategies for caching / offline behaviour which can be controlled through configuration.

A downside of the usage of Workbox in create-react-app is that (as with most things create-react-app) there's little scope for configuration of your own if the defaults don't serve your purpose. This may change in the future, indeed there's an open PR that adds this support.

Icons and splash screens and A2HS, oh my!

But it's not just an offline experience that makes this a PWA. Other important factors are:

  • That the app can be added to your home screen (A2HS AKA "installed").
  • That the app has a name and an icon which can be customised.
  • That there's a splash screen displayed to the user as the app starts up.

All of the above is "in the box" with create-react-app. Let's start customizing these.

First of all, we'll give our app a name. Fire up index.html and replace <title>React App</title> with <title>My PWA</title>. (Feel free to concoct a more imaginative name than the one I've suggested.) Next open up manifest.json and replace:

  "short_name": "React App",
  "name": "Create React App Sample",

with:

  "short_name": "My PWA",
  "name": "My PWA",

Your app now has a name. The question you might be asking is: what is this manifest.json file? Well to quote the good folks of Google:

The web app manifest is a simple JSON file that tells the browser about your web application and how it should behave when 'installed' on the user's mobile device or desktop. Having a manifest is required by Chrome to show the Add to Home Screen prompt.

A typical manifest file includes information about the app name, icons it should use, the start_url it should start at when launched, and more.

So the manifest.json is essentially metadata about your app. Here's what it should look like right now:

{
  "short_name": "My PWA",
  "name": "My PWA",
  "icons": [
    {
      "src": "favicon.ico",
      "sizes": "64x64 32x32 24x24 16x16",
      "type": "image/x-icon"
    },
    {
      "src": "logo192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "logo512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "start_url": ".",
  "display": "standalone",
  "theme_color": "#000000",
  "background_color": "#ffffff"
}

You can use the above properties (and others not yet configured) to control how your app behaves. For instance, if you want to replace icons your app uses then it's a simple matter of:

  • placing new logo files in the public folder
  • updating references to them in the manifest.json
  • finally, for older Apple devices, updating the <link rel="apple-touch-icon" ... /> in the index.html.

Where are we?

So far, we have a basic PWA in place. It's installable. You can run it locally and develop it with yarn start. You can build it for deployment with yarn build.

What this isn't, is recognisably a web app. In the sense that it doesn't have support for different pages / URLs. We're typically going to want to break up our application this way. Let's do that now. We're going to use react-router; the de facto routing solution for React. To add it to our project (and the required type definitions for TypeScript) we use:

yarn add react-router-dom @types/react-router-dom

Now let's split up our app into a couple of pages. We'll replace the existing App.tsx with this:

import React from "react";
import { BrowserRouter as Router, Switch, Route, Link } from "react-router-dom";
import About from "./About";
import Home from "./Home";

const App: React.FC = () => (
  <Router>
    <nav>
      <ul>
        <li>
          <Link to="/">Home</Link>
        </li>
        <li>
          <Link to="/about">About</Link>
        </li>
      </ul>
    </nav>
    <Switch>
      <Route path="/about">
        <About />
      </Route>
      <Route path="/">
        <Home />
      </Route>
    </Switch>
  </Router>
);

export default App;

This will be our root page. It has the responsiblity of using react-router to render the pages we want to serve, and also to provide the links that allow users to navigate to those pages. In making our changes we'll have broken our test (which checked for a link we've now deleted), so we'll fix it like so:

Replace the App.test.tsx with this:

import React from 'react';
import { render } from '@testing-library/react';
import App from './App';

test('renders about link', () => {
  const { getByText } = render(<App />);
  const linkElement = getByText(/about/i);
  expect(linkElement).toBeInTheDocument();
});

You'll have noticed that in our new App.tsx we import two new components (or pages); About and Home. Let's create those. First About.tsx:

import React from "react";

const About: React.FC = () => (
  <h1>This is a PWA</h1>
);

export default About;

Then Home.tsx:

import React from "react";

const Home: React.FC = () => (
  <h1>Welcome to your PWA!</h1>
);

export default Home;

Code splitting

Now we've split up our app into multiple sections, we're going to split the code too. A good way to improve loading times for PWAs is to ensure that the code is not built into big files. At the moment our app builds a single-file.js. If you run yarn build you'll see what this looks like:

  47.88 KB  build/static/js/2.89bc6648.chunk.js
  784 B     build/static/js/runtime-main.9c116153.js
  555 B     build/static/js/main.bc740179.chunk.js
  269 B     build/static/css/main.5ecd60fb.chunk.css

Notice the build/static/js/main.bc740179.chunk.js file. This is our single-file.js. It represents the compiled output of building the TypeScript files that make up our app. It will grow and grow as our app grows, eventually becoming problematic from a user loading speed perspective.

create-react-app is built upon webpack. There is excellent support for code splitting in webpack and hence create-react-app supports it by default. Let's apply it to our app. Again we're going to change App.tsx.

Where we previously had:

import About from "./About";
import Home from "./Home";

Let's replace with:

const About = lazy(() => import('./About'));
const Home = lazy(() => import('./Home'));

This is the syntax to lazily load components in React. You'll note that it internally uses the dynamic import() syntax which webpack uses as a "split point".

Let's also give React something to render whilst it waits for the dynamic imports to be resolved. Just inside our <Router> component we'll add a <Suspense> component too:

  <Router>
    <Suspense fallback={<div>Loading...</div>}>
    {/*...*/}
    </Suspense>
  </Router>

The <Suspense> component will render the <div>Loading...</div> whilst it waits for a routes code to be dynamically loaded. So our final App.tsx component ends up looking like this:

import React, { lazy, Suspense } from "react";
import { BrowserRouter as Router, Switch, Route, Link } from "react-router-dom";
const About = lazy(() => import("./About"));
const Home = lazy(() => import("./Home"));

const App: React.FC = () => (
  <Router>
    <Suspense fallback={<div>Loading...</div>}>
      <nav>
        <ul>
          <li>
            <Link to="/">Home</Link>
          </li>
          <li>
            <Link to="/about">About</Link>
          </li>
        </ul>
      </nav>
      <Switch>
        <Route path="/about">
          <About />
        </Route>
        <Route path="/">
          <Home />
        </Route>
      </Switch>
    </Suspense>
  </Router>
);

export default App;

This is now a code split application. How can we tell? If we run yarn build again we'll see something like this:

  47.88 KB          build/static/js/2.89bc6648.chunk.js
  1.18 KB (+428 B)  build/static/js/runtime-main.415ab5ea.js
  596 B (+41 B)     build/static/js/main.e60948bb.chunk.js
  269 B             build/static/css/main.5ecd60fb.chunk.css
  233 B             build/static/js/4.0c85e1cb.chunk.js
  228 B             build/static/js/3.eed49094.chunk.js

Note that we now have multiple *.chunk.js files. Our initial main.*.chunk.js and then 3.*.chunk.js representing Home.tsx and 4.*.chunk.js representing About.tsx.

As we continue to build out our app from this point we'll have a great approach in place to ensure that users load files as they need to and that those files should not be too large. Great performance which will scale.

Deploy your PWA

Now that we have our basic PWA in place, let's deploy it so the outside world can appreciate it. We're going to use Netlify for this.

The source code of our PWA lives on GitHub here: https://github.com/johnnyreilly/pwa-react-typescript

We're going to log into Netlify, click on the "Create a new site" option and select GitHub as the provider. We'll need to authorize Netlify to access our GitHub.

You may need to click the "Configure Netlify on GitHub" button to grant permissions for Netlify to access your repo like so:

Then you can select your repo from within Netlify. All of the default settings that Netlify provides should work for our use case:

Let's hit the magic "Deploy site" button! In a matter of minutes you'll find that Netlify has deployed your PWA.

If we browse to the URL provided by Netlify we'll be able to see the deployed PWA in action. (You also have the opportunity to set up a custom domain name that you would typically want outside of a simple demo such as this.) Importantly this will be served over HTTPS which will allow our Service Worker to operate.

Now that we know it's there, let's see how what we've built holds up according to the professionals. We're going to run the Google Chrome Developer Tools Audit against our PWA:

That is a good start for our PWA!

This post was originally published on LogRocket.

The source code for this project can be found here.

Tuesday, 21 January 2020

LICENSE to kill your PWA

Update: 26/01/2020 - LICENSE to kill revoked!

Following the original publication of this post I received this tweet suggesting we should change the behaviour of the underlying terser-webpack-plugin:

That seemed like an excellent idea! I raised this PR which changes the behaviour such that instead of .LICENSE files being produced, .LICENSE.txt files are pumped out instead. Crucially they are IIS (and other servers) friendly. The great news is that future users of webpack / create-react-app etc will not face this problem at all; result!

The tragedy

Recently my beloved PWA died. I didn't realise it at first. It wasn't until a week or so after the tragedy that I realised he'd gone. In his place was the stale memory of service workers gone by. Last week's code; cached and repeatedly served up to a disappointed audience. Terrible news.

What had happened? What indeed. The problem was quirky and (now that I know the answer) I'm going to share it with you. Because it's entirely non-obvious.

The mystery

Once I realised that I was repeatedly being served up an old version of my PWA, I got to wondering.... Why? What's happening? What's wrong? What did I do? I felt bad. I stared at the ceiling. I sighed and opened my Chrome devtools. With no small amount of sadness I went to the Application tab, hit Service Workers and then Unregister.

Then I hit refresh and took a look at console. I saw this:

What does this mean? Something about a "bad-precaching-response". And apparently this was happening whilst trying to load this resource: /static/js/6.20102e99.chunk.js.LICENSE?__WB_REVISION__=e2fc36

This 404 was preventing pre-caching from executing successfully. This was what was killing my PWA. This was the perpetrator. How to fix this? Read on!

The investigation

Time to find out what's going on. I dropped that URL into my browser to see what would happen. 404 city man:

So, to disk. I took a look at what was actually on the server in that location. Sure enough, the file existed. When I opened it up I found this:

/**
* A better abstraction over CSS.
*
* @copyright Oleg Isonen (Slobodskoi) / Isonen 2014-present
* @website https://github.com/cssinjs/jss
* @license MIT
*/
 
/*
object-assign
(c) Sindre Sorhus
@license MIT
*/
 
/** @license React v16.12.0
* react.production.min.js
*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
 
/** @license React v16.12.0
* react-dom.production.min.js
*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
 
/** @license React v0.18.0
* scheduler.production.min.js
*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
 
/** @license React v16.12.0
* react-is.production.min.js
*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

What is this? Well, as the name of the file suggests, it's licenses. For some reason, my build was scraping the licenses from the head of some of my files and placing them in a separate 6.20102e99.chunk.js.LICENSE file. Doing some more digging I happened upon this discussion against the create-react-app project. It's worth saying, that my PWA was an ejected create-react-app project.

It turned out the the issue was related to the terser-webpack-plugin. The default behaviour performs this kind of license file extraction. The app was being served by an IIS server and it wasn't configured to support the .LICENSE file type.

The resolution

The simplest solution was simply this: wave goodbye to LICENSE files. If you haven't ejected from your create-react-app then this might be a problem. But since I had, I was able to make this tweak to the terser settings in the webpack.config.js:

new TerserPlugin({
    /* TURN OFF LICENSE FILES - SEE https://github.com/facebook/create-react-app/issues/6441 */
    extractComments: false,
    /* TURN OFF LICENSE FILES - Tweak by John Reilly */
    terserOptions: {
        // ....           

And with this we say goodbye to our 404s and hello to a resurrected PWA!

Thursday, 2 January 2020

EF Core 3.1 breaks left join with no navigation property

Just recently my team took on the challenge of upgrading our codebase from .NET Core 2.2 to .NET Core 3.1. Along the way we encountered a quirky issue which caused us much befuddlement. Should you be befuddled too, then maybe this can help you.

Whilst running our app, we started encountering an error with an Entity Framework Query that looked like this:

var stuffWeCareAbout = await context.Things
    .Include(thing => thing.ThisIsFine)
    .Include(thing => thing.Problematic)
    .Where(thing => thing.CreatedOn > startFromThisTime && thing.CreatedOn < endAtThisTime)
    .OrderByDescending(thing => thing.CreatedOn)
    .ToArrayAsync();

Join me!

As EF Core tried to join from the Things table to the Problematic table (some obfuscation in table names here), that which worked in .NET Core 2.2 was not working in .NET Core 3.1. Digging into the issue, we discovered EF Core was generating an invalid LEFT JOIN:

fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
      Failed executing DbCommand (18ms) [Parameters=[@__startFromThisTime_0='?' (DbType = DateTime2), @__endAtThisTime_1='?' (DbType = DateTime2)], CommandType='Text', CommandTimeout='30']
      SELECT [o].[ThingId], [o].[AnonymousId], [o].[CreatedOn],  [o].[Status], [o].[UpdatedOn], [o0].[Id], [o0].[ThingId], [o0].[Name], [o1].[ThingId], [o1].[Status], [o1].[CreatedOn], [o1].[ThingThingId], [o1].[SentOn]
      FROM [Things] AS [o]
      LEFT JOIN [ThisIsFines] AS [o0] ON [o].[ThingId] = [o0].[ThingId]
      LEFT JOIN [Problematic] AS [o1] ON [o].[ThingId] = [o1].[ThingThingId]
      WHERE ([o].[CreatedOn] @__startFromThisTime_0) AND ([o].[CreatedOn] < @__endAtThisTime_1)
      ORDER BY [o].[CreatedOn] DESC, [o].[ThingId], [o1].[ThingId], [o1].[Status]
Microsoft.EntityFrameworkCore.Database.Command: Error: Failed executing DbCommand (18ms) [Parameters=[@__startFromThisTime_0='?' (DbType = DateTime2), @__endAtThisTime_1='?' (DbType = DateTime2)], CommandType='Text', CommandTimeout='30']
SELECT [o].[ThingId], [o].[AnonymousId], [o].[CreatedOn], [o].[Status], [o].[UpdatedOn], [o0].[Id], [o0].[ThingId], [o0].[Name], [o1].[ThingId], [o1].[Status], [o1].[CreatedOn], [o1].[ThingThingId], [o1].[SentOn]
FROM [Things] AS [o]
LEFT JOIN [ThisIsFines] AS [o0] ON [o].[ThingId] = [o0].[ThingId]
LEFT JOIN [Problematic] AS [o1] ON [o].[ThingId] = [o1].[ThingThingId]
WHERE ([o].[CreatedOn] @__startFromThisTime_0) AND ([o].[CreatedOn] < @__endAtThisTime_1)
ORDER BY [o].[CreatedOn] DESC, [o].[ThingId], [o1].[ThingId], [o1].[Status]

Do you see it? Probably not; it took us a while too... The issue lay here:

LEFT JOIN [Problematic] AS [o1] ON [o].[ThingId] = [o1].[ThingThingId]

This should actually have been:

LEFT JOIN [Problematic] AS [o1] ON [o].[ThingId] = [o1].[ThingId]

For some reason EF Core was looking for ThingThingId where it should have looked for ThingId. But why?

Navigation properties to the rescue!

This was the Problematic class:

using System;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace Treasury.Data.Entities
{
    public class Problematic
    {
        [ForeignKey("Thing")]
        [Required]
        public Guid ThingId { get; set; }
        [Required]
        public DateTime CreatedOn { get; set; }
        public DateTime SentOn { get; set; }
    }
}

If you look closely you'll see it has a ForeignKey but no accompanying Navigation property. So let's add one:

using System;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace Our.App
{
    public class Problematic
    {
        [ForeignKey("Thing")]
        [Required]
        public Guid ThingId { get; set; }
        [Required]
        public DateTime CreatedOn { get; set; }
        public DateTime SentOn { get; set; }

        /* THIS NAVIGATION PROPERTY IS WHAT WE NEEDED!!! */
        public virtual Thing Thing { get; set; }
    }
}

With this in place our app starts generating the SQL we need.