Skip to main content

3 posts tagged with "csharp"

View All Tags

NSwag: TypeScript and CSharp client generation based on an API

ยท 9 min read

Generating clients for APIs is a tremendous way to reduce the amount of work you have to do when you're building a project. Why handwrite that code when it can be auto-generated for you quickly and accurately by a tool like NSwag? To quote the docs:

The NSwag project provides tools to generate OpenAPI specifications from existing ASP.NET Web API controllers and client code from these OpenAPI specifications. The project combines the functionality of Swashbuckle (OpenAPI/Swagger generation) and AutoRest (client generation) in one toolchain.

There's some great posts out there that show you how to generate the clients with NSwag using an nswag.json file directly from a .NET project.

However, what if you want to use NSwag purely for its client generation capabilities? You may have an API written with another language / platform that exposes a Swagger endpoint, that you simply wish to create a client for. How do you do that? Also, if you want to do some special customisation of the clients you're generating, you may find yourself struggling to configure that in nswag.json. In that case, it's possible to hook into NSwag directly to do this with a simple .NET console app.

This post will:

  • Create a .NET API which exposes a Swagger endpoint. (Alternatively, you could use any other Swagger endpoint; for example an Express API.)
  • Create a .NET console app which can create both TypeScript and CSharp clients from a Swagger endpoint.
  • Create a script which, when run, creates a TypeScript client.
  • Consume the API using the generated client in a simple TypeScript application.

You will need both Node.js and the .NET SDK installed.

Create an API#

We'll now create an API which exposes a Swagger / Open API endpoint. Whilst we're doing that we'll create a TypeScript React app which we'll use later on. We'll drop to the command line and enter the following commands which use the .NET SDK, node and the create-react-app package:

mkdir srccd srcnpx create-react-app client-app --template typescriptmkdir server-appcd server-appdotnet new api -o APIcd APIdotnet add package NSwag.AspNetCore

We now have a .NET API with a dependency on NSwag. We'll start to use it by replacing the Startup.cs that's been generated with the following:

using Microsoft.AspNetCore.Builder;using Microsoft.AspNetCore.Hosting;using Microsoft.Extensions.Configuration;using Microsoft.Extensions.DependencyInjection;using Microsoft.Extensions.Hosting;
namespace API{    public class Startup    {        const string ALLOW_DEVELOPMENT_CORS_ORIGINS_POLICY = "AllowDevelopmentSpecificOrigins";        const string LOCAL_DEVELOPMENT_URL = "http://localhost:3000";
        public Startup(IConfiguration configuration)        {            Configuration = configuration;        }
        public IConfiguration Configuration { get; }
        // This method gets called by the runtime. Use this method to add services to the container.        public void ConfigureServices(IServiceCollection services)        {
            services.AddCors(options => {                options.AddPolicy(name: ALLOW_DEVELOPMENT_CORS_ORIGINS_POLICY,                    builder => {                        builder.WithOrigins(LOCAL_DEVELOPMENT_URL)                            .AllowAnyMethod()                            .AllowAnyHeader()                            .AllowCredentials();                    });            });
            // Register the Swagger services            services.AddSwaggerDocument();        }
        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.        public void Configure (IApplicationBuilder app, IWebHostEnvironment env)        {            if (env.IsDevelopment())            {                app.UseDeveloperExceptionPage();            }             else            {                app.UseExceptionHandler("/Error");                app.UseHsts ();                app.UseHttpsRedirection();            }
            app.UseDefaultFiles();            app.UseStaticFiles();
            // Register the Swagger generator and the Swagger UI middlewares            app.UseOpenApi();            app.UseSwaggerUi3();
            if (env.IsDevelopment())                app.UseCors(ALLOW_DEVELOPMENT_CORS_ORIGINS_POLICY);
            app.UseEndpoints(endpoints =>            {                endpoints.MapControllers();            });        }    }}

The significant changes in the above Startup.cs are:

  1. Exposing a Swagger endpoint with UseOpenApi and UseSwaggerUi3. NSwag will automagically create Swagger endpoints in your application for all your controllers. The .NET template ships with a WeatherForecastController.
  2. Allowing Cross-Origin Requests (CORS) which is useful during development (and will facilitate a demo later).

Back in the root of our project we're going to initialise an npm project. We're going to use this to put in place a number of handy npm scripts that will make our project easier to work with. So we'll npm init and accept all the defaults.

Now we're going add some dependencies which our scripts will use: npm install cpx cross-env npm-run-all start-server-and-test

We'll also add ourselves some scripts to our package.json:

"scripts": {    "postinstall": "npm run install:client-app && npm run install:server-app",    "install:client-app": "cd src/client-app && npm install",    "install:server-app": "cd src/server-app/API && dotnet restore",    "build": "npm run build:client-app && npm run build:server-app",    "build:client-app": "cd src/client-app && npm run build",    "postbuild:client-app": "cpx \"src/client-app/build/**/*.*\" \"src/server-app/API/wwwroot/\"",    "build:server-app": "cd src/server-app/API && dotnet build --configuration release",    "start": "run-p start:client-app start:server-app",    "start:client-app": "cd src/client-app && npm start",    "start:server-app": "cross-env ASPNETCORE_URLS=http://*:5000 ASPNETCORE_ENVIRONMENT=Development dotnet watch --project src/server-app/API run --no-launch-profile"  }

Let's walk through what the above scripts provide us with:

  • Running npm install in the root of our project will not only install dependencies for our root package.json, thanks to our postinstall, install:client-app and install:server-app scripts it will install the React app and .NET app dependencies as well.
  • Running npm run build will build our client and server apps.
  • Running npm run start will start both our React app and our .NET app. Our React app will be started at http://localhost:3000. Our .NET app will be started at http://localhost:5000 (some environment variables are passed to it with cross-env ).

Once npm run start has been run, you will find a Swagger endpoint at http://localhost:5000/swagger:

swagger screenshot

The client generator project#

Now we've scaffolded our Swagger-ed API, we want to put together the console app that will generate our typed clients.

cd src/server-appdotnet new console -o APIClientGeneratorcd APIClientGeneratordotnet add package NSwag.CodeGeneration.CSharpdotnet add package NSwag.CodeGeneration.TypeScriptdotnet add package NSwag.Core

We now have a console app with dependencies on the code generation portions of NSwag. Now let's change up Program.cs to make use of this:

using System;using System.IO;using System.Threading.Tasks;using NJsonSchema;using NJsonSchema.CodeGeneration.TypeScript;using NJsonSchema.Visitors;using NSwag;using NSwag.CodeGeneration.CSharp;using NSwag.CodeGeneration.TypeScript;
namespace APIClientGenerator{    class Program    {        static async Task Main(string[] args)        {            if (args.Length != 3)                throw new ArgumentException("Expecting 3 arguments: URL, generatePath, language");
            var url = args[0];            var generatePath = Path.Combine(Directory.GetCurrentDirectory(), args[1]);            var language = args[2];
            if (language != "TypeScript" && language != "CSharp")                throw new ArgumentException("Invalid language parameter; valid values are TypeScript and CSharp");
            if (language == "TypeScript")                 await GenerateTypeScriptClient(url, generatePath);            else                await GenerateCSharpClient(url, generatePath);        }
        async static Task GenerateTypeScriptClient(string url, string generatePath) =>            await GenerateClient(                document: await OpenApiDocument.FromUrlAsync(url),                generatePath: generatePath,                generateCode: (OpenApiDocument document) =>                {                    var settings = new TypeScriptClientGeneratorSettings();
                    settings.TypeScriptGeneratorSettings.TypeStyle = TypeScriptTypeStyle.Interface;                    settings.TypeScriptGeneratorSettings.TypeScriptVersion = 3.5M;                    settings.TypeScriptGeneratorSettings.DateTimeType = TypeScriptDateTimeType.String;
                    var generator = new TypeScriptClientGenerator(document, settings);                    var code = generator.GenerateFile();
                    return code;                }            );
        async static Task GenerateCSharpClient(string url, string generatePath) =>            await GenerateClient(                document: await OpenApiDocument.FromUrlAsync(url),                generatePath: generatePath,                generateCode: (OpenApiDocument document) =>                {                    var settings = new CSharpClientGeneratorSettings                    {                        UseBaseUrl = false                    };
                    var generator = new CSharpClientGenerator(document, settings);                    var code = generator.GenerateFile();                    return code;                }            );
        private async static Task GenerateClient(OpenApiDocument document, string generatePath, Func<OpenApiDocument, string> generateCode)        {            Console.WriteLine($"Generating {generatePath}...");
            var code = generateCode(document);
            await System.IO.File.WriteAllTextAsync(generatePath, code);        }    }}

We've created ourselves a simple .NET console application that creates TypeScript and CSharp clients for a given Swagger URL. It expects three arguments:

  • url - the url of the swagger.json file to generate a client for.
  • generatePath - the path where the generated client file should be placed, relative to this project.
  • language - the language of the client to generate; valid values are "TypeScript" and "CSharp".

To create a TypeScript client with it then we'd use the following command:

dotnet run --project src/server-app/APIClientGenerator http://localhost:5000/swagger/v1/swagger.json src/client-app/src/clients.ts TypeScript

However, for this to run successfully, we'll first have to ensure the API is running. It would be great if we had a single command we could run that would:

  • bring up the API
  • generate a client
  • bring down the API

Let's make that.

Building a "make a client" script#

In the root of the project we're going to add the following scripts:

"generate-client:server-app": "start-server-and-test generate-client:server-app:serve http-get://localhost:5000/swagger/v1/swagger.json generate-client:server-app:generate",    "generate-client:server-app:serve": "cross-env ASPNETCORE_URLS=http://*:5000 ASPNETCORE_ENVIRONMENT=Development dotnet run --project src/server-app/API --no-launch-profile",    "generate-client:server-app:generate": "dotnet run --project src/server-app/APIClientGenerator http://localhost:5000/swagger/v1/swagger.json src/client-app/src/clients.ts TypeScript",

Let's walk through what's happening here. Running npm run generate-client:server-app will:

  • Use the start-server-and-test package to spin up our server-app by running the generate-client:server-app:serve script.
  • start-server-and-test waits for the Swagger endpoint to start responding to requests. When it does start responding, start-server-and-test runs the generate-client:server-app:generate script which runs our APIClientGenerator console app and provides it with the URL where our swagger can be found, the path of the file to generate and the language of "TypeScript"

If you were wanting to generate a C# client (say if you were writing a Blazor app) then you could change the generate-client:server-app:generate script as follows:

"generate-client:server-app:generate": "dotnet run --project src/server-app/ApiClientGenerator http://localhost:5000/swagger/v1/swagger.json clients.cs CSharp",

Consume our generated API client#

Let's run the npm run generate-client:server-app command. It creates a clients.ts file which nestles nicely inside our client-app. We're going to exercise that in a moment. First of all, let's enable proxying from our client-app to our server-app following the instructions in the Create React App docs and adding the following to our client-app/package.json:

"proxy": "http://localhost:5000"

Now let's start our apps with npm run start. We'll then replace the contents of App.tsx with:

import React from "react";import "./App.css";import { WeatherForecast, WeatherForecastClient } from "./clients";
function App() {  const [weather, setWeather] = React.useState<WeatherForecast[] | null>();  React.useEffect(() => {    async function loadWeather() {      const weatherClient = new WeatherForecastClient(/* baseUrl */ "");      const forecast = await weatherClient.get();      setWeather(forecast);    }    loadWeather();  }, [setWeather]);
  return (    <div className="App">      <header className="App-header">        {weather ? (          <table>            <thead>              <tr>                <th>Date</th>                <th>Summary</th>                <th>Centigrade</th>                <th>Fahrenheit</th>              </tr>            </thead>            <tbody>              {{ date, summary, temperatureC, temperatureF }) => (                <tr key={date}>                  <td>{new Date(date).toLocaleDateString()}</td>                  <td>{summary}</td>                  <td>{temperatureC}</td>                  <td>{temperatureF}</td>                </tr>              ))}            </tbody>          </table>        ) : (          <p>Loading weather...</p>        )}      </header>    </div>  );}
export default App;

Inside the React.useEffect above you can see we create a new instance of the auto-generated WeatherForecastClient. We then call weatherClient.get() which sends the GET request to the server to acquire the data and provides it in a strongly typed fashion (get() returns an array of WeatherForecast). This is then displayed on the page like so:

load data from server

As you an see we're loading data from the server using our auto-generated client. We're reducing the amount of code we have to write and we're reducing the likelihood of errors.

This post was originally posted on LogRocket.

Nullable reference types; CSharp's very own strictNullChecks

ยท 4 min read

'Tis the season to play with new compiler settings! I'm a very keen TypeScript user and have been merrily using strictNullChecks since it shipped. I was dimly aware that C# was also getting a similar feature by the name of nullable reference types.

It's only now that I've got round to taking at look at this marvellous feature. I thought I'd share what moving to nullable reference types looked like for me; and what code changes I found myself making as a consequence.

Turning on nullable reference types#

To turn on nullable reference types in a C# project you should pop open the .csproj file and ensure it contains a &lt;Nullable&gt;enable&lt;/Nullable&gt;. So if you had a .NET Core 3.1 codebase it might look like this:

<PropertyGroup>    <TargetFramework>netcoreapp3.1</TargetFramework>    <Nullable>enable</Nullable>  </PropertyGroup>

When you compile from this point forward, possible null reference types are reported as warnings. Consider this C#:

[ApiController]    public class UserController : ControllerBase    {        private readonly ILogger<UserController> _logger;
        public UserController(ILogger<UserController> logger)        {            _logger = logger;        }
        [AllowAnonymous]        [HttpGet("UserName")]        public string GetUserName()        {            if (User.Identity.IsAuthenticated) {                _logger.LogInformation("{User} is getting their username", User.Identity.Name);                return User.Identity.Name;            }
            _logger.LogInformation("The user is not authenticated");            return null;        }    }

A dotnet build results in this:

dotnet build --configuration release
Microsoft (R) Build Engine version 16.7.1+52cd83677 for .NETCopyright (C) Microsoft Corporation. All rights reserved.
  Determining projects to restore...  Restored /Users/jreilly/code/app/src/server-app/Server/app.csproj (in 471 ms).Controllers/UserController.cs(38,24): warning CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]Controllers/UserController.cs(42,20): warning CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]  app -> /Users/jreilly/code/app/src/server-app/Server/bin/release/netcoreapp3.1/app.dll  app -> /Users/jreilly/code/app/src/server-app/Server/bin/release/netcoreapp3.1/app.Views.dll
Build succeeded.
Controllers/UserController.cs(38,24): warning CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]Controllers/UserController.cs(42,20): warning CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]    2 Warning(s)    0 Error(s)

You see the two "Possible null reference return." warnings? Bingo

Really make it hurt#

This is good - information is being surfaced up. But it's a warning. I could ignore it. I like compilers to get really up in my face and force me to make a change. I'm not into warnings; I'm into errors. Know what works for you. If you're similarly minded, you can upgrade nullable reference warnings to errors by tweaking the .csproj a touch further. Add yourself a &lt;WarningsAsErrors&gt;nullable&lt;/WarningsAsErrors&gt; element. So maybe your .csproj now looks like this:

<PropertyGroup>    <TargetFramework>netcoreapp3.1</TargetFramework>    <Nullable>enable</Nullable>    <WarningsAsErrors>nullable</WarningsAsErrors>  </PropertyGroup>

And a dotnet build will result in this:

dotnet build --configuration release
Microsoft (R) Build Engine version 16.7.1+52cd83677 for .NETCopyright (C) Microsoft Corporation. All rights reserved.
  Determining projects to restore...  Restored /Users/jreilly/code/app/src/server-app/Server/app.csproj (in 405 ms).Controllers/UserController.cs(38,24): error CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]Controllers/UserController.cs(42,20): error CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]
Controllers/UserController.cs(38,24): error CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]Controllers/UserController.cs(42,20): error CS8603: Possible null reference return. [/Users/jreilly/code/app/src/server-app/Server/app.csproj]    0 Warning(s)    2 Error(s)

Yay! Errors!

What do they mean?#

"Possible null reference return" isn't the clearest of errors. What does that actually amount to? Well, it amounts to the compiler saying "you're a liar! (maybe)". Let's look again at the code where this error is reported:

[AllowAnonymous]        [HttpGet("UserName")]        public string GetUserName()        {            if (User.Identity.IsAuthenticated) {                _logger.LogInformation("{User} is getting their username", User.Identity.Name);                return User.Identity.Name;            }
            _logger.LogInformation("The user is not authenticated");            return null;        }

We're getting that error reported where we're returning null and where we're returning User.Identity.Name which may be null. And we're getting that because as far as the compiler is concerned string has changed. Before we turned on nullable reference types the compiler considered string to mean string ORnull. Now, string means string.

This is the same sort of behaviour as TypeScripts strictNullChecks. With TypeScript, before you turn on strictNullChecks, as far as the compiler is concerned, string means stringORnullORundefined (JavaScript didn't feel one null-ish value was enough and so has two - don't ask). Once strictNullChecks is on, string means string.

It's a lot clearer. And that's why the compiler is getting antsy. The method signature is string, but it can see null potentially being returned. It doesn't like it. By and large that's good. We want the compiler to notice this as that's the entire point. We want to catch accidental nulls before they hit a user. This is great! However, what do you do if have a method (as we do) that legitimately returns a string or null?

Widening the type to include null#

We change the signature from this:

public string GetUserName()

To this:

public string? GetUserName()

That's right, the simple addition of ? marks a reference type (like a string) as potentially being null. Adding that means that we're potentially returning null, but we're sure about it; there's intention here - it's not accidental. Wonderful!

Task.WhenAll / Select is a footgun ๐Ÿ‘Ÿ๐Ÿ”ซ

ยท 6 min read

This post differs from my typical fayre. Most often I write "here's how to do a thing". This is not that. It's more "don't do this thing I did". And maybe also, "how can we avoid a situation like this happening again in future?". On this topic I very much don't have all the answers - but by putting my thoughts down maybe I'll learn and maybe others will educate me. I would love that!

Doing things that don't scale#

The platform that I work on once had zero users. We used to beg people to log in and see what we had built. Those days are (happily) but a memory. We're getting popular.

As our platform has grown in popularity it has revealed some bad choices we made. Approaches that look fine on the surface (and that work just dandy when you have no users) may start to cause problems as your number of users grows.

I wanted to draw attention to one approach in particular that impacted us severely. In this case "impacted us severely" is a euphemism for "brought the site down and caused a critical incident".

You don't want this to happen to you. Trust me. So, what follows is a cautionary tale. The purpose of which is simply this: reader, do you have code of this ilk in your codebase? If you do: out, damn'd spot! out, I say!

So cool, so terrible#

I love LINQ. I love a declarative / functional style of coding. It appeals to me on some gut level. I find it tremendously readable. Read any C# of mine and the odds are pretty good that you'll find some LINQ in the mix.

Imagine this scenario: you have a collection of user ids. You want to load the details of each user represented by their id from an API. You want to bag up all of those users into some kind of collection and send it back to the calling code.

Reading that, if you're like me, you're imagining some kind of map operation which loads the user details for each user id. Something like this:

var users = userIds.Select(userId => GetUserDetails(userId)).ToArray(); // users is User[]

Lovely. But you'll note that I'm loading users from an API. Oftentimes, APIs are asynchronous. Certainly, in my case they were. So rather than calling a GetUserDetails function I found myself calling a GetUserDetailsAsync function, behind which an HTTP request is being sent and, later, a response is being returned.

So how do we deal with this? Task.WhenAll my friends!

var userTasks = userIds.Select(userId => GetUserDetailsAsync(userId));var users = await Task.WhenAll(tasks); // users is User[]

It worked great! Right up until to the point where it didn't. These sorts of shenanigans were fine when we had a minimal number of users... But there came a point where problems arose. It got to the point where that simple looking mapping operation became a cause of many, many, many HTTP requests being fired concurrently. Then bad things started to happen. Not only did we realise we were launching a denial of service attack on the API we were consuming, we were bringing our own application to collapse.

Not a proud day.

What is the problem?#

Through log analysis, code reading and speculation, (with the help of the invaluable Robski) we came to realise that the cause of our woes was the Task.WhenAll / Select combination. Exercising that codepath was a surefire way to bring the application to its knees.

As I read around on the topic I happened upon Mark Heath's excellent list of Async antipatterns. Number #6 on the list is "Excessive parallelization". It describes a nearly identical scenario to my own:

Now, this does "work", but what if there were 10,000 orders? We've flooded the thread pool with thousands of tasks, potentially preventing other useful work from completing. If ProcessOrderAsync makes downstream calls to another service like a database or a microservice, we'll potentially overload that with too high a volume of calls.

We're definitely overloading the API we're consuming with too high a volume of calls. I have to admit that I'm less clear on the direct reason that a Task.WhenAll / Select combination could prove fatal to our application. Mark suggests this approach will flood the thread pool with tasks. As I read around on async and await it's repeated again and again that a Task is not the same thing as a Thread. I have to hold my hands up here and say that I don't understand the implementation of async / await in C# well enough. These docs are helpful but I still don't think the penny has fully dropped for me yet. I will continue to read.

One thing we learned as we debugged the production k8s pod was that, prior to its collapse, our app appeared to be opening up 1 million connections to the API we were consuming. Which seemed a bit much. Worthy of investigation. It's worth saying that we're not certain this is exactly what is happening; we have less instrumentation in place than we'd like. But some fancy wc grepping on Robski's behalf suggested this was the case.

What will we change in future?#

A learning that came out of this for us was this: we need more metrics exposed. We don't understand our application's behaviour under load as well as we'd like. So we're planning to do some work with App Metrics and Grafana so we've a better idea of how our application performs. If you want to improve something, first measure it.

Another fly in the ointment was that we were unable to reproduce the issue when running locally. It's worth saying here that I develop on a Windows machine and, when deployed, our application runs in a (Linux) Docker container. So there's a difference and a distance between our development experience and our running one.

I'm planning to migrate to developing in a devcontainer where that's possible. That should narrow the gap between our production experience and our development one. Reducing the difference between the two is always useful as it means you're less likely to get different behaviour (ie "problems") in production as compared to development. I'm curious as to whether I'll be able to replicate that behaviour in a devcontainer.

What did we do right now?#

To solve the immediate issue we were able to pivot away to a completely different approach. We moved aggregation from our ASP.NET Core web application to our TypeScript / React client with a (pretty sweet) custom hook. The topic for a subsequent blog post.

Moving to a different approach solved my immediate issue. But it left me puzzling. What was actually going wrong? Is it thread pool exhaustion? Is it something else? So many possibilities!

If anyone has any insights they'd like to share that would be incredible! I've also asked a question on Stack Overflow which has kindly had answers from generous souls. James Skimming's answer lead me to Steve Gordon's excellent post on connection pooling which I'm still absorbing and seems like it could be relevant.