Azure App Service has a feature which is intended to allow Authentication and Authorization to be applied outside of your application code. It's called "Easy Auth". Unfortunately, in the context of App Services it doesn't work with .NET Core and .NET. Perhaps it would be better to say: of the various .NETs, it supports .NET Framework. To quote the docs:
At this time, ASP.NET Core does not currently support populating the current user with the Authentication/Authorization feature. However, some 3rd party, open source middleware components do exist to help fill this gap.
Thanks to Maxime Rouiller there's a way forward here. However, as I was taking this for a spin today, I discovered another issue.