Dogs Chasing Squirrels

A software development blog

About Mike Bennett

Streaming a response in .NET Core WebApi

1

We, as web developers, should try to avoid loading files into memory before returning them via our APIs. Servers are a shared resource and so we’d like to use as little memory as we can. We do this by writing large responses out as a stream.

In the ASP.NET MVC days, I would use PushStreamContent to stream data out in a Web API. That doesn’t seem to exist in .NET core and, even if it did, we don’t need it anyway. There’s an easy way to get direct access to the output stream and that’s just with the controller’s this.Response.Body, which is a Stream.

In this sample, I just grab a file out of my downloads folder and stream it back out:

[HttpGet]
[Route( "streaming" )]
public async Task GetStreaming() {
    const string filePath = @"C:\Users\mike\Downloads\dotnet-sdk-3.1.201-win-x64.exe";
    this.Response.StatusCode = 200;
    this.Response.Headers.Add( HeaderNames.ContentDisposition, $"attachment; filename=\"{Path.GetFileName( filePath )}\"" );
    this.Response.Headers.Add( HeaderNames.ContentType, "application/octet-stream"  );
    var inputStream = new FileStream( filePath, FileMode.Open, FileAccess.Read );
    var outputStream = this.Response.Body;
    const int bufferSize = 1 << 10;
    var buffer = new byte[bufferSize];
    while ( true ) {
        var bytesRead = await inputStream.ReadAsync( buffer, 0, bufferSize );
        if ( bytesRead == 0 ) break;
        await outputStream.WriteAsync( buffer, 0, bytesRead );
    }
    await outputStream.FlushAsync();
}

This does the same thing in F#:

[<HttpGet>]
[<Route("streaming")>]
member __.GetStreaming() = async {
    let filePath = @"C:\Users\mike\Downloads\dotnet-sdk-3.1.201-win-x64.exe"
    __.Response.StatusCode <- 200
    __.Response.Headers.Add( HeaderNames.ContentDisposition, StringValues( sprintf "attachment; filename=\"%s\"" ( System.IO.Path.GetFileName( filePath ) ) ) )
    __.Response.Headers.Add( HeaderNames.ContentType, StringValues( "application/octet-stream" ) )
    let inputStream = new FileStream( filePath, FileMode.Open, FileAccess.Read )
    let outputStream = __.Response.Body
    let bufferSize = 1 <<< 10
    let buffer = Array.zeroCreate<byte> bufferSize
    let mutable loop = true
    while loop do
        let! bytesRead = inputStream.ReadAsync( buffer, 0, bufferSize ) |> Async.AwaitTask
        match bytesRead with
        | 0 -> loop <- false
        | _ -> do! outputStream.WriteAsync( buffer, 0, bytesRead ) |> Async.AwaitTask
    do! outputStream.FlushAsync() |> Async.AwaitTask
    return EmptyResult()
}

A couple of important notes:
1. By default, you have to write to the stream using the Async methods. If you try to write with non-Async methods, you’ll get the error “Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true instead.” and, as the error says, you’ll have to enable the AllowSynchronousIO setting.
2. On C# you can have your streaming controller method return nothing at all. If you try the same on F#, you’ll get the error, midway through the response, “StatusCode cannot be set because the response has already started”. The solution to this is to have the method return an EmptyResult().

Starting and Stopping Azure Web Apps with a DevOps pipeline

0

We’re using Azure web services for our dev and test environments. Since we only use these environments during the day, I wanted to write some automated functions to turn them off at night and turn them back on in the morning. It can be done with Azure Functions, but I thought it would be easier to build into an Azure Pipeline where developers are all doing their work anyway, so if someone needed to work after hours they would know how to get at it.

First is the PowerShell script to start or stop a service, based on this post.

# This controls an app service.
Param(
    # The tenant ID
    [Parameter(Mandatory=$true)][string] $TenantId,
    # The subscription ID
    [Parameter(Mandatory=$true)] [string] $SubscriptionId,
    # The App Registration client ID
    [Parameter(Mandatory=$true)][string] $ClientId,
    # The App Registration client secret
    [Parameter(Mandatory=$true)][string] $ClientSecret,
    # The resource group of the service to control
    [Parameter(Mandatory=$true)][string] $ResourceGroup,
    # The name of the service to control
    [Parameter(Mandatory=$true)][string] $AppService,
    # The switch if we want to start
    [switch] $Start = $false,
    # The switch if we want to start
    [switch] $Stop = $false
)

$StartOrStop = ""
if ( $Start ) {
    $StartOrStop = "start"
}
if ( $Stop ) {
    $StartOrStop = "stop"
}

# Get the authentication token
$Auth = Invoke-RestMethod `
 -Uri "https://login.microsoftonline.com/$TenantId/oauth2/token?api-version=1.0" `
 -Method Post `
 -Body @{"grant_type" = "client_credentials"; `
 "resource" = "https://management.core.windows.net/"; `
 "client_id" = "$ClientId"; `
 "client_secret" = "$ClientSecret"}

$HeaderValue = "Bearer " + $Auth.access_token

# Control the service
Write-Output "Executing $AppService $StartOrStop..."
Invoke-RestMethod `
-Uri "https://management.azure.com/subscriptions/$SubscriptionId/resourceGroups/$ResourceGroup/providers/Microsoft.Web/sites/$AppService/$StartOrStop`?api-version=2018-02-01" `
-Method Post `
-Headers @{Authorization = $HeaderValue}   
Write-Output "Done $Command $AppService"

Then there's the pipeline yaml to run the script. This is the "Start" version. I'll leave "Stop" as an exercise for the reader.

name: $(Date:yyyyMMdd)-$(Rev:r)
jobs:
  - job: "Build"
    pool:
      vmImage: 'windows-2019'
    variables:
      tenantId: 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
      subscriptionId: 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
      clientId: 'XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
      # Note: clientSecret set in Azure Pipeline
      resourceGroup: 'MY_RESOURCE_GROUP'
      appService: 'MY_APP_SERVICE'
    steps:
      - task: PowerShell@2
        displayName: 'Start $(appService)'
        inputs:
          errorActionPreference: Stop
          targetType: filePath
          filePath: '$(Build.SourcesDirectory)\ControlAppService.ps1'
          arguments: '-Start -AppService $(appService) -TenantId $(tenantId) -SubscriptionId $(subscriptionId) -ClientId $(clientId) -ClientSecret $(clientSecret) -ResourceGroup $(resourceGroup)'

I have the clientSecret set up in the Azure Pipeline’s variables as a secret value.

That’s pretty much it. The pipelines are set to run on a timer: Stop at 7pm; Start at 7am.

Railway-Oriented Programming in F# and WebAPI

0

If you’re interested in learning about using railway-oriented programming in F#, you should be reading Railway oriented programming at fsharpforfunandprofit.com and The Marvels of Monads at microsoft.com. Stylish F# by Kit Eason has also been a help.

Functional programming is about functional composition and pipelineing functions from one to the next to get a result. Railway-oriented programming is about changing that pipeline to a track where if an operation succeeds, it goes forwards and if it fails, it cuts over to a failure track. F# already has built-in the Result object, a discriminated union giving success (Ok) or failure (Error) and the monadic functions, bind, map, and mapError.

I was interested in how these could be applied to a WebAPI endpoint. Let’s say we’re passing these results along a pipeline. What’s failure? In the end, it will be an IActionResult of some kind, probably a StatusCodeResult. 404, 401, 500, whatever. In the end, that’s an IActionResult, too, though with a status code of 200.

There’s an F#, functional web project that already does something like this, Suave.io, though it doesn’t look to be maintained anymore. Even so, they have some good, async implementations of the various railway/monadic functions like bind, compose, etc. I’ve tried to adapt them in to my own AsyncResult module.

Result

The result object is unchanged:

type Result<'TSuccess,'TFailure> =
    | Ok of 'TSuccess
    | Error of 'TFailure

Bind

First, bind, which takes some Result input and a function and, if the input is Ok, calls the function with the contents, and, if the input is Error, short-cuts and returns the error.
An async bind looks like this:

let bind f x = async {
    let! x' = x
    match x' with
    | Error e -> return Error e
    | Ok x'' -> return! f x''
}

Map

Next we have map. Say we have a function that takes some object and manipulates it returning another object. Map lets us insert that into our railway, with the function operating on the contents of the Ok result.
An async map looks like this:

let map f x = async {
    let! x' = x
    match x' with
    | Error e -> return Error e
    | Ok x'' ->
        let! r = f x''
        return Ok( r )

MapError

MapError is like map but instead we expect the function to operate on the Error result.
This is my async mapError:

let mapError f x = async {
    let! x' = x
    match x' with
    | Error e ->
        let! r = f e
        return Error( r )
    | Ok ok ->
        return Ok ok
}

Compose

Next we have compose, which lets us pipe two bound functions together. If the first function returns an Ok(x) as output, the second function takes the x as input and returns some Result. If the first function returns an Error, the second is never called.
This is the async compose:

let compose f1 f2 =
    fun x -> bind f2 (f1 x)

Custom Operators

We can create a few custom operators for our functions:

// bind operator
let (>>=) a b =
    bind b a

// compose operator
let (>=>) a b =
    compose a b

An Example WebAPI Controller

Let’s imagine a WebAPI controller endpoint that implements GET /thing/{id} where we return some Thing with the given ID. Normally we would:
* Check that the user has permission to get the thing.
* Get the thing from the database.
* Format it into JSON.
* Return it.
If the user doesn’t have permissions, we should get a 401 Unauthorized. If the Thing with the given ID isn’t found, we should get a 404 Not Found.

The functions making up our railway

Usually we want a connection to the database but I’m just going to fake it for this example:

let openConnection(): Async<IDbConnection> =
    async {
        return null
    }

We might also have a function that, given the identity in the HttpContext and a database connection could fetch the user’s roles. Again we’ll fake it. For testing purposes, we’ll say the user is an admin unless the thing ID ends in 99

let getRole ( connection : IDbConnection ) ( context : HttpContext ) =
    async {
        if context.Request.Path.Value.EndsWith("99") then return Ok "user"
        else return Ok "admin"
    }

Now we come to our first railway component. We want to check the user has the given role. If he does, we return Ok, if not an Error with the 401 Unauthorized code (not yet a StatusCodeResult)

let ensureUserHasRole requiredRole userRole =
    async {
        if userRole = requiredRole then return Ok()
        else return Error( HttpStatusCode.Unauthorized )
    }

Next we have a railway component that fetches the thing by ID. For testing purposes, we’ll say that if the ID is 0 we’ll return an Option.None and otherwise return an Option.Some. Although I haven’t added it here, I could imagine adding a try/catch that returns an Error 500 Internal Server Error when an exception is caught.

let fetchThingById (connection: IDbConnection) (thingId: int) () =
    async {
        match thingId with
        | 0 ->
            // Pretend we couldn't find it.
            return Ok( None )
        | _ ->
            // Pretend we got this from the DB
            return Ok( Some ( { Id = thingId; Name = "test" } ) )
    }

Our next railway component checks that a given object is found. If it’s Some, it returns Ok with the result. If it’s None, we get an Error, 404 Not Found.

let ensureFound ( value : 'a option ) = async {
    match value with
    | Some value' -> return Ok( value' )
    | None -> return Error( HttpStatusCode.NotFound )
}

Next we’ll create a function that just converts a value to a JSON result (maybe pretending there might be more complicated formatting going on here):

let toJsonResult ( value : 'a ) =
    async {
        return ( JsonResult( value ):> IActionResult )
    }    

Finally, we’ll add a function to convert that HttpStatusCode to a StatusCodeResult (also overkill – we could probably inline it):

let statusCodeToErrorResult ( code : HttpStatusCode ) = async {
    return ( StatusCodeResult( (int)code ) :> IActionResult )
}

When we end up, we’re going to have an Ok result of type IActionResult and an Error, also of type IActionResult. I want to coalesce the two into whatever the result is, regardless of whether it’s Ok or Error:

// If Error and OK are of the same type, returns the enclosed value.
let coalesce r = async {
    let! r' = r
    match r' with
    | Error e -> return e
    | Ok ok -> return ok
}

Putting it together

Here’s our railway in action:

// GET /thing/{thingId}
let getThing (thingId: int) (context: HttpContext): Async<IActionResult> =
    async {
        // Create a DB connection
        let! connection = openConnection()
        // Get the result
        let! result =
            // Starting with the context...
            context |> (
                // Get the user's role
                ( getRole connection )
                // Ensure the user is an admin.  
                >=> ( ensureUserHasRole "admin" )
                // Fetch the thing by ID
                >=> ( fetchThingById connection thingId ) 
                // Ensure if was found
                >=> ensureFound
                // Convert it to JSON
                >> ( map toJsonResult )
                // Map the error HttpStatusCode to an error StatusCodeResult
                >> ( mapError statusCodeToErrorResult )
                // Coalese the OK and Error into one IAction result
                >> coalesce
            )
        // Return the result
        return result
}

To summarize, we
* Get the user’s role, resulting in an Ok with the role (and no Error, though I could imagine catching an exception and returning a 500).
* See if the user has the role we need resulting in an Ok with no content or an Error(401).
* Fetch a Thing from the database, resulting in an Object.Some or Object.None.
* Check that it’s not None, returning an Error(404) if it is or an Ok(Thing).
* Mapping the Ok(Thing) into a Thing and turning the Thing into a JsonResult.
* or mapping the Error(HttpStatusCode) into a HttpStatusCode and turning the Error(HttpStatusCode) into a StatusCodeResult.
* Taking whichever result we ended up with, the JsonResult or StatusCodeResult and returning it.

If we run the website and call https://localhost:5001/thing/1 we get the JSON for our Thing.

{"id":1,"name":"test"}

If we call /thing/0 we get 404 Not Found. If we call thing/99 we get 401 Unauthorized.

There’s room here for some other methods. I could imagine wanting to wrap a call in a try/catch and return a 500 Server Error if it fails, for example.

The best part is that it’s a nice, readable, railway of functions. And our custom operators make it look good.

The code for this post can be found on GitHub.

Signature validation failed. Unable to match ‘kid’

0

I came across this error trying to validate tokens between a React app and an Okta developer page and Stack Overflow was giving me nothing.

On the client side, I was using the oidc-client.js, like so:

  const oidcConfiguration: any = {
    authority: env.__AUTH_AUTHORITY__,
    redirect_uri: env.__AUTH_REDIRECT_URI__,
    post_logout_redirect_uri: env.__AUTH_POST_LOGOUT_REDIRECT_URI__,
    silent_redirect_uri: env.__AUTH_SILENT_RENEW_URI__,
    client_id: env.__AUTH_CLIENT_ID__,
    response_type: 'id_token token',
    scope: 'openid profile email',
  }

The authority was my dev account, something like https://dev-123456.okta.com. I got this working and managed to get myself an access token.

On the .net core side, I was using basic JWT validation. I had been using Okta’s example but they amount to the same thing.

// Add JWT Bearer authentication
services.AddAuthentication( sharedOptions => {
      sharedOptions.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
      sharedOptions.DefaultChallengeScheme    = JwtBearerDefaults.AuthenticationScheme;
   } )
   .AddJwtBearer( options => {
      options.Authority = this._configuration["Auth:Authority"];
      options.Audience  = this._configuration["Auth:Audience"];
   } );

Using the same authority as getting the token, I got an error message that told me I needed to set up an authorization server. I set one up that was, by default, something like https://dev-123456.okta.com/oauth2/default. When I tried to authenticate my token against this server, I got the error “Signature validation failed. Unable to match ‘kid’”.

The solution turned out to be pretty simple: The client has to be changed to also use https://dev-123456.okta.com/oauth2/default to get the token instead of just https://dev-123456.okta.com.

Signing MSI Installers with a Code Signing Certificate

0

Signing installer MSIs with a code signing certificate prevents Windows from showing a big red “This application is untrusted!” warning when an installer is launched.

I recently had to set up code signing with a certificate we got from GoDaddy and it’s a little convoluted so I’ll document it here.

Creating and using a code signing certificate involves three passwords which we’ll call
* REQUEST_PASSWORD
* EXPORT_PASSWORD
* SIGNING_PASSWORD

Getting a Code Signing Certificate

We get our code signing certificates from GoDaddy.

Generating a Certificate Request

For this we’ll need our REQUEST_PASSWORD.
Following the instructions here we’ll end up with the files
* request.csr
* request.pfx

The pfx file has our private key embedded in it. These files need to be submitted to GoDaddy.

When the request is processed, GoDaddy will send us certificate files. These are randomly named, something like:
* SOMERANDOM-SHA2.pem
* SOMERANDOM-SHA2.spc

Extract the Private Key from the Certificate Request

We need the private key in the certificate request as a .key file. To do this we need to install OpenSSL. It can be installed as part of cygwin.

Generate the key via (where $ is the cygwin bash prompt):

$ openssl pkcs12 -in request.pfx -nocerts -out request.key.pem -nodes
Enter Import Password: REQUEST_PASSWORD 

The key will be in request.key.pem

Create a PVK File

Next we need to create a PVK file. For this we need pvk.exe.

Run:

PS C:\tmp\ssl> .\pvk.exe -in .\request.key.pem -topvk -strong -out cert.key.pvk
Enter Password: EXPORT_PASSWORD 
Verifying - Enter Password: EXPORT_PASSWORD 

This generates cert.key.pvk

Combined the PVK and SPC into a PFX

Installers are signed with a PFX file which is a combination of the key and certificate. For this we need pvk2pfx.exe.

Run:

pvk2pfx.exe -pvk cert.key.pvk -pi EXPORT_PASSWORD -spc SOMERANDOM-SHA2.spc -pfx codesign.pfx -po SIGNING_PASSWORD -f

This generates codesign.pfx. This, along with SIGNING_PASSWORD is what we need to sign the MSI. When the code signing certificate expires we’ll need to repeat the steps above.

Signing the Installer

Once we have the PFX and the signing password, we can sign the installer. For this we need signtool.exe.

The command to sign the installer is:

.\signtool.exe sign /f .\codesign.pfx /p SIGNING_PASSWORD /d "(some description)" /tr http://timestamp.digicert.com /v "PATH_TO_MSI"

There are other timeservers you can use.

Adding roles to IdentityServer4 OpenID Results

0

The IdentityServer4 Quickstart projects make it look so easy to add new custom properties to identity users. In their sample, they just add a “website” claim to a user and it shows up in the client. Easy!

new TestUser {
    SubjectId = "1",
    Username = "alice",
    Password = "password",

    Claims = new List<Claim> {
        new Claim("name", "Alice"),
        new Claim("website", "https://alice.com")
    }
},
User logged in
{
  "sid": "94ffdf2501942878493f60cc14291a83",
  "sub": "1",
  "auth_time": 1534264648,
  "idp": "local",
  "name": "Alice",
  "website": "https://alice.com",
  "amr": [
    "pwd"
  ]
}

And yet when I add a claim of my own, like “role” it’s nowhere to be seen.

    Claims = new List<Claim> {
        new Claim("name", "Alice"),
        new Claim("website", "https://alice.com"),
        new Claim("role", "admin")
    }
},

What’s going on???

Well, it turns out that “website” is already in the list of standard profile claims. Role is not. So what do we need to do to add it?

First of all, if we want the claim to show up in the JavaScript client and not in the API then we want to create a Identity Resource not an Api Resource.

public static IEnumerable<IdentityResource> GetIdentityResources() => new List<IdentityResource> {
    new IdentityResources.OpenId(),
    new IdentityResources.Profile(),
    new IdentityResource( "roles", new [] { "role" } )
};

And then have the client request it:

var config = {
    authority: "http://localhost:5000",
    client_id: "js",
    redirect_uri: "http://localhost:5003/callback.html",
    response_type: "id_token token",
    scope:"openid profile api1 roles",
    post_logout_redirect_uri : "http://localhost:5003/index.html",
};

Then it shows up in the client:

User logged in
{
  "sid": "94ffdf2501942878493f60cc14291a83",
  "sub": "1",
  "auth_time": 1534264648,
  "idp": "local",
  "name": "Alice",
  "website": "https://alice.com",
  "role": "Admin",
  "amr": [
    "pwd"
  ]
}

React and .NET Core WebAPI with F# Part 2: React

0

Following Part 1, we’re going to try to create an application with a pure React frontend and an F#-based WebApi backend.

I’m going to inject a bit of editorializing here: With client-side rendering, server-side rendering including ASP.NET MVC is dead. Why bother mixing cshtml and JavaScript when JavaScript alone will do it? This reduces the server-side code to business logic and data fetching returning JSON. Which further means that on the server we can use whatever technology is best suited to that job and switch it out as necessary. Once the back end is separate from the front end it’s easy enough to switch a C# WebAPI with F# or maybe with Erlang. We can use whatever is most efficient.

The F# Solution

We’re going to break with what we did in Part 1 and start a new F# solution. In Visual Studio, create a new solution called “ReactWebApiDemo” and under that a new F# WebAPI project called, again, “ReactWebApiDemo”. It will create a blank project with a controller called ValuesController.

If you run the application it will open with a blank screen. You can navigate to the API to see it return some JSON, e.g. http://localhost:54111/api/values

Enabling static web content

We want the WebApi side producing JSON while our main site runs on static HTML and JavaScript. The project should have a folder called “wwwroot”. This is where our static pages go. Start by putting an “index.html” in that folder, e.g.

<html>
<head>
    <meta charset="utf-8">
    <title>Test</title>
</head>
<body>
    <h1>Test</h1>
</body>
</html>

Now if you run the solution and load the root URL, e.g. “http://localhost:54111/&#8221;, what do you see? The answer: nothing. We have to both enable static files (to get it to read from the folder) and default paths (to get it to recognize that / should load /index.html). To so this, modify Startup.fs and add change the Configure method to do this:

    member this.Configure(app: IApplicationBuilder, env: IHostingEnvironment) =
        app
            .UseDefaultFiles()
            .UseStaticFiles()
            .UseMvc() |> ignore

Adding our TypeScript, Less, React, etc. back in

If you have the files from part 1, copy
* .babelrc
* package.json
* package-lock.json
* tsconfig.json
* webpack.config.json
into the project folder and run

npm install

This will restore the modules.

Our files are going to wwwroot now, so first let’s establish this layout:
* wwwroot/
* wwwroot/components – Our React components
* wwwroot/css – Our CSS files
* wwwroot/js – Our scripts
* wwwroot/dist – Our bundled output

We need to change webpack.config.js with the entry and the output:

    // Entry: a.k.a. "Entry Point", the JS file that will used to build the JavaScript dependency graph
    // If not specified, src/index.js is the default.
    entry: "./wwwroot/js/index.js",
    // Output: Where we'll output the build files.
    output: {
        // Path: The directory to which we'll write transformed files
        path: path.resolve(__dirname, 'wwwroot/dist'),
        // Filename: The name to which we'll write our bundled JavaScript.
        // If not specified, dist/main.js is the default.
        filename: 'main.js'
    },

Running webpack on build

One more thing we can do is make webpack run automatically every time we do a build. Edit your .fsproj file and add the following:

    <Target Name="WebpackDebug" BeforeTargets="Build" Condition=" '$(Configuration)' == 'Debug'">
        <Message Importance="high" Text="Performing webpack build (Debug)..." />
        <Exec Command="npm run debug" />
    </Target>

    <Target Name="WebpackRelease" BeforeTargets="Build" Condition=" '$(Configuration)' == 'Release'">
        <Message Importance="high" Text="Performing webpack build (Release)..." />
        <Exec Command="npm run release" />
    </Target>

Stopping Visual Studio TypeScript compilation

By default, Visual Studio is going to try to compile our typescript and put it in its own “dist” folder. We’re using webpack so we don’t need it. To disable it, edit .fsproj and add:

    <PropertyGroup>
        <TypeScriptCompileBlocked>true</TypeScriptCompileBlocked>
    </PropertyGroup>

Building and Running

Let’s put some code in input.js and run the project to make sure it works.

import style from '../css/site.less'

function test() {
    console.log("test");
}

window.onload = test;

Now we run it from Visual Studio, the browser opens, and we get… a page with script and style errors.

Content Security Policy

Here we’re going to run into a little problem with webpack and the application’s Content Security Polity. Using webpack’s “import style from ‘blah.css'” is an unsafe style import. Firefox or Chrome will show you a message like “The page’s settings blocked the loading of a resource at self (“style-src”).” and they’re right. A default Content Security Policy defines where you can load scripts and styles from and the default is “script-src ‘self’; style-src ‘self’;”. If we want to load CSS like this we need to override the security policy in .NET Core. I don’t actually recommend this, but until I find a better way, you can modify the Configure method like so:

    member this.Configure(app: IApplicationBuilder, env: IHostingEnvironment) =
        let staticFileOptions = StaticFileOptions()
        staticFileOptions.OnPrepareResponse <- fun ( context ) ->
            context.Context.Response.Headers.Add(
                        "Content-Security-Policy",
                        StringValues(
                            "script-src 'self'; " +
                            "style-src 'self' 'unsafe-inline'; " +
                            //"style-src 'self'; " +
                            "img-src 'self'" 
                        ) );
        app
            .UseDefaultFiles()
            .UseStaticFiles( staticFileOptions )
            .UseMvc() 
            |> ignore

This applies the content security policy to static files. To apply it to all files, use:

        app
            .UseDefaultFiles()
            .UseStaticFiles()
            .UseMvc() 
            .Use( fun ( context : HttpContext ) ( next : Func<Task> ) ->
                async {
                    context.Response.Headers.Add(
                        "Content-Security-Policy",
                        StringValues(
                            "script-src 'self'; " +
                            "style-src 'self' 'unsafe-inline'; " +
                            //"style-src 'self'; " +
                            "img-src 'self'" 
                        ) );
                    return! next.Invoke() |> Async.AwaitTask
                } |> Async.StartAsTask :> Task
            )
            |> ignore

At the time of writing this, webpack has the beginning of the concept of adding a nonce but they don’t seem to be doing it right in that it appears to be a static value rather than a per-request value making the application less secure.

Now with that temporarily solved…

React Router

We’re going to use react-router for page navigation. Add it with npm.

npm install react-router react-router-dom
  • react-router – The core react-router library.
  • react-router-dom – The DOM bindings (as opposed to React Native)
    And for TypeScript compatibility:
npm install @types/react-router @types/react-router-dom
  • @types/react-router – TypeScript types for react-router
  • @types/react-router-dom – TypeScript types for react-router-dom

Let’s follow react-router’s basic example. Create a file called “App.jsx” and add the following code (taken verbatim from the given link):

import React from 'react'
import {
    BrowserRouter as Router,
    Route,
    Link
} from 'react-router-dom'

const Home = () => (
    <div>
        <h2>Home</h2>
    </div>
);

const About = () => (
    <div>
        <h2>About</h2>
    </div>
);

const Topic = ({ match }) => (
    <div>
        <h3>{match.params.topicId}</h3>
    </div>
);

const Topics = ({ match }) => (
    <div>
        <h2>Topics</h2>
        <ul>
            <li>

                    Rendering with React

            </li>
            <li>

                    Components

            </li>
            <li>

                    Props v. State

            </li>
        </ul>


         (
            <h3>Please select a topic.</h3>
        )} />
    </div>
);

const BasicExample = () => (

        <div>
            <ul>
                <li>Home</li>
                <li>About</li>
                <li>Topics</li>
            </ul>

            <hr />




        </div>

);

export default BasicExample

Now change index.js to render our React routes:

import style from '../css/site.less'
import React from 'react'
import ReactDOM from 'react-dom'
import BasicExample from './App'

function test() {
    console.log("test");
    ReactDOM.render(BasicExample(), document.getElementById('root'));
}

window.onload = test;

Since we’re rendering in the element “root” we need to define that in index.html:

<div id="root">
   <h1>Test</h1>        
</div>

Run the page and you should see the example in action.

URL handling

Note that react-router will update the URL as you click on the various links. But you’ll also notice that if you bookmark one of the links, e.g. “/about” and try to go back to it, it’ll fail.
What we need to do is route all our URLs to the React page. We can do this with a URL rewrite rule in web.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.webServer>
        <rewrite>
            <rules>
                <rule name="React Routes" stopProcessing="true">
                    <match url="(.*)" />
                    <conditions logicalGrouping="MatchAll">
                        <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
                        <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
                        <add input="{REQUEST_URI}" pattern="^/(api|scripts)" negate="true" />
                    </conditions>
                    <action type="Rewrite" url="/" />
                </rule>
            </rules>
        </rewrite>
    </system.webServer>
</configuration>

Basically, everything that isn’t /api will get routed back to the React controller. This isn’t actually ideal – it also redirects scripts and images. We have a couple of options:
* We can ensure all our router paths have some distinct prefix and route only those. Imagine we put all our route paths under /r/ then we could do: (note "^" means "start of the path").
* We could exclude other paths, like /images, the same way we exclude /api. e.g.
.

Using TSX instead of JSX

If we want to use TypeScript instead of JavaScript we need to convert App.jsx to App.tsx. The converted example looks like this:

import * as React from 'react'
import * as ReactRouter from 'react-router'
import {
    BrowserRouter as Router,
    Route,
    Link

} from 'react-router-dom'

const Home = () => (
    <div>
        <h2>Home TS</h2>
    </div>
);

const About = () => (
    <div>
        <h2>About TS</h2>
    </div>
);

const Topic = ({ match } : { match: ReactRouter.match } ) => (
    <div>
        <h3>{match.params.topicId}</h3>
    </div>
);

const Topics = ({ match }: { match: ReactRouter.match }) => (
    <div>
        <h2>Topics</h2>
        <ul>
            <li>

                    Rendering with React

            </li>
            <li>

                    Components

            </li>
            <li>

                    Props v. State

            </li>
        </ul>


         (
            <h3>Please select a topic.</h3>
        )} />
    </div>
);

const BasicExample = () => (

        <div>
            <ul>
                <li>Home TS</li>
                <li>About TS</li>
                <li>Topics TS</li>
            </ul>

            <hr />




        </div>

);

export default BasicExample

The main change is to convert untyped parameters like Topic = ({ match }) to typed parameters like Topic = ({ match } : { match: ReactRouter.match } ) where match<P> is a type found in @types/react-router.

Fetching data from the API

If you created the default F# project, you will already have a ValuesController that supports a Get returning an array:

[<Route("api/[controller]")>]
type ValuesController () =
    inherit Controller()

    [<HttpGet>]
    member this.Get() =
        [|"value1"; "value2"; "value A"; "value B"|]

The usual way, I’m told, for a React component to load values via AJAX is in componentDidMount. Let’s change our topics to fetch data from the API. It can’t be an arrow function anymore.

class Topics extends React.Component<any, { values: any[] }>{
    constructor(props: any) {
        super(props);
        console.log("Topics.ctor");
        this.state = { values: [] };
    }
    public componentDidMount() {
        console.log("componentDidMount");
        fetch("/api/values")
            .then((response) => {
                console.log(" got response " + response);
                return response.json();
            })
            .then((json) => {
                console.log("got json " + json);
                this.setState({ values: json });
            })
            ;

    }
    public render() {
        if (null === this.state) return null;
        return <div>
            <h2>Topics</h2>
            <ul>
                {this.state.values.map(function (value: any, index: number) {
                    return <li>{value}</li>;
                })}
            </ul>
        </div>
            ;
    }
}

And that’s it. When we load the /topic URL we’ll see the list from our API.

React and .NET Core WebAPI with F# Part 1: React

0

I’m going to go through a step-by-step guide to getting React and .NET Core WebAPI working together. In this guide I’m going to try to document everything so there are no hidden steps and very little assumed knowledge.

In this first part, I’m just going to get a vanilla solution working with TypeScript and React.

Create the solution and project directories

We’re going to have the layout of a standard Visual Studio solution here, so create a folder for the solution, e.g. ReactWebApiDemo and then under that a folder for our web project, e.g. ReactWebApiDemo again.

Install npm

Most modern web projects use the Node.js package manager, npm, so the first step is to install it from either of the provided links. I’ll note that yarn is a possible alternative to npm and you’re welcome to try it instead, though the usage will be slightly different. At the time of writing, the npm version was 5.6.0.

First we need to initialize our project with

npm init

This gives the following:

PS C:\Projects\ReactWebApiDemo\ReactWebApiDemo> npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help json` for definitive documentation on these fields
and exactly what they do.

Use `npm install ` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (reactwebapidemo)
version: (1.0.0)
description: React WebAPI Demo
entry point: (index.js)
test command:
git repository:
keywords:
author:
license: (ISC)
About to write to C:\Projects\ReactWebApiDemo\ReactWebApiDemo\package.json:

{
  "name": "reactwebapidemo",
  "version": "1.0.0",
  "description": "React WebAPI Demo",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}


Is this ok? (yes) yes

After that, the syntax for installing packages that will end up in production code (like React) is:

npm install {module name}

or, for libraries that are used in development but will not end up in production:

npm install --save-dev {module name}

"–save-dev" can be replaced by "-D", e.g.

npm install -D {module name}

Install and Configure Webpack

The first thing we're going to set up is webpack. Webpack is a modular utility for minimizing and transforming our other files to make them production ready. We'll use it to:
* Convert TypeScript (.ts) files to JavaScript (.js) files
* Convert React TypeScript (.tsx) files to JavaScript (.js) files
* Less CSS (.less) files to CSS (.css) files.
* Pack our JavaScript and CSS files together with those of the third-party libraries we're using.
* Minimize and compress our JavaScript and CSS files.

The first step is to install webpack and its command-line interface, webpack-cli, which we can do with npm:

PS C:\Projects\ReactWebApiDemo\ReactWebApiDemo> npm install --save-dev webpack webpack-cli
[..................] - fetchMetadata: sill resolveWithNewModule webpack@4.10.2 checking installable status
  • webpack – The package and minimization utility.
  • webpack-cli – The webpack command line interface.

Webpack runs off a configuration file, webpack.config.js. This is the barest of configuration files to start with:

// Let us use the core webpack module as a library
const webpack = require( 'webpack' );
// Let us use the built-in webpack path module as a library
const path = require( 'path' );


module.exports = {
    // Entry: a.k.a. "Entry Point", the JS file that will used to build the JavaScript dependency graph
    // If not specified, src/index.js is the default.
    entry: "./src/index.js",
    // Output: Where we'll output the build files.
    output: {
        // Path: The directory to which we'll write transformed files
        path: path.resolve( __dirname, 'dist' ),
        // Filename: The name to which we'll write our bundled JavaScript.
        // If not specified, dist/main.js is the default.
        filename: 'main.js'
    },
    // The processing mode.  Accepted alues are "development", "production", or "none".
    mode: 'production'
}

You'll note that it needs an input file at "src/input.js" and an output directory at "dist", so create those in the web project.
After that we can run it with:

node .\node_modules\webpack\bin\webpack.js

e.g.

PS C:\Projects\ReactWebApiDemo\ReactWebApiDemo> node .\node_modules\webpack\bin\webpack.js
Hash: c4097b5edb272ec4b73c
Version: webpack 4.10.2
Time: 125ms
Built at: 2018-06-03 20:48:45
    Asset       Size  Chunks             Chunk Names
bundle.js  930 bytes       0  [emitted]  main
[0] ./src/index.js 0 bytes {0} [built]

We can make this easier on ourselves by setting this command up in the "scripts" section of package.json. E.g.

  "scripts": {
    "debug": "node ./node_modules/webpack/bin/webpack.js"
  },

Then:

npm run-script debug

or

npm run debug

We can make this even easier by putting the mode in the scripts and simplifying the scripts to just call "webpack" as in:

  "scripts": {
    "debug": "webpack --mode none",
    "dev": "webpack --mode development",
    "release": "webpack --mode production"
  },

We're going to make sure we have this working, so for now, change input.js to:

function test() {
}

Run npm run-script debug. It should create “dist\bundle.js”. Open it up. You should see some webpack overhead stuff and our test() method at the bottom.

One last change we can make is to let webpack know about common extensions so we can import files as just “import ‘./blah'” and not “import ‘./blah.js'”.
Add the following after the “module” section:

    resolve: {
        extensions: ['.js', '.ts', '.jsx', '.tsx', '.json']
    }

Setting up Less CSS

Run

npm install --save-dev less less-loader css-loader style-loader
  • less – The Less CSS library.
  • less-loader – The webpack module for Less-to-CSS conversion.
  • css-loader – The webpack module that allows us to import CSS into JavaScript.
  • style-loader – The webpack module that, with css-loader, lets us import styles into JavaScript.

Following the instructions on the less-loader site, we add this to webpack.config.js:

    // Define our modules here
    module : {
        rules: [
            { 
                test: /\.less$/, // Match all *.less files
                use: [{
                    loader: 'style-loader' // creates style nodes from JS strings
                  }, {
                    loader: 'css-loader' // translates CSS into CommonJS
                  }, {
                    loader: 'less-loader' // compiles Less to CSS
                  }]
            }
        ]
    }

Add a less file to src, e.g. “site.less”.

body {
    font-family: 'Times New Roman', Times, serif;
}

Have index.js import the style from the file, e.g.

import style from './site.less'

If you run webpack again, you’ll see bundle.js get updated.
You can test that the style is applied by creating a small HTML file, e.g.

<html>
    <head>
        <meta charset="utf-8">
        <title>Test</title>
        <a href="http://../dist/bundle.js">http://../dist/bundle.js</a>
    </head>
    <body>
        <h1>Test</h1>
    </body>
</html>

If you load the file in a browser, you’ll see the CSS is used.

Setting up TypeScript

I’m basically following the instructions here except we’re going to use awesome-typescript-loader instead. Once again, we start by installing the prerequisites. Run

npm install --save-dev typescript awesome-typescript-loader

TypeScript needs its own config file, tsconfig.json:

{
    "compilerOptions": {
        "outDir": "./dist/",
        "noImplicitAny": true,
        "module": "es6",
        "moduleResolution": "node",
        "target": "es5",
        "jsx": "react",
        "allowJs": true
    }
}

Note “moduleResolution”:”node”. This will allow us to use “import from ‘blah'” to import node modules.

In webpack.config.js we need the module definition:

            // Typescript
            {
                test: /\.tsx?$/, // Match *.ts and *.tsx
                use: 'awesome-typescript-loader', // Converts TypeScript to JavaScript
                exclude: /node_modules/ // Don't look in NPM's node_modules
            }

We can test it by putting a TypeScript file in the src folder. This is the TypeScript straight from the “TypeScript in 5 minutes” tutorial:

interface Person {
    firstName: string;
    lastName: string;
}

export function greeter(person: Person) {
    return "Hello, " + person.firstName + " " + person.lastName;
}

Then reference it in index.js:

import greeter from './test.ts';

If you run webpack again, you’ll see the greeter code added to bundle.js.

Setting up React

Again, install react and react-dom with npm. No “–save-dev” this time – these are going in production!

npm install react react-dom

We’re also going to want to use babel to let us use EC6+ features in EC5 browsers. Start by installing babel (Dev only):

npm install --save-dev babel-core babel-loader 

We then install babel presets to tell it the plugins to set up.

npm install --save-dev babel-preset-env babel-preset-react

We need to set the babel loader up in our webpack.config.js:

            // Babel
            {
                test: /\.jsx?$/, // Match *.js and *.jsx
                use: 'babel-loader', // Converts ES2015+ JavaScript to browser-compatible JS
                exclude: /node_modules/ // Don't look in NPM's node_modules
            }

And babel needs its own configuration file, .babelrc, to tell it about the plugins:

{
    "presets": ["env", "react"]
}

Let’s test this out. We’ll change our index.js to include the React code given in the React tutorial:

import style from './site.less';
import React from "react";
import ReactDOM from "react-dom";

class ShoppingList extends React.Component {
    render() {
        return (
            <div>
                <h1>Shopping List for {this.props.name}</h1>
                <ul>
                    <li>Instagram</li>
                    <li>WhatsApp</li>
                    <li>Oculus</li>
                </ul>
            </div>
        );
    }
}

function renderShoppingList() {
    ReactDOM.render(
        <ShoppingList />,
        document.getElementById('shopping-list')
    );
}

window.onload = renderShoppingList;

If we load index.html we’ll see the React component render.

Let’s try loading files from a JSX. We’ll save this as ShoppingList.jsx:

import React from "react";
import ReactDOM from "react-dom";

class ShoppingList extends React.Component {
    render() {
      return (
        <div>
          <h1>Shopping List for {this.props.name}</h1>
          <ul>
            <li>A</li>
            <li>B</li>
            <li>C</li>
          </ul>
        </div>
      );
    }
  }

  module.exports = {
    renderShoppingList : function() {
      console.log( "renderShoppingList" );
      ReactDOM.render(
          <ShoppingList />,
          document.getElementById('shopping-list')
        );
    }
  }

Then modify our index.js like so:

import style from './site.less';
import React from "react";
import ReactDOM from "react-dom";

var shoppingList = require( "./ShoppingList" );

function test() {
    console.log( "test" );
    shoppingList.renderShoppingList();
}
window.onload = test;

And put the ID it requires in index.html:

    <body>
        <h1>Test</h1>
        <div id="shopping-list"></div>
    </body>

Now if we load our index.html in the browser we’ll see our React component.

Finally, let’s see if we can get this working with a TSX file.

For TypeScript to be able to import node modules properly we need to import the TypeScript type packages.

npm install --save-dev @types/react @types/react-dom

If we fail to do this we’ll get errors like “TS7016: Could not find a declaration file for module ‘react-dom'”.

Let’s make another component called AnotherComponent.tsx with the following code:

import * as React from 'react';
import * as ReactDOM from 'react-dom';

class AnotherComponent extends React.Component {
    public render() {
        return (
            <div>
                <h1>TSX</h1>
            </div>
        );
    }
}


export function renderAnotherComponent() {
    console.log("renderAnotherComponent");
    ReactDOM.render(
        <AnotherComponent />,
        document.getElementById('another-component')
    );
}

Now change index.js to call it:

<br />var shoppingList = require( "./ShoppingList" );
var anotherComponent = require( "./AnotherComponent" );

function test() {
    console.log( "test" );
    shoppingList.renderShoppingList();
    anotherComponent.renderAnotherComponent();
}
window.onload = test;

And put the ID it requires in index.html:

    <body>
        <h1>Test</h1>
        <div id="shopping-list"></div>
        <div id="another-component"></div>
    </body>

Now if we load index.html in a browser we’ll see our JSX component and our TSX component.

On to Part 2.

VBA for Engineers

0

I make oil and gas software and work with engineers a lot. I am one myself, by education. Engineers like to use Excel and if they remember the little programming they took in university, they like to extend what Excel can do by using Visual Basic for Applications, or VBA. VBA is terrible. It’s based on VB6 which is also terrible. Together, VB6, VBA, and VB.Net make up three of the top five most dreaded languages in the most recent Stack Overflow Developer Survey.

The saying goes that when the only tool you have is a hammer, every problem looks like a nail. Engineers’ tool is VBA so you find them using it where it’s not appropriate or useful. Moreover, they’re amateur programmers so to a professional software developer like me, opening up a macro-enabled Excel file is like a carpenter walking up to a woodworking project and seeing a screw that’s been pounded double surrounded by a bunch of circular indentations.

They still ask me for help with their Visual Basic problems and rather than turn them away I’ve finally said that I’m glad to help. After all, the first step in getting help with a VBA problem is admitting you have a VBA problem. To that end, I’ve tried to establish a 12-step program based on the traditional 12-step programs like AA. VBAA, if you will.

AA VBAA
We admitted we were powerless over alcohol—that our lives had become unmanageable. We admitted we were powerless over VBA—that our Excel-based workflow had become unmanageable.
Came to believe that a Power greater than ourselves could restore us to sanity. Came to believe that a department greater than ourselves could restore us to sanity.
Made a decision to turn our will and our lives over to the care of God as we understood Him. Made a decision to turn our will and our lives over to the care of IT as we understood them.
Made a searching and fearless moral inventory of ourselves. Made a searching and fearless moral inventory of our Excel-based workflow.
Admitted to God, to ourselves, and to another human being the exact nature of our wrongs. Admitted to IT, to ourselves, and to another human being the exact nature of our wrongs.
Were entirely ready to have God remove all these defects of character. Were entirely ready to have IT remove all these defects of character.
Humbly asked Him to remove our shortcomings. Humbly asked IT to remove our VBA macros and modules.
Made a list of all persons we had harmed, and became willing to make amends to them all. Made a list of all persons we had harmed, and became willing to make amends to them all.
Made direct amends to such people wherever possible, except when to do so would injure them or others. Made direct amends to such people wherever possible, except when to do so would injure them or others.
Continued to take personal inventory, and when we were wrong, promptly admitted it. Continued to take personal inventory, and when we were wrong, promptly admitted it.
Sought through prayer and meditation to improve our conscious contact with God as we understood Him, praying only for knowledge of His will for us and the power to carry that out. Sought through prayer and meditation to improve our conscious contact with IT, praying only for knowledge of better programming languages for us and the power to carry that out.
Having had a spiritual awakening as the result of these steps, we tried to carry this message to alcoholics, and to practice these principles in all our affairs. Having had a spiritual awakening as the result of these steps, we tried to carry this message to other Engineers, and to practice these principles in all our affairs.

I encourage all engineers suffering from a VBA problem to seek a sponsor in their IT’s software development department.

Mocking delegates with Moq

3

Using Delegates

In C#, a delegate is a function signature that can be passed around as a parameter. This is a delegate that takes a couple of parameters and returns a value:

public delegate int DoSomething( double x, string y );

This is a method that puts it to work with Invoke:

    public int CallDelegate( DoSomething doSomething, double x, string y ) {
        return doSomething?.Invoke( x, y ) ?? 0;
    }

You don’t need to use Invoke, you can use it directly via:

        return doSomething( x, y );

but Invoke is nice because you can guard against nullables.

Mocking Delegates

When unit testing with Moq, you may find yourself wanting to mock a delegate or to verify that it was called. It’s straightforward, just make sure you mock the method itself and not Invoke:

        [TestMethod]
        public void TestMethod() {

            var mockDoSomething = new Mock<MyClass.DoSomething>();
            mockDoSomething.Setup( _ => _( It.IsAny<double>(), It.IsAny<string>() ) ).Returns( 5 );
            // NOT
            // mockDoSomething.Setup( _ => _.Invoke( It.IsAny<double>(), It.IsAny<string>() ) ).Returns( 5 );
            var subject = new MyClass();
            var result = subject.CallDelegate( mockDoSomething.Object, 1.1, "x" );
            Assert.AreEqual( 5, result );
            mockDoSomething.Verify( _ => _( 1.1, "x" ), Times.Once );
            mockDoSomething.Verify( _ => _.Invoke( 1.1, "x" ), Times.Once );            
        }
    }

If you try to mock Invoke itself, you’ll get an error like:

System.InvalidCastException: Unable to cast object of type ‘System.Linq.Expressions.InstanceMethodCallExpressionN’ to type ‘System.Linq.Expressions.InvocationExpression’.