Logging and Application Insights with ASP.NET core

Obviously when you’re running an ASP.NET core application in Azure, we’re going to want the ability to capture logs to Azure. This usually means logging to Application Insights.

Adding Logging

Let’s start out by just looking at what we need to do to enable logging from ASP.NET core.

Logging is included by default in the way of the ILogger interface (ILogger<T>), hence we can inject into our code like this (this example uses minimal API)

app.MapGet("/test", (ILogger<Program> logger) =>
{
    logger.LogCritical("Critical Log");
    logger.LogDebug("Debug Log");
    logger.LogError("Error Log");
    logger.LogInformation("Information Log");
    logger.LogTrace("Trace Log");
    logger.LogWarning("Warning Log");
})
.WithName("Test")
.WithOpenApi();

To enable/filter logging we have something like the following within the appsettings.json file

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
}

The LogLevel, Default section sets the minimum logging level for all categories. So for example a Default of Information means only logging of Information level and above (i.e. Warning, Error and Critical) are captured.

The Microsoft.AspNetCore is a category specific logging in that it logs Microsoft.AspNetCore namespace logging using the supplied log level. Because we can configure by namespace we can also use categories such as Microsoft, System, Microsoft.Hosting.Lifetime. We can also do the same with our code, i.e. MyApp.Controllers, so this allows us to really start to tailor different sections of our application an what gets captured in the logs.

Logging Levels

There various logging levels are as follows

  • LogLevel.Trace: The most detailed level, use for debugging and tracing (useful for entering/existing methods and logging variables).
  • LogLevel.Debug: Detailed but less so than Trace (useful for debugging and workflow logging).
  • LogLevel.Information: Information messages at a higher level than the previous two levels (useful for logging steps of processing code).
  • LogLevel.Warning: Indicates potentially problems that do not warrant error level logging.
  • LogLevel.Error: Use for logging errors and exceptions and other failures.
  • LogLevel.Critical: Critical issues that may cause an application to fail, such as those that might crash your application. Could also include things like missing connection strings etc.
  • LogLevel.None: Essentially disables logging

Application Insights

Once you’ve created an Azure resource group and/or Application Insights service, you’ll be able to copy the connection string to connect to Application Insights from your application.

Before we can use Application Insights in our application we’ll need to

  • Add the nuget package Microsoft.ApplicationInsights.AspNetCore to our project
  • Add the ApplicationInsights section to the appsettings.json file, something this
    "ApplicationInsights": {
      "InstrumentationKey": "InstrumentationKey=xxxxxx",
      "LogLevel": {
        "Default": "Information",
        "Microsoft": "Warning"
      }
    },
    

    We can obviously set the InstrumentKey in code if preferred, but the LogLevel is specific to what is captured within Application Insights

  • Add the following to the Program.cs file below CreateBuilder

    var configuration = builder.Configuration;
    
    builder.Services.AddApplicationInsightsTelemetry(options =>
    {
        options.ConnectionString = configuration["ApplicationInsights:InstrumentationKey"];
    });
    

Logging Providers in code

We can also add logging via code, so for example after the CreateBuilder line in Program.cs we might have

builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Logging.AddDebug(); 

In the above we start by clearing all currently logging providers, the we add a provider for logging to console and debug. The appsettings.json log levels are still relevant to which logs we wish to capture.

Using secrets in your appsettings.json via Visual Studio 2022 and dotnet CLI

You’ve got yourself an appsettings.json file for your ASP.NET core application and you’re using sensitive data, such as passwords or other secrets. Now you obviously don’t want to commit those secrets to source control, so you’re not going to want to store these values in your appsettings.json file.

There’s several ways to achieve this, one of those is to use Visual Studio 2022 “Manage User Secrets” option which is on the context menu off of your project file. There’s also the ability to use to dotnet CLI for this as we’ll see later.

This context menu option will create a secrets.json in %APPDATA%\Microsoft\UserSecrets\{Guid}. The GUID is stored within your .csproj in a PropertyGroup like this

<UserSecretsId>0e6abf63-deda-47fc-9a80-1cb56abaeead</UserSecretsId>

So the secrets file can be used like this

{
  "ConnectionStrings:DefaultConnection": "my-secret"
}

and this will map to your appsettings.json, that might look like this

{
  "ConnectionStrings": {
    "DefaultConnection": "not set"
  },
}

Now we can access the configuration in the usual way, for example

builder.Configuration.AddUserSecrets<Program>();

var app = builder.Build();
var connectionString = app.Configuration.GetSection("ConnectionStrings:DefaultConnection");
var defaultConnection = connectionString.Value;

When somebody else clones your repository you’ll need to recreate the secrets file, we could use _dotnet user-secrets_ for example

dotnet user-secrets set "ConnectionStrings:DefaultConnection" "YourConnectionString"

and you can list the secrets using

dotnet user-secrets list

Disable the Kestrel server header

We generally don’t want to expose information about the server we’re running our ASP.NET core application on.

In the case of Kestrel we can disable the server header using

var builder = WebApplication.CreateBuilder(args); 

builder.WebHost.UseKestrel(options => 
   options.AddServerHeader = false);

Protocols and Behaviours in Elixir

Protocols and Behaviours in Elixir are similar to interfaces in languages such as C#, Java etc.

Protocols can be thought of as interfaces for data whereas behaviours are like interfaces for modules, let’s see what this really means…

Protocols

A protocol is available for a data type, so let’s assuming we want a toString function on several data types but we obviously cannot cover all possible types that may be created in the future, i.e a Person struct or the likes. We can define a protocol which can be applied to data types, like this…

Let’s start by define the protocol

defprotocol Utils do
  @spec toString(t) ::String.t()
  def toString(value)
end

Basically we’re declaring the specification for the protocol using the @spec annotation. This defines the inputs and outputs, taking any params the after the :: is the return type. Next we define the function.

At this point we have now implementations, so let’s create a couple of implementations for a couple of the standard types, String and Integer

defimpl Utils, for: String  do
  def toString(value), do: "String: #{value}"
end

defimpl Utils, for: Integer  do
  def toString(value), do: "Integer: #{value}"
end

The for is followed by the data type supported by this implementation. So as you can see, we have a couple of simple implementation, but where protocols become more important is that we can now define the toString function on other types, let’s assume we have the Person struct from a previous post

defmodule Person do
  @enforce_keys [:firstName, :lastName]
  defstruct [:age, :firstName, :lastName]

  def create() do
    %Person{ firstName: "Scooby", lastName: "Doo", age: 30 }
  end
end

and we want to give it a toString function, we would simply define a new implementation of the protocol for the Person data type, like this

defimpl Utils, for: Person  do
  def toString(value), do: "Person: #{value.firstName} #{value.lastName}"
end

Now from iex or your code you can do sometihing like this

scooby = Parson.create()
Utils.toString(scooby)

and you’ve got toString working with the Person type.

Behaviours

Behaviours are again similar to interfaces but are used to define what a module is expected to implement. Let’s stick with the idea of a toString function which just outputs some information about the module that’s implementing it, but this time we’re expecting a module to implement this function, so we declare the behaviour as follows

defmodule UtilBehaviour do
  @callback toString() :: String.t()
end

We use the @callback annotation to declare the expected function(s) and @macrocallback for macros. As per the protocol we give the signature of the function followed by :: and the expected return type.

Now to implement this, let’s again go to our Person struct (remember this version of toString is just going to output some predefined string that represents the module)

defmodule Person do
  @behaviour UtilBehaviour

  @enforce_keys [:firstName, :lastName]
  defstruct [:age, :firstName, :lastName]

  def create() do
    %Person{ firstName: "Scooby", lastName: "Doo", age: 30 }
  end

  def toString() do
    "This is a Person module/struct"
  end
end

Now our module implements the behaviour and using Person.toString() outputs “This is a Person module/struct”.

We can also use the @impl annotation to ensure that you explicitly define the behaviour being implement like this

@impl UtilBehaviour
def toString() do
  "This is a Person module/struct"
end

This @impl annotation tells the compiler explicitly what you’re implementing, this is just an aid to development by making it clear what’s implementing what. If you use @impl once you have to use it on every behaviour.

CSS units of measurement

CSS offers several types of units of measurement as part of your web design, i.e. for font sizes, spacing etc.

px (pixels)

If you’ve come from any other UI development you’ll probably be used to using pixels to define window sizes etc. Pixels allow us to specify positioning and sizes specifically, however these are not scalable, i.e. a 10px * 10px button might look fine on a lower resolution monitor but for a high resolution look tiny.

1px = 1/96th of an inch

cm

Centimetres, and absolute measurement, 1cm equals 37.8px, which equals 25.2/64in

mm

Millimeters, 1mm = 1/10th of 1 cm

Q

Quarter-millimeteres, 1Q = 1/40th of a cm

in

Inches, 1 in = 2.54cm = 96px

pc

Picas, 1pc = 1/6th of an inch

pt

Points, 1pc = 1/72nd of an inch

em

This is a unit which is relative to the font size of the parent element. So for example 3em will be twice the size of the parent element’s font size. In other words let’s assume the font size is 32px then 3em would be 3 * 32px, i.e. 96px.

rem (root em)

This is similar (as the name suggests) to em, but it’s relative to the root HTML font size. Otherwise we can calculate things as per em, but using the root font size not the parent element.

Collections in Elixir

Disclaimer: I’m going through some old posts that were in draft and publishing one’s which look relatively complete in case they’re of use: This post may not be 100% complete but does give a good overview of Elixir collections.

Lists in Elixir are implemented as linked lists which handle handle different types.

[3, "Three" :three]

Prepending to a list is faster than appending

list = [3, "Three" :three]
["pre" | list]

Appending

list = [3, "Three" :three]
list ++ ["post"]

List concat

[3, "Three" :three] ++ ["four", :4, 4]

List subtraction,

[2] -- [2.0]

Head and tail

hd [3, "Three" :three]
tl [3, "Three" :three]

Pattern matching

We can split the head an tail using Z

[head | tail] = [3.14, :pie, "Apple"]

The equivalent of a dictionary known as keyword lists in Elixir

[foo: "bar", hello: "world"]
[{:foo, "bar"}, {:hello, "world"}]

Keys can be atoms, keys are ordered and do not have to be unique

Maps

Unlike keyword lists they allows keys of any type and are unordered the syntax for a ,ap is %{}

map = %{:foo => "bar", "hello" => :world}

SQL Server and IDENTITY_INSERT

I’m creating an SQL script to seed my SQL Server database, the tables include a primary key with autoincrement set (i.e. IDENTITY(1,1)). Inserting data within supplying a primary key works fine, but if you decide to seed the primary key data as well then I need to make a couple of changes to my SQL script.

Why would I want to seed the primary key if it’s autoincrementing, you may ask. The reason is that I intend to also seed some of the relationship data as well and this ofcourse means I already have the table’s primary key value (because I set it) and thus can easily create the relationship links.

What you need to do is wrap your INSERTs for a specific table in, so for example below we have a Country table and the primary key is Id.

SET IDENTITY_INSERT [dbo].[Country] ON

INSERT INTO [dbo].[Country] ([Id], [Name]) 
VALUES (1, 'Australia')

SET IDENTITY_INSERT [dbo].[Country] OFF

Adding Playwright to a React web app.

Playwright is an automation testing framework for the web. Let’s add it to our React app. and demonstrate how to use it

Installation

  • Install Playwright
    yarn create playwright
    
  • You’ll be asked where to put your end-to-end tests, default is e2e so let’s stick with that
  • Next, you’ll be asked whether to add a GitHub actions workflow, default is N< but I want them, so selected Y
  • Now you’re asked whether to install Playwright browsers, default is Y so let’s stick with that
  • Now Playwright is downloaded and installed

Writing Tests

Within the folder we set for our tests, I used the default e2e we can start adding our *.spec.ts test files, for example here’s a simple example test just to check the title on my web app.

import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  // obviously needs changing to your deployed web app, but
  // fine for local testing
  await page.goto('http://localhost:3000/');
});

test('Ensure title is as expected', async ({ page }) => {

  await expect(page).toHaveTitle(/My Web App/);
  await page.getByText('End Sat Dec 31 2022').click();
});

In the above we simple create a test and using playwright we automate testing of the web app.

Now to run this, as I’m using React, add to package.json the following to the scripts section

"playwright_test": "playwright test",
"playwright_report": "playwright show-report",

Now we can run yarn playwright_test to run the tests within e2e or whatever your test folder was named.

Mix and project dependencies in Elixir

Like many other languages, there’s a lot of third party packages and shared code for the Elixir language.

If we use mix to create a new project you’ll get a mix.exs script file generated. Let’s take a look at one created for one of my projects

defmodule TestLang.MixProject do
  use Mix.Project

  def project do
    [
      app: :test_lang,
      version: "0.1.0",
      elixir: "~> 1.17",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  # Run "mix help compile.app" to learn about applications.
  def application do
    [
      extra_applications: [:logger]
    ]
  end

  # Run "mix help deps" to learn about dependencies.
  defp deps do
    [
      # {:dep_from_hexpm, "~> 0.3.0"},
      # {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
    ]
  end
end

The project section is where we set the application name, version etc. The deps section is where we add dependencies that we want to pull into our project.

We can pull in code from git or from https://hex.pm/.

Let’s start by adding a dependency on a package from hex.pm, I’m going to add the UUID package https://hex.pm/packages/uuid, so amend the deps section to look like this

defp deps do
  [
    {:uuid, "~> 1.1"}
  ]
end

The code was copied from the packages hex.pm page where it has code for various configurations.

We can check the state of our dependencies by typing mix deps. In my case it tells me the dependency is not available, I need to run mix deps.get.

We don’t need to compiler the dependency as it’ll automatically compile when we need it, but you can compile it using mix deps.compile if you prefer.

When you bring dependencies into your project you’ll see a deps folder created and similar to node_modules in web/node development, you’ll see the dependency code in this folder.

Let’s try UUID out (as per it’s documentation https://hexdocs.pm/uuid/readme.html), I created a lib/try_uuid.ex file and added the following code

defmodule TryUuid do

  def create() do
    UUID.uuid1()
  end
end

Run up iex using iex -S mix to allow access to the project and it’s dependencies. Finally run the code TryUuid.create() and if all went well you’ll get a UUI created.

Let’s add a dependency from GitHub, back to the mix.exs and change the deps section to like this

defp deps do
  [
    {:uuid, "~> 1.1"},
    {:better_weighted_random, git: "https://github.com/JohnJocoo/weighted_random.git" }
  ]
end

I basically picked the package from GitHub by searching for packages and looking for something fairly simple to try – this one seemed as good as any. There were no tags so I’ve not included a version with the dependency.

Now go through the process of mix get.deps now add a file, mine’s try_random.ex with the following code

defmodule TryRandom do
  def create() do
    WeightedRandom.take_one([{:'1', 0.5}, {:'2', 1.0}, {:'3', 2.0}])
  end
end

Compile using mix compile and then run up iex -S mix and finally execute the command TryRandom.create() and if all went well you’ll get some randomly selected value from the list give within the code.

AWS CloudFormation

Disclaimer: This post sat in draft for a while as I’m not using AWS at the moment I cannot guarantee that all works still or the post is 100% complete, but I’m going to publish it anyway, in case it’s of use.

AWS CloudFormation is a service that is essentially a way to group together AWS resources into a stack. We can define the stack using the AWS Dashboard or using either JSON or YAML.

For example, in JSON

{
  "Resources": {
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "BucketName": "mybucket"
      }
    }
  }
}

or in YAML

Resources:
  S3Bucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      BucketName: mybucket

In this example the S3 bucket will be created within the Stack. Remember the bucket name must be unique across all accounts.

  • From the dashboard type CloudFormation into the search box
  • Once you’re on the CloudFormation page click the Create stack button

From the Create stack page you can select a JSON or YAML template from your local machine, an Amazon S3 URL or from a Git repos. using the Template is ready option. Or you can Use a sample template to load a pre-existing stack, for example LAMP, Ruby on Rails, WordPress etc. Or you can select the Create template in designer option to use the stack designer.

Let’s see what we can do with CloudFormation by creating our simple Echo service web API and let some tools generate our CloudFormation configuration file for us.

Setting up our credentials

First we’re going to need to set up our credentials.

You’ll hopefully have stored the credentials when you create your IAM user, access id and secret are what’s required for this next step.

If you prefer to handle this from a UI such as Visual Studio Extensions, then you can use the edit credential button in the AWS Explorer (load via the menu View | AWS Explorer) but let’s first do this ourselves.

Credentials are obviously meant to be kept secret, so they’re stored on your local machine in the folder C:\Users\<your-username>\.aws in the file named credentials. The file is an INI type file and should look like this when you’ve added your credentials

[profilename]
aws_access_key_id = IDSUPPLIEDBYAWS
aws_secret_access_key = SECRETSUPPLIEDBYAWS

The profilename is the profile name as seen in tools such as AWS Explorer or will be used in the CLI, this allows us to have multiple access key/secrets for multiple apps.

Note: if you edit this file via AWS Explorer it will add further information to the profile

Using Visual Studio Extensions

If you install the AWS extensions for Visual Studio, you can create CloudFormation based applications using the project templates. If you create yourself a project based upon AWS Serverless Application (.NET Core – C#) for example, you’ll then get the option to choose a project type, I chose Minimal Wweb API.

The resultant project includes everything you need to build and deploy your application to AWS serverless.

For example from Visual Studio 2022 with the current AWS Extensions installed

  • Create a new project, select AWS Serverless Application (.NET Core – C#) or the with Tests version
  • Once create simply replace the minimal API root method with the following
    app.MapGet("/", () => "Use /echo?text=<text>");
    app.MapGet("/echo", (string text) => $"Echo: {text}");
    
  • Open the serverless.template file and change the Descriptiom to something meaningful to your project
  • Build and run locally to check everything works, i.e. /echo?text=Scooby should echo back Scooby
  • We can now simply right mouse click on the serverless.template file and select Publish to AWS Lambda
  • From the Publish to AWS Lambda popup, enter a Stack Name if it doesn’t exist it will be created
  • Enter an S3 Bucket name or click the New button and supply a name
  • You might need to change the AWS Credentials if required for your specific application and Region
  • Click Publish and if all goes well, your application will be published to a stack in AWS and upon completion the AWS extension will show the URL for you service and you can go and try it

At this point if you log into AWS using the same IAM account that your stack was deployed to, go to CloudFormation and you’ll see our stack was added, it should show Status as CREATE_COMPLETE it’ll have the desription we changed in the serverless.template file. In the Outputs tab we can see the ApiURL (in case you forgot to note it down).

Now if you enter S3 into the search box and go and look at our buckets, you’ll see the bucket we created and finally if you enter Lambda into the search box and go to the Lambda Functions page you’ll see the function name for our Web API.

I’m not going to dig too much into the serverless.template file but note that the Type

"Type": "AWS::Serverless::Function",

Essentially creates the web API as a lamba function, sets up the IAM execution role and adds the HTTP triggers to invoke the function.

Valid types are as followis

  • AWS::Serverless::Function as already looked at this will create the Lambda function, the execution role and any required triggers
  • AWS::Serverless::Api denotes a resource type for creating an API Gateway
  • AWS::Serverless::HttpApi denotes a resource type used to create REST API’s
  • AWS::Serverless::SimpleTable denotes a resource type to create a DynamoDB table with a single primary key