Category Archives: Application Insights

Basics of KQL

In the previous two post we’ve looked at logging and using the TelemetryClient to send information to Application Insights. Application Insights offers a powerful query language (Kusto Query Language – KQL) for filtering logs.

We cannot possibly cover all options of KQL, so this post will just cover some of the basics and useful queries.

Tables

Application Insights supplied several tables as ways tracking events, information and various logging data.

The main tables are as follows

  • requests: Logs information regarding HTTP requests in your application.
  • dependencies: Tracks calls made to external services or databases.
  • exceptions: Logs exceptions within your application.
  • traces: Diagnostic log messages and traces from your application.
  • pageViews: Page views and user interactions within your web application.
  • customEvents: Custom events that are defined to track specific actions or user interactions.
  • metrics: Tracks performance metrics and custom metrics.
  • availabilityResults: Availability tests that check uptime and responsiveness of your application/
  • appExceptions: Like exceptions, specifically for application exceptions.
  • appMetrics: Like metrics, specifically for application metrics.
  • appPageViews: Like pageViews, specifically for application page views.
  • appPerformanceCounters: Performance counters for your application.
  • appSystemEvents: System level events within your application.
  • appTraces: Like traces, specifically for your application traces.
  • azureActivity: Azure activity logs.
  • browserTimings: Captures detailed timings information about the browser’s performance when loading web pages.

We can combine timespan, so 1d6h30m means 1 day, 6 hours and 30 minutes.

Get everything

We can get everything across all tables (see below for information on the tables) using

search *

Tables

We can just get everything from a table by running the query against a table, so for example for the traces table

traces

Filtering

Obviously returning all traces for example, is probably returning more rows than we want, so we can filter using the where keyword

traces
| where severityLevel == 3

Projections

traces
| where severityLevel == 3
| project timestamp, message

Aggregation

traces
| where severityLevel == 3
| summarize count() by bin(timestamp, 1h)

Ordering

requests
| where success == "false"
| summarize count() by bin(timestamp, 1h)
| order by bin(timestamp, 1h) desc

Samples

Get all the tables that have data

search * 
| distinct $table

Get all records within a table for the last 10 minutes

traces
| where timestamp > ago(1m)

The ago function allows us to use a timespan which includes

  • d: Days, for example 3d for three days
  • h: Hours, for example 2h for two hours
  • m: Minutes, for example 30m for thirty minutes
  • s: Seconds, for example 10s for ten seconds
  • ms: Milliseconds, for example 100ms for a hundred milliseconds
  • microsecond: Microseconds, for example 20microsecond for 20 microseconds
  • tick: Nanoseconds, for example 1tick for 100 nanoseconds

Summarizing each day’s request count into a timechart (a line chart). We also have options got a bar chart (barchart), pie chart (piechart), area chart (areachart) and scatter chart (scatterchart)

requests
| summarize request_count = count() by bin(timestamp, 1d)
| render timechart 

For some of the other chart types we need to supply difference information, so let’s look at a pie chart of the different requests

requests
| summarize request_count = count() by name
| render piechart    

We can get requests between two dates, including using of the now() function

requests
| where timestamp between (datetime(2025-02-14T00:00:00Z) .. now())

References

Kusto Query Language (KQL) overview

Tracking events etc. with Application Insights

In my previous post I looked at what we need to do to set-up and using Application Insights for our logs, but we also have access to the TelemetryClient in .NET (Microsoft also have clients for other languages etc.) and this allows us to send information to some of the other Application Insights, for example tracking events.

Tracking events is useful as a specific type of logging, i.e. we want to track, potentially, whether one of our application options is ever used, or to what extent it’s used. Imagine we have a button that runs some long running calculation – well if nobody ever uses it, maybe it’s time to deprecate and get rid of it.

Ofcourse we can just use logging for this, but the TelemetryClient allows us to capture data within the customEvents and customMetrics tables within Application Insights (we’re look at the available tables in the next post on the basics if KQL) and hence reduce the clutter of lots of logs.

Take a look at my post Logging and Application Insights with ASP.NET core. To see code for a simple test application. We’re going to simply change the app.MapGet code to look like this (note I’ve left the logging on in place as well so we can see all the options for telemetry and logging)

app.MapGet("/test", (ILogger<Program> logger, TelemetryClient telemetryClient) =>
{
    telemetryClient.TrackEvent("Test Event");
    telemetryClient.TrackTrace("Test Trace");
    telemetryClient.TrackException(new Exception("Test Exception"));
    telemetryClient.TrackMetric("Test Metric", 1);
    telemetryClient.TrackRequest("Test Request", DateTimeOffset.Now, TimeSpan.FromSeconds(1), "200", true);
    telemetryClient.TrackDependency("Test Dependency", "Test Command", DateTimeOffset.Now, TimeSpan.FromSeconds(1), true);
    telemetryClient.TrackAvailability("Test Availability", DateTimeOffset.Now, TimeSpan.FromSeconds(1), "Test Run", true);
    telemetryClient.TrackPageView("Test Page View");

    logger.LogCritical("Critical Log");
    logger.LogDebug("Debug Log");
    logger.LogError("Error Log");
    logger.LogInformation("Information Log");
    logger.LogTrace("Trace Log");
    logger.LogWarning("Warning Log");
})
.WithName("Test")
.WithOpenApi();

As you can see, we’re injecting the TelemetryClient object and Application Insights is set up (as per my previous post) using

builder.Services.AddApplicationInsightsTelemetry(options =>
{
    options.ConnectionString = configuration["ApplicationInsights:InstrumentationKey"];
});

From the TelemetryClient we have these various “Track” methods and as you can no doubt summise, these map to

  • TrackEvent: maps to the customEvents table
  • TrackTrace: maps to the trace table
  • TrackException: maps to the exeptions table
  • TrackMetric: maps to the customMetrics table
  • TrackRequest: maps to the requests table
  • TrackDependency: maps to the dependencies table
  • TrackAvailability: maps to the availablilityResults table
  • TrackPageView: maps to the pageViews table

Telemetry along with standard logging to Application Insights gives us a wealth of information that we can look at.

Ofcourse, assuming we’re sending information to Application Insights, we’ll then want to look at features such as the Application Insights | Monitoring | Logs where we can start to query against the available tables.

Logging and Application Insights with ASP.NET core

Obviously when you’re running an ASP.NET core application in Azure, we’re going to want the ability to capture logs to Azure. This usually means logging to Application Insights.

Adding Logging

Let’s start out by just looking at what we need to do to enable logging from ASP.NET core.

Logging is included by default in the way of the ILogger interface (ILogger<T>), hence we can inject into our code like this (this example uses minimal API)

app.MapGet("/test", (ILogger<Program> logger) =>
{
    logger.LogCritical("Critical Log");
    logger.LogDebug("Debug Log");
    logger.LogError("Error Log");
    logger.LogInformation("Information Log");
    logger.LogTrace("Trace Log");
    logger.LogWarning("Warning Log");
})
.WithName("Test")
.WithOpenApi();

To enable/filter logging we have something like the following within the appsettings.json file

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
}

The LogLevel, Default section sets the minimum logging level for all categories. So for example a Default of Information means only logging of Information level and above (i.e. Warning, Error and Critical) are captured.

The Microsoft.AspNetCore is a category specific logging in that it logs Microsoft.AspNetCore namespace logging using the supplied log level. Because we can configure by namespace we can also use categories such as Microsoft, System, Microsoft.Hosting.Lifetime. We can also do the same with our code, i.e. MyApp.Controllers, so this allows us to really start to tailor different sections of our application an what gets captured in the logs.

Logging Levels

There various logging levels are as follows

  • LogLevel.Trace: The most detailed level, use for debugging and tracing (useful for entering/existing methods and logging variables).
  • LogLevel.Debug: Detailed but less so than Trace (useful for debugging and workflow logging).
  • LogLevel.Information: Information messages at a higher level than the previous two levels (useful for logging steps of processing code).
  • LogLevel.Warning: Indicates potentially problems that do not warrant error level logging.
  • LogLevel.Error: Use for logging errors and exceptions and other failures.
  • LogLevel.Critical: Critical issues that may cause an application to fail, such as those that might crash your application. Could also include things like missing connection strings etc.
  • LogLevel.None: Essentially disables logging

Application Insights

Once you’ve created an Azure resource group and/or Application Insights service, you’ll be able to copy the connection string to connect to Application Insights from your application.

Before we can use Application Insights in our application we’ll need to

  • Add the nuget package Microsoft.ApplicationInsights.AspNetCore to our project
  • Add the ApplicationInsights section to the appsettings.json file, something this
    "ApplicationInsights": {
      "InstrumentationKey": "InstrumentationKey=xxxxxx",
      "LogLevel": {
        "Default": "Information",
        "Microsoft": "Warning"
      }
    },
    

    We can obviously set the InstrumentKey in code if preferred, but the LogLevel is specific to what is captured within Application Insights

  • Add the following to the Program.cs file below CreateBuilder

    var configuration = builder.Configuration;
    
    builder.Services.AddApplicationInsightsTelemetry(options =>
    {
        options.ConnectionString = configuration["ApplicationInsights:InstrumentationKey"];
    });
    

Logging Providers in code

We can also add logging via code, so for example after the CreateBuilder line in Program.cs we might have

builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Logging.AddDebug(); 

In the above we start by clearing all currently logging providers, the we add a provider for logging to console and debug. The appsettings.json log levels are still relevant to which logs we wish to capture.