Category Archives: Web Api

Looking at security features and the web

Let’s take a look at various security features around web technologies, although I’ll we concentrating on their use in ASP.NET, but the information should be valid for other frameworks etc.

Note: We’ll look at some code for implementing this in a subsequent set of posts.

Authentication and Authorization

We’re talking Identity, JWT, OAuth, Open ID Connect.

Obviously the use of proper authentication and authorisation ensure only legitimate users have access to resources and forcing a least privilege and role based access ensures authenticated users can only access resources befitting their privileges.

OWASP risks mitigation:

  • A01 Broken Access Control and improper enforcement of permissions
  • A07 Identification and Authentication failures, weak of missing authentication flows

      • Data protection API (DPAPI) / ASP.NET Core Data Protection

      This is designed to protect “data at reset”, such as cookies, tokens CSRF keys etc. and providers key rotation and encryption services.

      OWASP risks mitigation:

      • A02 Cryptographic failures, weak or missing encryption of sensitive data

          • HTTPS Enforcement and HSTS

          This forces encrypted transport layers and prevents protocol downgrade attacks.

          OWASP risks mitigation:

          • A02 Cryptographic failures, sensitive data exposure
          • A05 Security misconfigurations, missing TLS or insecure defaults

              • Anti-Forgery Tokens (CSRF Protection)

              This prevents cross site request forgery by validation of user intent.

              OWASP risks mitigation:

              • A01 Broken access control
              • A05 Security misconfigurations
              • A08 Software and Data integrity failures such as session integrity

                  • Input Validation and Model Binding Validation

                  This prevents malformed or malicious input from reaching the business logic.

                  OWASP risks mitigation:

                  • A03 Injection, such as SQL, NoSQL and command injections
                  • A04 Insecure design, lacking validation rules
                  • A05 Security misconfigurations

                      • Output Encoding

                      This prevents untrusted data from being rendered, for example covers things like Razor, Tag Helpers, HTML Encoders.

                      OWASP risks mitigation:

                      • A03 Injection
                      • A05 Security misconfigurations
                      • A06 Vulnerable and outdated components

                          • Security Headers

                          Covers things such as CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy and mitigates XSS, click jacking, MIME sniffing and data leakage.

                          OWASP providers explicit guidance on recommended headers.

                          OWASP risks mitigation:

                          • A03 Injection, CSP reduces XSS
                          • A05 Security misconfigurations, missing headers
                          • A09 Security logging and monitoring failures, via reporting endpoints

                              • Rate limiting and throttling

                              This is included, but need to be enabled as ASP.NET built in middleware.

                              This prevents brute force, credential stuffing and resource exhaustion attacks.

                              OWASP risks mitigation:

                              • A07 Identification and Authentication failures
                              • A10 Server side request forgery (SSRF) limit abuse
                              • A04 Insecure design, lack of abuse protection

                                  • CORS (Cross‑Origin Resource Sharing)

                                  This controls which origins can access API’s and prevents unauthorized cross-site API calls.

                                  OWASP risks mitigation:

                                  • A05 Security misconfiguration
                                  • A01 Broken access control

                                      • Cookie Security

                                      Protects session cookies from theft or misuse.

                                      OWASP risks mitigation:

                                      • A07 Identification and Authentication failures
                                      • A02 Cryptographic Failures
                                      • A01 Broken access control

                                          • Dependency Management

                                          When using third party dependencies via NuGet, NPM etc. we need to ensure libraries are patched and up to date.

                                          OWASP risks mitigation:

                                          • A06 Vulnerable and outdated components

                                              • Logging and Monitoring

                                              This covers things like Serilog, Application Insights and built-in logging etc.

                                              Used to detect suspicious activites, as well as support incident response.

                                              OWASP risks mitigation:

                                              • A09 Security Logging and monitoring failures

                                                  • Secure deployment and configuration

                                                  This covers all forms of configuration, including appsettings.json, key vault, environment seperation etc.

                                                  Here we want to prevent secrets being exposed and enforce secure defaults.

                                                  OWASP risks mitigation:

                                                  • A05 Security misconfiguration
                                                  • A02 Cryptographic Failures

Base64 encoding

Base64 encoding is used when embedding binary in text based formats such as JSON, XML, YML etc. in such cases if we need to add a binary type, such as images or files etc. and we must pass then via a text format, then we need to Base64 encode this type of data first.

Use cases

Within web applications this is often used to pass binary within a JSON request/response object, but can be also seen when embedding images directly into HTML, for example

<img src="data:image/png;base64,your-binary-encoded-data..." />

It’s also used for Email attachments (MIME) as well as Authentication tokens – JWT tokens often use Base64URL (a variant of Base64).

Other use cases include clipboard copy/pasting of blobs (images/files etc.) into a text based clipboard format as well as being used from transporting over text only channels.

Where and why not to use Base64 encoding?

Base64 should NOT be used for streaming raw binary (application/octet-stream), large files or in binary safe protocols such as gRPC, websockets and HTTP when using the aforementioned large binary data etc.

First off, these protocols already support raw binary data so the affects of encoding are only on the negative side – if we encode to Base64 we will see, potentially, significant increases in the data size…

To Base54 encode using Javascript in the browser we can use

// encode 
const text = "Hello, world!";
const encoded = btoa(text);

// decode
const decoded = atob(encoded);

In C# we can use

// encode
byte[] bytes = Encoding.UTF8.GetBytes("Hello, world!");
string base64 = Convert.ToBase64String(bytes);

// decode
byte[] decodedBytes = Convert.FromBase64String(base64);
string decoded = Encoding.UTF8.GetString(decodedBytes);

Calculating the Base64 encoding affects

Base64 encoding encodes every 3 bytes of binary data in 4 ASCII characters, so we essentially expand a binary data payload when using Base64 encoding

var base64Size = (binarySize/3) * 4

Or we can approximate with

var base64Size = 1.33 * binarySize;

Plus up to 2 padding characters “=” if the binary size is not divisible by 3.

This means that for every 1MB (1048576 bytes) the Base64 size is about 1.4MB (1398104 chars) giving us a 33% overhead.

This ofcourse is significant in streaming of data as it adds to the bandwidth and memory overhead along with increases in CPU usage for the encoding/decoding.

Rust Rocket (and openapi/swagger)

Rocket provides code that allows us to build web servers and web based applications such as web APIs.

We’ll start by just creating a simple endpoint, and then we’ll look at adding Open API and swagger support.

Starting simple

Create yourself a Rust package, for example with cargo

cargo new myapi --bin

add the following dependency to your Cargo.toml

rocket = "0.5.1"

Now create a Rocket.toml so we can configure rocket’s server and mine looks like this

[default]
port = 8080
address = "127.0.0.1"

We need a main.rs (so you can delete the lib.rs if you wish for now) and here’s a very basic starting point

#[macro_use] extern crate rocket;

#[get("/")]
fn index() -> &'static str {
    "Hello, world!"
}

#[launch]
fn rocket() -> _ {
    rocket::build().mount("/", routes![index])
}

Now, run the application using

cargo run

As you can see this is a minimal API style, i.e. we create a function supplying it with an HTTP method and we add it to the routes list.

Adding the usual echo endpoint

Now let’s add my version of “Hello World” for API’s, a simple echo endpoint.

#[get("/echo?<text>")]
fn echo(text: &str) -> String {
    format!("Echo: {}", text)
}

Now add this to the routes i.e.

#[launch]
fn rocket() -> _ {
    rocket::build().mount("/", routes![index, echo])
}

Add Open API and Swagger

Now we have a couple of simple endpoints, let’s add Open API and Swagger and change the echo endpoint to use Json. I’m purposefully going to keep the index as non-Open API just to demonstrate running both Open API and non-Open API endpoints.

We’re going to need a few addition to our Cargo.toml – now, unfortunately it’s easy to get multiple version dependencies for these, so the one’s shown here will work together without warning/errors

[dependencies]
rocket = { version = "0.5.1", features = ["json"] }
openapi = "0.1.5"
serde = "1.0.219"
rocket_okapi = { version = "0.9.0", features = ["swagger"] }
schemars = "0.8.22"

Notice we’re adding features to the rocket crate and we’ve got some creates for swagger and open api. The schamars crate 0.8.22 was being used by other crates, hence I locked this down to the same version.

We’ll extend our echo endpoint to return Json, but before we do I’ll list the use clauses that are listed after #[macro_use] extern crate rocket;

use rocket::serde::{Serialize, json::Json};
use rocket_okapi::{openapi, openapi_get_routes};
use rocket_okapi::swagger_ui::{make_swagger_ui, SwaggerUIConfig};
use schemars::JsonSchema;

We’ll create a response object for the echo endpoint and update the echo endpoint to both return this and add the openapi attribute to allow this endpoint to have an open api spec generated for it

#[derive(Serialize, JsonSchema)]
struct EchoResponse {
    message: String,
}

#[openapi]
#[get("/echo?<text>")]
fn echo(text: &str) -> Json<EchoResponse> {
    Json(EchoResponse {
        message: format!("Echo: {}", text),
    })
}

Next up we need to change the rocket function, so let’s just see the latest version

#[launch]
fn rocket() -> _ {
    rocket::build()
        .mount("/", routes![index])
        .mount("/", openapi_get_routes![echo])
        .mount(
            "/swagger",
            make_swagger_ui(&SwaggerUIConfig {
                url: "/openapi.json".to_owned(),
                ..Default::default()
            })
        )
}

Now I purposefully left the index route without an open api attribute just to demonstrate, if you have such endpoints, you need to still use the routes! macro, if you add index to openapi_get_routes! without the open api attribute you’ll get some slight ambiguous error’s such as a function with a similar name exists.

Now run your application and go to http://localhost:8080/swagger/index.html and you can interact with your endpoints via the Swagger UI you can also access the openapi.json file using http://localhost:8080/openapi.json.

Code

Available on GitHub.

Elixir and Phoenix

Most languages which I start to learn, I’ve found I learn the basics of the language (enough to feel at relative ease with the language) but then want to see it in real world scenarios. One of those is usually a web API or the likes. So today I’m looking at using Elixir along with the Phoenix framework.

Note: I’m new to Elixir and Phoenix, this post is based upon my learnings, trying to get a basic web API/service working and there may be better ways to achieve this that I’m not aware of yet.

Phoenix is a way to (as they say on their site) to “build rich, interactive web applications”. Actually I find it builds too much as by default it will create a website, code for working with a DB (PostgresQL by default) etc. In this post I want to create something more akin to a web API or microservice.

In this post I want to create a simple API service so instead we’ll use phx.new to create a service named my_api and well remove the website/HTML and ecto (the DB) side of things 

mix phx.new my_api --no-html --no-ecto --no-mailer

If this fails with a “not found” error, try installing phx using “mix archive.install hex phx_new”

If you run the command above you’ll get a new application generated. Just cd my_api to allow us to run the service etc.

If you’d like to see what the default generated application is then run the following

mix phx.server

By default this will start a server against localhost:4000. If you open the browser you’ll see a default dashboard/page which likely says there’s no route for GET / and then lists some available routes.

The /dev/dashboard route takes you to a nice LiveDashboard showing information about the Elixir and Phoenix.

To shutdown the Phoenix server CTRL+C twice within the terminal that you ran it up from.

For my very simple web service, I do not even what the live dashboard. So if you created that new app. delete your new app folder and then run this minimal code version (unless you’d prefer to keep live dashboard etc.)

mix phx.new my_api --no-html --no-ecto --no-mailer --no-dashboard --no-assets --no-gettext

This will then generate a fairly minimal server which is a good starting point for our service. You’ll notice first off that there are now, no routes when you run this via mix phx.server
.

Let’s add a controller, this will acts as the controller for our web service, so within the /lib/my_api_web/controllers folder add a new file named math-controller.ex and past the following code into it (obviously change the module name to suite your application name)

defmodule MyApiWeb.MathController do
  #use MyApiWeb, :controller
  use Phoenix.Controller, formats: [:html, :json]

  def index(conn, _params) do
   json(conn, "{name: Scooby}")
  end
end

We now need to hook up our controller to a route, so go to the router.ex file within the /lib/my_api_web/ folder and alter the scope section to look like this

scope "/", MyApiWeb do
  pipe_through :api

  resources "/api", MathController, except: [:new, :edit, :create, :delete, :update, :show]
end

If you run mix phx.server you should see a route to /api, typing http://localhost:4000/api will return “{name: Scooby}” as defined in the math-controller index. This is not very math-like so let’s create a couple of functions, one for adding numbers and one for subtracting.

Remove the resources section (or comment out using #) in the scope then add the following routes

get "/add", MathController, :add
get "/sub", MathController, :subtract

Go to the math-controler.ex and add the following functions

def add(conn, %{"a" => a, "b" => b}) do
  text(conn, String.to_integer(a) + String.to_integer(b))
end

def subtract(conn, %{"a" => a, "b" => b}) do
  text(conn, String.to_integer(a) - String.to_integer(b))
end

Notice we destructuring params to values a and b – we’ll convert those values to integers and use the text function to return raw text (previously we expected JSON hence uses the json function). Now when you browse the add method, for example http://localhost:4000/add?a=10&b=5 or subtract method, for example http://localhost:4000/sub?a=10&b=5 you should see raw text returned with answers to the math functions.

What routes are we exposing

Another useful way of checking the available routes (without running the server) is, as follows

mix phx.routes

Config

If you’ve looked around the generated code you’ll notice the config folder.

One thing you might like to do now is change localhost to 0.0.0.0 so edit dev.exs and replace

http: [ip: {127, 0, 0, 1}, port: 4000],

with

http: [ip: {0, 0, 0, 0}, port: 4000],

If you do NOT do this and you decide to deploy the dev release to Docker, you’ll find you cannot access your service from outside of Docker (which ofcourse is quite standard).

Releases

Generating a release will precompile any files that can be compiled and allows us to run the server without the source code (as you’d expect) you will need to tell the compiler what configuration to use, we do that by setting the MIX_ENV like this

export MIX_ENV=prod

(No MIX_ENV environment variable will default dev)

Then running

mix release

This will create and assemble your compiled files to _build/prod/rel/my_api/bin/my_api (obviously replacing the last part with your app name). The results of a release build show using

Note: Replace /prod/ with /dev/ above etc. as per the environment you’ve compiled for

_build/prod/rel/my_api/bin/my_api start

to start your application, this will need start a server. By default the above does not start a server so instead we need to set the following environment variable

export PHX_SERVER=true

You’ll also able to run following it will automatically generate the bin/server and sets the PHX_SERVER environment variable

mix phx.gen.release

One last thing, you may find when you use the start command (against PROD) that this fails saying you are missing the SECRET_KEY_BASE. We can generate this using

mix phx.gen.secret

Then simply 

export SECRET_KEY_BASE=your-generated-key

This is for signing cookies etc. and you can see where the exception comes from within the runtime.exs file. This is set as an environment variable, best not to check into source control.

Dockerizing our service

Okay, it’s not Elixir specific, but I feel that the natural conclusion to our API/service development is to have it all running in a container. Let’s start by creating a container image based upon the build and using the phx.server call…

Create yourself a Dockerfile which looks like this

FROM elixir:latest

RUN mkdir /app
COPY . /app
WORKDIR /app

RUN mix local.hex --force
RUN mix do compile

EXPOSE 4000

CMD ["mix", "phx.server"]

I’m assuming we’re going to stick with port 4000 in the above and in the commands below, so I’ll document this via the EXPOSE command.

Now to build and run our container let’s use the following

docker build -t pp/my-api:0.1.0 .
docker run --rm --name my-api -p 4000:4000 -d pp/my-api:0.1.0

Now you should be able to uses http://localhost:4000 to access your shiny new Elixir/Phoenix API/service.

Note: Remember that if you cannot access the service outside of the docker image, ensure you’ve set the http ip in dev.exs to 0.0.0.0

If we want to instead containerize our release build then we could use the following

FROM elixir:latest

ENV PHX_SERVER=true

RUN mkdir /app
COPY /_build/dev/ /app
WORKDIR /app/rel/my_api/bin

EXPOSE 4000

CMD ["./my_api", "start"]

Again using the previous build and run commands, will start the server (if all went to plan).

Code

Code is available in my GitHub blog project repo.

Multilingual support for a ASP.NET web API application

We sometimes wish to make our web API return error messages or other types of string data in different languages. The process for this is similar to MAUI and WinForms, we just do the following

  • Create a folder names Resources in our web APIT project
  • Add a RESX file, which we’ll name AppResources.resx. This will be the default language, so in my case this will include en-GB strings
  • Ensure the file has a Build Action of Embedded resource and Custom Tool of ResXFileCodeGenerator
  • Add a name (which is the key to your resource string) and then add the value. This is the string (i.e. the translated string) for the given key
  • Let’s add another RESX file, but this type name it AppResources.{language identifier}.resx, for example AppResources.de-DE.resx which will contain the German translation of the key/name’s
  • Again ensure the Build Action and Custom Tool are correctly set

The ResXFileCodeGenerator will generate properties in the AppResources class for us to access the resource strings. For example

AppResources.ExceptionMessage

If we need to test our translations without changing our OS language, we simply use code such as the following in the Program.cs of the web API

AppResources.Culture = new CultureInfo("de-DE");

Blazor and the GetFromJsonAsync exception TypeError: Failed to Fetch

I have an Azure hosted web api. I also have a simple Blazor standalone application that’s meant to call the API to get a list of categories to display. i.e. the Blazor app is meant to call the Azure web api, fetch the data and display it – should be easy enough, right ?

The web api can easily accessed via a web browser or a console app using the .NET HttpClient, but the Blazor code using the following simply kept throwing an exception with the cryptic message “TypeError: Failed to Fetch”

@inject HttpClient Http

// Blazor and other code

protected override async Task OnInitializedAsync()
{
   try
   {
      _categories = await Http.GetFromJsonAsync<string[]>("categories");
   }
   catch (Exception e)
   {
      Debug.WriteLine(e);
   }
}

What was happening is I was actually getting a CORS error, sadly not really reported via the exception so not exactly obvious.

If you get this error interacting with your web api via Blazor then go to the Azure dashboard. I’m running my web api as a container app, type CORS into the left search bar of the resource (in my case a Container App). you should see the Settings section CORS subsection.

Add * to the Allowed Origins and click apply.

Now your Blazor app should be able to interact with the Azure web api app.