Category Archives: GitHub Actions

Github action build scripts for various languages

I’ve been through a bit of a wave of writing my unit of measurement library code for various programming languages, originally starting in F# through C# and Java to Go, Rust, Swift, Typescript and Python. Each time I’ve needed/wanted to create a Github actions build workflow for each language. To be honest Github gives you all the information you need, but I’m going to list what I’ve got and what worked for me etc.

Creating the workflow file

I’ve covered much of this before, but I’ll recreate here for completeness.

You’ll need to start by following these steps

  • Create a .github folder in the root of your github repository
  • Within the .github folder create a workflows folder
  • Within the workflows folder create a file – it can be named whatever you like, let’s use build.yml (yes it’s a YAML file)

The workflow configuration

Note: YAML is a format where whitespace is significant. In the code snippets below I will left justify the code, basically breaking the format but making it easier to read. At the end of the post I’ll put all the snippets together to show the correct format, if you’re just here for the code, then I’d suggest scrolling to the bottom of the post.

Your build.yml (if you followed my naming) will start with the name of the workflow, so let’s simply name it Build, i.e.

name: Build

Next up we need to list the events that cause the workflow to start, usually this will be things like “on push” or “on pull_request”. So now we add the following to the .yml file

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
  workflow_dispatch: # Manual Run

As Github has moved away from using master for the root branch to main obviously change master to main in the above. I tend to (at least initially) also include a way to manually run the workflow if I need to, hence include workflow_dispatch.

So we have a name of the workflow and the events that trigger the workflow, now we need the tasks or in Github action terms, the jobs to run. So we’ll just add the following, to our .yml

jobs:
   build:

Now for the next step, we can either simply list the VM OS to run on using

runs-on: ubuntu-latest

Or, as all the languages listed at the start of this post are cross platform, we might want to try building them on each OS we want to support, in this case we create a strategy and matrix of OS’s to support. It’d also be good to see in the build what OS we’re building for so we’ll create the following

name: Build on ${{ matrix.os }}
strategy:
  matrix:
    os: [macos-latest, ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}

In the above, we’ll try to run our build on Mac, Ubuntu and Windows latest OS environments. If you only care about one environment you can use the previous runs-on example or just reduce the os array to the single environment.

Note: If you take a look at virtual-environments, you’ll see a list of supported TAML labels for the OS environment.

Within the matrix we can declare variables that are used in runs-on but also for steps, so for example if we’re wanted to list the version(s) that we want to support of a toolchain or language, we could do something like

matrix:
  os: [macos-latest, ubuntu-latest, windows-latest]
  go-version: [1.15.x]

So now we have matrix.go-version available for deciding which version of Go (for example) we’ll install.

We’ve setup the environments now we need to actually list the steps of our build. The first step is to checkout the code, so we’ll have something like this

steps:
- uses: actions/checkout@v2

Next we’re create each step of the build process, usually with a name for each step to make it easier to see what’s happening as part of the build workflow. At this point we’ll start to get into the specifics for each toolchain and language. For example for F# and C# we’ll start by setting up dotnet. For Swift we can just start calling the swift CLI. So let’s go through each toolchain/language setup I have for my unit conversion libraries/packages.

.NET steps

For .NET we’ll need to setup .NET with the version(s) we want to build against. Then install any required nuget dependencies, then build our code, run any tests and if required package up and even deploy to NuGet (which is covered in my post Deploying my library to Github packages using Github actions).

- name: Setup .NET Core SDK 6.0.x
  uses: actions/setup-dotnet@v1
  with:
    dotnet-version: '6.0.x'
- name: Install dependencies
  run: dotnet restore
- name: Build
  run: dotnet build --configuration Release --no-restore
- name: Test
  run: dotnet test FSharp.Units.Tests/FSharp.Units.Tests.fsproj --no-restore --verbosity normal
- name: Create Package
  run: dotnet pack --configuration Release

Note: Ofcourse change the FSharp.Units.Tests/FSharp.Units.Tests.fsproj to point to your test project(s).

Java steps

For Java we’re just going to have a setup step and then use Maven to build and run tests

- name: Setup JDK 8
  uses: actions/setup-node@v1
  with:
    java-version: '8'
    distribution: 'adopt'
- name: Build with Maven
  run: mvn --batch-mode --update-snapshots verify
[code]

<strong>Go steps</strong>

With Go we're using a previously declared <em>go-version</em> to allow us to target the version of Go we want to setup, ofcourse we could do the same for .NET, Java etc. Next we're installing dependencies, basically I want to install golint to lint my code. Next up we build our code using the Go CLI then vet and lint before finally running the tests.

[code]
- name: Setup Go
  uses: actions/setup-go@v2
  with:
    go-version: ${{ matrix.go-version }}
- name: Install dependencies
  run: |
    go version
    go get -u golang.org/x/lint/golint
- name: Build
  run: go build -v ./...
- name: Run vet & lint
  run: |
    go vet ./...
    golint ./...
- name: Run testing
  run: go test -v ./...

Rust steps

For Rust we’ll keep the steps pretty simple

- name: Build
  run: cargo build --verbose
- name: Run tests
  run: cargo test --verbose

Swift steps

- name: Build
  run: cargo build --verbose
- name: Run tests
  run: cargo test --verbose

Typescript steps

- name: Use Node.js
  uses: actions/setup-node@v1
  with:
    node-version: '12.x'
- name: Install dependencies
  run: yarn
- run: yarn run build
- run: yarn run test

Python steps

- name: Set up Python
  uses: actions/setup-python@v2
  with:
    python-version: '3.x'
- name: Test with pytest
  run: |
    python -m unittest

Building and testing TypeScript code using Github actions

In previous posts we’ve used Github actions to build .NET core code, now let’s create a workflow to build/transpile a TypeScript application.

  • Create a file (mine’s named build.yml) in your Github repo. in the folder .github/workflows
  • As per other examples of such a .yml action file we need to give the build a name and set up the triggers, so let’s do that, here’s the code
    name: Build
    
    on:
      push:
        branches: [ master ]
      pull_request:
        branches: [ master ]
    
    jobs:
      build:
    
        runs-on: ubuntu-latest
    

    In the above we’ve simply named the action Build which is triggered by pushes to master (and pull requests). We then set the action up to run on an ubuntu VM

  • Next we’ll add the following code to create a strategy (basically think, job configurations and application wide variables)
    strategy:
      matrix:
        node-version: [12.x]
    

    In this case we’re creating a variable named node-version with the version of node we want to use.

  • Now we’ll create the steps for the build
    steps:
      - uses: actions/checkout@v2
      - name: Node.js
        uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm install -g yarn    
      - name: yarn install, build and test
        run: | 
          yarn 
          yarn build
          yarn test
    

    Firstly we declare the checkout action to be used, then we used the setup-node action to (as the name suggests) setup node with the version we defined. In this example we then run npm to install yarn. Obviously we could have simply used npm instead (and you can replace the yarn commands with npm if you wish).

    Finally we have a step to install, build and test our code running the relevant yarn commands to carry out these tasks.

The above expects the package.json of your project to have scripts named test and build. For example

"scripts": {
  "test": "jest",
  "build": "tsc"
}

Deploying my library to Github packages using Github actions

In my previous post I explained the steps to use Github actions to package and deploy your .nupkg to NuGet, but Github also includes support for you to deploy your package alongside your project sources.

If you take a look at the right hand of your project within Github (i.e. where you see your main README.md) you’ll notice the Packages section, if you click on the Publish your first package it tells you the steps you need to take to create and deploy your package, but it wasn’t quite that simple for me, hence this post will hopefully help others out a little if they hit similar issues.

The first thing you need to do is make sure you are using a version of the dotnet CLI that supports nuget commands (specifically the add command). Here’s a set-up step with a compatible version of dotnet CLI.

 - name: Setup .NET Core
      uses: actions/setup-dotnet@v1
      with:
        dotnet-version: 3.1.401

The version must be 3.1.401 or above. There may be another before this but I was using 3.1.101 and was getting the following failure in the build output

Specify --help for a list of available options and commands.
error: Unrecognized command or argument 'add'

So the first command listed in Publish your first package will fail if you’re not upto date with the version of dotnet. If you’re using ubuntu instead of Windows for your builds, then you need to include the –store-password-in-clear-text on your dotnet nuget add command. Also change GH_TOKEN to ${{ secrets.GITHUB_TOKEN }}

Hence your first command will look more like this now

dotnet nuget add source https://nuget.pkg.github.com/your_user/index.json -n github -u your_user -p ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text

Replace “your_user” with your GitHub username and if you’ve created a permission that you prefer to use in place of GITHUB_TOKEN, then replace that also.

The second line shown in Publish your first package is

dotnet pack --configuration Release

Thankfully this is worked without an issue, however the Step 3 from didn’t work for me. It requires that I set the API key, similar to way we do this with NuGet publishing but using GITHUB_TOKEN again, hence the third step for me to publish to GitHub is

dotnet nuget push your_path/bin/Release/*.nupkg --skip-duplicate --api-key ${{secrets.GITHUB_TOKEN}} --source "github"

Replacing “your_path” with the path of your package to be published. Use –skip-duplicate so that any change to your code will not fail when a build is triggered, as without this option the command tries to publish an existing/unchanged package, causing a failure. Also set the –api-key as shown.

As I am already creating a package for NuGet and want to do the same for GitHub, we can condense the commands to something like

run: |
  dotnet nuget add source https://nuget.pkg.github.com/your_user/index.json -n github -u your_user -p ${{ secrets.GITHUB_TOKEN }} --store-password-in-clear-text
  dotnet nuget push your_path/bin/Release/*.nupkg --skip-duplicate --api-key ${{secrets.GITHUB_TOKEN}} --source "github"

Don’t Forget

One more thing, in your nuspec or via the Properties | Package tab for your project, ensure you add a Repository URL to your source code. The Github package will fail without this.

For example in the SDK style csproj we have

<RepositoryUrl>https://github.com/putridparrot/PutridParrot.Randomizer.git</RepositoryUrl>
<RepositoryType>git</RepositoryType>

The RepositoryType is not required for this to work, just using RepositoryUrl with https://github.com/putridparrot/PutridParrot.Randomizer worked fine.

Within a nuspec the format is as follows (if I recall)

<repository type="git" url="https://github.com/putridparrot/PutridParrot.Randomizer.git" />

Finally you’ll want to add https://nuget.pkg.github.com/your_user/index.json to you nuget.config to access Github’s NuGet package management from a project using this file and you will need a permission set up to access to API to pull in your package, alternatively simply download the package from Github and set-up a local nuget repository.

Deploying my library to NuGet using Github actions

I have a little library that I wanted to package up and make available on nuget.org. It’s nothing special, seriously it’s really not special (source available here PutridParrot.Randomizer).

Note: at the time of writing the library is very much in an alpha state, so don’t worry about the code etc., instead what we’re interested in is the .github/workflows/dotnet-core.yml.

Setting up NuGet

  • If you don’t already have a NuGet account then go to NuGet Register and sign up.
  • Next, click on the top right hand account name drop down. Clicking on this will display a drop down, from here select API Keys.
  • Click the Create option, and give your library a key name and ensure Push is checked and Push new packages and package version is selected, finally in the Glob Pattern just enter an *
  • Finally click the Create button to generate an API key

The API key should be kept secret as it will allow anyone who has access to it to upload package to your account. If for any reason you need to regenerate it, then from the Manage option on the API keys screen, click Regenerate.

Creating our deployment action

I’m using Github actions to build and then deploy my package. As you’ve seen, we need to use an API key to upload our package and obviously we do not want this visible in the build/deployment script, so the first thing we need to do is create a Github secret.

  • Go to the Settings for your project on Github
  • Select the Secrets tab on the left of the screen
  • Click New Secret
  • Give your secret a name, i.e. NUGET_API_KEY
  • In the value, of the secret, place the API key you got from NuGet

This secret is then accessible from your Github action scripts.

Note: You might be thinking, “didn’t this post say the key was to be kept secret”, I’m assuming as Microsoft owns Github and NuGet that we’re fairly safe letting them have access to the key.

Now we need to add a step to package and then a step to deploy to nuget to our Github workflow/actions.

We’re not going to be doing anything clever with our package, so not creating a nuspec for the project or signing it (at this time). So we’ll just use the dotnet CLI pack option. Here’s the addition to these dotnet-core.yml workflow for step

- name: Create Package
  run: dotnet pack --configuration Release
- name: Publish to Nuget
  run: dotnet nuget push /home/runner/work/PutridParrot.Randomizer/PutridParrot.Randomizer/PutridParrot.Randomizer/bin/Release/*.nupkg --skip-duplicate --api-key ${{secrets.NUGET_API_KEY}} --source https://api.nuget.org/v3/index.json

In the above, we have a step “Create Package” to create our package using the dotnet CLI’s pack command using release configuration. If (as I am) you’re using an ubuntu image to run your workflow this will be stored a a folder similar to this one listed below

/home/runner/work/PutridParrot.Randomizer/PutridParrot.Randomizer/PutridParrot.Randomizer/bin/Release/

Where you replace PutridParrot.Randomizer with you project name. Or better still, check the build logs to see where the output should show where the .nupkg was written to.

As you can see from the “Publish to Nuget” step, you’ll also need to reuse the path there where we again use the dotnet cli to push our nupk file to NuGet using the secret we defined earlier as the API key. In this usage we’re also not pushing if the package is a duplicate.

That’s it.

_Note: One side note – if you go onto nuget.org to check your packages, they may by in “Unlisted” state initially. This should change automatically after a few minutes to Published._

GitHub Actions – publishing changes to a branch

If you want to do something like GitHub pages does with Jekyll, i.e. takes master and generates the website and then publishes the resultant files to the gh-pages branch, then you’ll need to set-up personal access tokens and use them in your GitHub action, for example

  • Go to Personal access tokens
  • Click on the “Generate new token” button
  • In the Note field, give it a descriptive name so you know the purpose of the token
  • If you’re wanting to interact with the repo (as we want to for this example) then check the repo checkbox to enable all repo options
  • Click the “Generate token” button
  • Copy the generated token for use in the next section

Once we have a token we’re going to use this in our repositories. So assuming you have a repo created, do the following to store the token

  • Go to your repository and click the “Settings” tab
  • Select the “Secrets<" option
  • Click on the “New secret” button
  • Give the secret a name, for example PUBLISH_TOKEN
  • Paste the token from the previous section in the “Value” textbox
  • Finally click the “Add secret” button

This now stores the token along with the name/key, which can then be used in our GitHub action .yml files, for example here’s a snippet of a GitHub action to publish a website that’s stored in master to the gh-pages branch.

- name: GitHub Pages Publish
  if: ${{ github.ref == 'refs/heads/master' }} 
    uses: peaceiris/actions-gh-pages@v3.6.1
    with:
      github_token: ${{ secrets.PUBLISH_TOKEN }}
      publish_branch: gh-pages
      publish_dir: ./public

In this example action, we check for changes on master, then use GitHub Actions for GitHub Pages to publish to the gh-pages branch from the ./public folder on master. Notice we use the secrets.PUBLISH_TOKEN which means GitHub actions will supply the token from our secrets setting using the name we gave for the secret.

Obviously this example doesn’t build/generate or otherwise do anything with the code on master, it simply takes what’s pushed to master/public and publishes that to the gh-pages branch. Ofcourse if we combine this action with previous build/generate steps as part of a build pipleline.

Getting started with GitHub and the .NET Core Action

GitHub actions allows us to add workflows to our projects.

In the previous three posts we looked at using Appveyor, Circleci and Travis CI to create our CI/CD pipelines, now let’s look at using GitHub Actions.

  • From your GitHub project, select the Actions tab
  • GitHub kindly lists a suggested workflow, so as my project is in C# it suggests a .NET Core build action, select Set up this workflow if it suits your needs (it does for this example)
  • You’ll now get to edit the Actions yml configuration, I’ll accept it by clicking Start commit and this will add the dotnet-core.yml to .github/workflows. Just click commit to add it to your repository.
  • From the Actions | .NET Core workflow, press the Create status badge, then Copy status badge Markdown and place this markdown into your README.md

Note: If you’re using the .NET Core workflow and using multi targets (i.e. <TargetFrameworks>netcoreapp3.1;net472</TargetFrameworks> then you may find the build failing because the .NET 4.7.2 frameworks is not installed.

The process described above, demonstrated that we have a YML based DSL for creating our workflow, checkout Workflow syntax for GitHub Actions.

Let’s take a look at a workflow file for one of my projects – in this case .NET core was not suitable, instead I wanted .NET Framework

name: .NET Core

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: windows-latest

    steps:
    - uses: actions/checkout@v2
    - name: Setup MSBuild Path
      uses: warrenbuckley/Setup-MSBuild@v1
      
    - name: Setup NuGet
      uses: NuGet/setup-nuget@v1.0.2
     
    - name: Restore NuGet Packages
      run: nuget restore MyApplication.sln
 
    - name: Build
      run: msbuild MyApplication.sln /p:Configuration=Release /p:DeployOnBuild=true

The name is what you’ll see in your list of GitHub actions (via the GitHub project’s Actions tab/button, this workflow monitors pushes and pull requests on master and then has a list of jobs to undertake once this workflow is triggered.

We’re going to run this workflow on Windows as the project is a .NET framework application. The steps of the workflow specify the “parts” required to build and test the project, so require the checkout action and Setup-MSBuild, setup-nuget libraries. Then we run the nuget restore to get all nuget package then build the application.

Note: I’ve not include tests as yet on this workflow, so I’ll leave that to the reader.

As usual we’ll want to create a badge, in the Actions tab in GitHub, select the workflow and on the right of the screen is a button Create status badge. Click this then press the Copy status badge Markdown button and this will place the Markdown into your clipboard. Here’s an example of mine

![.NET Core](https://github.com/putridparrot/MyApplication/workflows/.NET%20Core/badge.svg)