Docker, spring boot and mongodb

I wanted to create a docker build to run a spring boot based application along with it’s mongodb database which proved interesting. Here’s what I fouand out.

Dockerfile

To begin with, we need to create a docker configuration file, named Dockerfile. This will be used to create a docker image which we will host a spring boot JAR. Obviously this will require that we create an image based upon a Java image (or create our own). So let’s base our image on a light weight open JDK 1.8 image, openjdk:8-apline.

Below is an example Dockerfile

FROM openjdk:8-alpine
MAINTAINER putridparrot

RUN apk update

ENV APP_HOME /home
RUN mkdir -p $APP_HOME

ADD putridparrot.jar $APP_HOME/putridparrot.jar

WORKDIR $APP_HOME

EXPOSE 8080

CMD ["java","-Dspring.data.mongodb.uri=mongodb://db:27017/","-jar","/home/putridparrot.jar"]

The above will be used to create our image, based upon openjdk:8-apline, we then run an update (in case its required) we create an environment variable for our application folder (we’ll simply install our application into /home, but it could be more specific, such as /home/putridparrot/app or whatever), we then create that folder.

Next we ADD our JAR, so this is going to in essence copy our JAR from our host machine into the docker image so that when we run the image it’ll also include the JAR within that image.

I’m also exposing port 8080 as my JAR will be exposing port 8080, hence when we interact with port 8080 docker will proxy it through to our JAR application.

Finally we add a command (CMD) which will run when the docker image is run. So in this case we run the executable JAR passing in some configuration to allow it to access a mongodb instance (which will be running in another docker instance.

Note: The use if the db host is important. It need not be named db but the name needs to be the same as we’ll be using within the upcoming docker-compose.yml file

Before we move onto the mongodb container we need to try to build our Dockerfile, here’s the commands

docker rmi putridparrot --force
docker build -t putridparrot .

Note: These commands should be run from the folder containing our Dockerfile.

The first command will force remove any existing images and the second command will then build the docker image.

docker-compose.yml

So we’ve created a Dockerfile which will be used to create our docker image but we now want to create a docker-compose file which will be used to run both our newly created image and then a mongodb image and by use of commands such as depends_on and the use of the name of our mongo service (which we used within the JAR execution command). Here’s the docker-compose.yml file

version: "3.1"

services:
  putridparrot:
    build: .
    restart: always
    ports: 
      - "8080:8080"
    depends_on:
      - db

  db:
    image: mongo
    volumes:
      - ./data:/data/db
    ports:
      - "27017:27017"
    restart: always

The first line simply sets the version of the docker-compose syntax, in this case 3.1. This is followed by the services which will be run by docker-compose. The first service listed is our JAR’s image. In fact we do not use the image, we rebuild it (if required) via the build command – this looks for a Dockerfile in the supplied folder (in this case we assume it’s in the same folder as the docker-compose.yml file). We then set up the port forwarding to the docker image. This service depends on a mongodb running, hence the depends_on option.

The next service is our mongodb image. As mentioned previously, the name here can be whatever you want, but to allow our other service connect to it, should be used within our JAR configuration. Think of it this way – this name is the hostname of the mongodb service and docker will handle the name resolution between docker instances.

Finally, we obviously use the mongo image, and we want to expose the ports to allow access to the running instance and also store the data from the mongodb on our host machine, hence allow it to be used when a new instance of this service is started.

Now we need to run docker-compose using

docker-compose up

If all goes well, this will then, possibly build a new image of our JAR, the will bring up the services. As the first service depends_on the second, it will in essence be executed once the mongodb service is up and running, obviously allow it to then connect to the database.

Adding TypeScript to Electron

So we’ve seen that Electron is basically a browser window with integrations which allows us to use JavaScript to interact with it, but as somebody who prefers the type safety features that come with TypeScript, obviously I’d want to integrate TypeScript as well.

If you’ve not got TypeScript installed globally then run

npm install --save-dev typescript

If we take our application from the Getting Started post, lets simply started by adding a tsconfig, just run the following from your project’s root folder

tsc --init

Now change our main.js to main.ts. We’ll obviously need a build step to transpile the TypeScript to JavaScript which can then be uses via Electron, so add the following to the scripts section in package.json

"build": "tsc"

you might also like to either rename the start script or add another script to both build/transpile and run the application, i.e.

"go": "tsc && electron ."

Obviously this will litter your code base with generated .js files, so it’s best to transpile our code to a folder of it’s own, in this case we’ll call it src. Just add the following to the tsconfig.json

"outDir": "./src",

Then change package.json “main” to the following

"main": "./src/main.js",

That’s all there is to it, now we can add some type checking to our code and write TypeScript.

Getting started with Electron

Let’s start by installing the latest version of electron using

  • npm i -D electron@latest

The Writing Your First Electron App goes through the process of setting up your first electron application, I’ll recreate some of these steps below…

  • Create a folder, mine’s test1
  • Run npm init, when asked for the entry point type main.js instead of index.js
  • Add main.js
  • Add index.html
  • Run npm install –save-dev electron
  • Add scripts section to package.json, i.e.
    "scripts": {
      "start": "electron ."
    }
    

Next up let’s add the following to the main.js file

const electron = require('electron');

const { app, BrowserWindow } = require('electron')

function createWindow () {
  // Create the browser window.
  let win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true
    }
  })

  // and load the index.html of the app.
  win.loadFile('index.html')
}

app.on('ready', createWindow)

As you can see, we require electron, along with the app object and BrowserWindow. Next we create a BrowserWindow and load the index.html into it, so let’s supply something for it to load.

Change the index.html to the following

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <title>Hello World!</title>
    <!-- https://electronjs.org/docs/tutorial/security#csp-meta-tag -->
    <meta http-equiv="Content-Security-Policy" content="script-src 'self' 'unsafe-inline';" />
  </head>
  <body>
    <h1>Hello World!</h1>
    We are using node <script>document.write(process.versions.node)</script>,
    Chrome <script>document.write(process.versions.chrome)</script>,
    and Electron <script>document.write(process.versions.electron)</script>.
  </body>
</html>

This demonstrates HTML content as well as using JavaScript within it.

Now run npm start and view your first electron application.

Rust modules

Rust includes a module system for grouping code into logical groups, which may include structs, implementations, other modules etc. Module also give us the ability to manage module code’s visibility.

There’s a couple of ways for declaring our modules.

Option 1

Assuming we’ve used cargo init to create our project or simply laid out our code in the same way, then we’ll have a src folder and within that we’ll create a folder named math which will become our module name. Within math we add a file named mod.rs

src
  |--- math
    |--- mod.rs
  |--- main.rs

So here’s a very basic mod.rs file

pub fn add(a: i32, b: i32) -> i32 {
    a + b
}

and here’s the main.rs file

mod math;

fn main() {
    println!("{}", math::add(1, 2));
}

Option 2

The second option is to name the module file math.rs and this is stored at the same level as the main.rs file, i.e.

src
  |--- math.rs
  |--- main.rs

Nested Modules

We can also nest modules (modules within modules), for example

pub mod nested {
  pub fn add(a: i32, b: i32) -> i32 {
    a + b
  }
}

// in use 

mod math;

fn main() {
    println!("{}", math::nested::add(1, 2));
}

Svn to git migration

I’ve been moving some repositories from SVN to GIT, so for reference, here’s the basic steps…

Note: These steps to not handle changing the author names etc. For a good explanation of this, checkout Migrate to Git from SVN.

Step are, as follows (these steps also assume you’ve create a repository within GIT for you code to migrate to)

  • git svn clone your_svn_repo
  • cd into your newly created folder
  • git remote add origin your_git_repo
  • git push -u origin master

Debugging Rust in Visual Code

I’m using Visual Code a lot for Rust development, so it’d be good to be able to debug a Rust application within it. Simply change the settings.json file, i.e.

File | Preferences | Settings

from the settings.json and now add

"debug.allowBreakpointsEverywhere": true,

Now add the C/C++ Microsoft extension if you’ve not already added it to Visual Code.

Next up, select

Debug | Add Configuration...

and the option

C++ (Windows)

Here’s my resultant launch.json file

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "(Windows) Launch",
      "type": "cppvsdbg",
      "request": "launch",
      "program": "${workspaceFolder}/target/debug/test.exe",
      "args": [],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "environment": [],
      "externalConsole": false
    }
  ]
}

and that’s it, now add a break point to your application, select Debug | Start Debugging (F5) and you’re off and running in the debugger.

println! structs

So we’ve got ourselves a simply little struct

struct Point {
    x: i32,
    y: i32
}

We then decide that we’d like to output the current state of the Point using println!, so we write

fn main() {
    let p = Point { x: 20, y: 3 };

    println!("{}", p);
}

Running this will result in `Point` doesn’t implement `std::fmt::Display` and `Point` cannot be formatted with the default formatter. In fact we do not really need to implement std::fmt::Display, we can just annotate our struct with #[derive(Debug)] and then use the println! formatters (:? or :#?), for example

#[derive(Debug)]
struct Point {
    x: i32,
    y: i32
}

fn main() {
    let p = Point { x: 20, y: 3 };

    println!("{:?}", p);
}

The use of :? will result in the following output Point { x: 20, y: 3 } whereas :#? will display the values on lines of their own (a “prettier” formatter). Both :? and :#? are debug formatters, hence require the annotation #[derive(Debug)] or we can implement std::fmt::Debug, for example

impl std::fmt::Debug for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(f, "(x is {}, y is {})", self.x, self.y)
    }
}

For situations where we simply want to create our own custom display (not just for Debug), then, as per the original error `Point` doesn’t implement `std::fmt::Display`, we would need to implement the std::fmt::Display trait, i.e.

impl std::fmt::Display for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(f, "(x is {}, y is {})", self.x, self.y)
    }
}

This means we no longer requiring the annotation or special formatters, hence our full code will look like this

struct Point {
    x: i32,
    y: i32
}

impl std::fmt::Display for Point {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        write!(f, "(x is {}, y is {})", self.x, self.y)
    }
}

fn main() {
    let p = Point { x: 20, y: 3 };

    println!("{}", p);
}

and as you’d expect our output is now (x is 20, y is 3).

Rust constructors

Rust doesn’t have the concept of a constructor in the sense of C++, C#, Java etc. You create new data structures by simply using the following syntax

struct Point {
   x: i32,
   y: i32
}

let pt = Point { x: 10, y: 20 };

However, by convention you might create an impl to create/initialize your structures. Rust code, by convention suggests such functions be named new. For example

impl Point {
    pub fn new() -> Point {
        Point {
            x: 0, 
            y: 0
        }
    }
}

let pt = Point::new();

Ofcourse, we might declare parameters/arguments on the function just like any other functions.

Basics of unit testing in Rust

I’m messing around with some Rust code at the moment, so expect a few posts in the near future. In this post I’ve going to jump straight into unit testing in Rust.

You don’t need to have a dependency on any unit testing frameworks as Rust has a unit testing framework integrated within it.

Our unit tests can sit along side our existing code using the conditional compilation annotation #[cfg(test)]. Let’s create a simple example test, which assumed we have a Stack implementation to test

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn initial_state() {
        let s: Stack<i32> = Stack::new();
        assert_eq!(s.length, 0);
    }
}

The following line simply allows us to use code from the parent scope (i.e. allows us to use the Stack code).

use super::*;

Next up with have the #[test] annotation which (probably fairly obviously) declares the function initial_state to be a test function.

Ofcourse we need some form of assertation code, hence the assert_eq! macro.

We can also test whether our code panics by placed the #[should_panic] annotation after the #[test] annotation. This denotes that the system under test should panic (similar to exceptions in other languages).

Some times we need to ignore a test, in such cases we can use the #[ignore] annotation

Obviously we need run our tests, we can use cargo for this, simple run

cargo test

We can also run tests in parallel using the -test-threads option, for example if we want test to be run in parallel on two threads, we use

cargo test -- --test-threads=2

See further options around controlling how your tests are run controlling how your tests are run.

netstat

Note: This post is primarily on using netstat on Windows

I’ve been using netstat more lately to keep track on websocket’s being left open etc. and thought it worth creating a post regarding what things mean in nestat, as I’m bound to forget once all the code I’m working on is complete.

We’ll start with a few obvious things by looking at the switch/params available (as taken from netstat -h but included here for completeness)

NETSTAT [-a] [-b] [-e] [-f] [-n] [-o] [-p proto] [-r] [-s] [-t] [interval]

  • -h Display the help
  • -a Displays all connections and listening ports
  • -b Displays the executable involved in creating each connection. This option required elevated permissions, i.e. run as admin
  • -e Displays ethernet statistics (may be combined with -s)
  • -f Displays fully qualified domains names (FQDN) for foreign addresses
  • -n Displays address and port numbers in numerical form
  • -o Displays the owning process id (PID) associated with each connection
  • -p proto Shows connections for the protocol specified by the proto which may be TCP, UDP, TCPv6 or UDPv6. If used with the -s option proto may be IP, IPv6, ICMP, ICMPv6, TCP, TCPv6, UDP or UDCPv6.
  • -r Display the routing table
  • -s Displays per protocol statistics, by default statistics are shown for IP, IPv6, ICMP, ICMPv6, TCP, TCPv6, UDP and UDPv6. The -p option may be used to specify a subset.
  • -t Displays the current connection offload state
  • internal Redisplays the selected data/statistics every interval seconds. Press CTRL+C to stop

Possible states displays might be

  • CLOSED indicates the server has received an ACK signal from the client and is closed
  • CLOSE_WAIT indicates the server has received the first FIN signal, to acknowledge no more data is to be sent from the client, hence the connection is closing
  • ESTABLISHED indicates that the server received a synchronize, SYN, signal. This is only sent in the first packet from the client and the session is established
  • FIN_WAIT_1 indicates the connection is still active but not being used
  • FIN_WAIT_2 indicates the client just received acknoledgement of the first FIN signal from the server
  • LAST_ACK indicates the server is in the process of sending it’s own FIN signal
  • LISTENING indicates the server is ready to accept a connection
  • SYN_RECEIVED indicates the server just received a SYN signal from the client
  • SYN_SEND indicates the connection is open and active
  • TIME_WAIT indicates the client recognizes the connection as active but it’s not currently being used

Obviously if you’ve got grep installed you might prefer to pipe through grep to locate specific data, in PowerShell use Select-String, i.e. the following will run netstat in default mode and then pipe to Select-String which will report lines with port 4000. Not wholly useful in all situations

netstat | Select-String :4000

Within PowerShell on Windows 10 is the Get-NetTCPConnection cmdlet which give us the power of PowerShell for querying the resultant data, for example

Get-NetTCPConnection | ? {$_.State -eq "Listen"}

This will show all results with the state of Listen.

On Windows 7 (without grep) we can use Find and pipe results like this

netstate -an | Find ":4000"

Don’t forget you can pipe this again to find LISTENING state using

netstat -an | Find ":4000" | Find "LISTENING"

What do the results mean?

Obviously the protocol is listed along with the state (possible options listed previously), but we’ll often see local or foreign addresses such as 0.0.0.0 which means the address/port is listening (etc.) on all network interfaces. 127.0.0.1 is ofcourse your local host and processes are listening for connections from the PC itself (i.e. not network). If the address is your local network IP then the port is listening to connections for the local network.

Common use cases

I’m going to stick with netstat (over Get-NetTCPConnection) as this post is, after all, about netstat.

Which software is making a connection to the outside world?

netstat -b

Get a summary of the current number of bytes send/received etc.

netstat -e