Category Archives: Ubuntu

MongoDB revisited

As I’m busy setting up my Ubuntu server, I’m going to revisit a few topics that I’ve covered in the past, to see whether there are changes to working with various servers. Specifically I’ve gone Docker crazy and want to run these various server in Docker.

First up, let’s see what we need to do to get a basic MongoDB installation up and running and the C# client to access it (it seems some things have changes since I last did this).

Getting the server up and running

First off we’re going to get the Ubuntu server setup with an instance of MongoDB. So let’s get latest version on mongo for Docker

docker pull mongo:latest

this will simply download the latest version of the MongoDB but not run it. So our next step is to run the MongoDB Docker instance. By default the port MongoDB uses is 27017, but this isn’t available to the outside world. So we’re going to want to map this to a port accessible to our client machine(s). I’m going to use port 28000 (there’s no specific reason for this port choice). Run the following command from Ubuntu

docker run -p 28000:27017 --name my-mongo -d mongo

We’ve mapped MongoDB to the previously mentioned port and named the instance my-mongo. This will run MongoDB in the background. We can now look to write a simple C# client to access the instance.

Interactive Shell

Before we proceed to the client, we might wish to set-up users etc. in MongoDB and hence run its shell. Now running the following

docker exec -t my-mongo mongo

Didn’t quite work as expected, whilst I was placed inside the MongoDB shell, commands didn’t seem to run.

Note: This could be something I’m missing here, but when pressing enter, the shell seemed to think I was about to add another command.

To work with the shell I found it simpler to connect to the Docker instance using bash, i.e.

docker exec -t my-mongo bash

then run

mongo

to access the shell.

I’m not going to set up any users etc. at this point, we’ll just used the default setup.

Creating a simple client

Let’s fire up Visual Studio 2015 and create a console application. Then using NuGet add the MongoDB.Driver by MongoDB, Inc. Now add the following code to your Console application

public class Person
{
   public ObjectId Id { get; set; }
   public string FirstName { get; set; }
   public string LastName { get; set; }
   public int Age { get; set; }
}

class Program
{
static void Main(string[] args)
{
   var client = new MongoClient("mongodb://xxx.xxx.xxx.xxx:28000");
   var r = client.GetDatabase("MyDatabase");
   var collection = r.GetCollection<Person>("People");
   collection.InsertOne(new Person 
   { 
      FirstName = "Scooby", 
      LastName = "Doo", 
      Age = 27 
   });
}

Obviously replace the xxx.xxx.xxx.xxx with the IP address of your server (in my case my Ubuntu server box), the port obviously matches the port we exposed via Docker. You don’t need to “create” the database explicitly via the shell or a command, you can just run this code and it’ll create MyDatabase then the table People and then insert a record.

Did it work?

Hopefully your Console application just inserted a record. There should have been no timeout or other exception. Ofcourse we can use the Console application, for example

var client = new MongoClient("mongodb://xxx.xxx.xxx.xxx:28000");
var r = client.GetDatabase("MyDatabase");
var collection = r.GetCollection<Person>("People");
foreach (var p in collection.FindSync(_ => true).ToList())
{
   Console.WriteLine($"{p.FirstName} {p.LastName}");                
}

I’m using the synchronous methods to find and create the list, solely because my Console application is obviously pretty simple, but the MongoDB driver library offers Async versions of these methods as well.

The above code will write out Scooby Doo as the only entry in our DB, so all worked fine. How about we do the same thing using the shell.

If we now switch back to the server and if its not running, run the MongoDB shell as previously outlined. From the shell run the following

use MyDatabase
db.People.find()

We should now see a single entry

{ 
  "_id" : ObjectId("581d9c5065151e354837b8a5"), 
  "FirstName" : "Scooby", 
  "LastName" : "Doo", 
  "Age" : 27 
}

Just remember, we didn’t set this instance of MongoDB up to use a Docker Volume and hence when you remove the Docker instance the data will disappear.

So let’s quickly revisit the code to run Mongo DB within Docker and fix this. First off exit back to the server’s prompt (i.e. out of the Mongo shell and out of the Docker bash instance).

Now stop my-mongo using

docker stop my-mongo

You can restart mongo at this point using

docker start my-mongo

and your data will still exist, but if you run the following after stopping the mongo instance

docker rm my-mongo

and execute Mongo again the data will have gone. If we add a volume command to the command line argument, and so we will execute the following

docker run -p 28000:27017 -v mongodb:/data/mongodb --name my-mongo -d mongo

the inclusion of the /v will map the mongodb data (/data/mongodb) to the volume on the local machine named mongodb. By default this is created in /var/lib/docker/volumes, but ofcourse you could supply a path to an alternate location

Remember, at this point we’re still using default security (i.e. none), I will probably create a post on setting up mongo security in the near future

Building a Linux based NAS

I’ve put together a Linux (Ubuntu) based NAS device. I’m going to list the steps I took to get it all up and running. As a disclaimer though I need to state I am not a Linux expert, so don’t take this as the perfect solution/setup.

Starting point

First off, I actually had a Windows Home Server NAS device, but the hard disk controller died. This means I have a bunch of NTFS formatted drives with lots of media files on and so I need the new NAS to be able to use these drives.

I bought myself a DELL PowerEdge T20 Tower Server which will act as my NAS, it’s a really well spec’d and priced piece of hardware, although a little larger than I would normally want for a NAS and without any easy to access hard drive bays – basically it’s a small tower cased computer, but for the price it’s superb.

Next up I’ve installed Ubuntu Server. So obviously this is so much more than just a NAS, but I’ll concentrate on creating that side of things in this post.

Mounting my drives

After all the hard drives were connected I needed to mount them. As stated, these are already formatted to NTFS.

The first thing we need to do is create some folders to act as the mount points using

mkdir /mnt/foldername

Next up we need to actually mount the drives and assign them to the mount point, for this we’ll use

sudo mount -t ntfs /dev/sda2 /mnt/foldername

Obviously the /dev/sda2 needs to be set to your drive. The easiest way to check what devices/drives exists is use

sudo fdisk -l

or if, like me, you’re looking for the NTFS drives you can use

sudo fdisk -l | grep NTFS

Mounting the drives like this, is transient and the mount will be lost when the NAS reboots, so we need to set them up to “automount” at startup. To do this we need to edit the fstab file, i.e.

sudo nano /etc/fstab

Whilst we can use the /dev/sda2 way of mounting, it’s far better to use the device’s UUID as this will allow us to hot-swap the drives or the like. To find the UUID of your drives simply run

sudo blkid

Now in the fstab file we’ll add lines like this

UUID=123456789ABCDEF / ntfs defaults 0 2

Using Samba to access the drives

Whilst the previous steps have mounted the drives which are now accessible via the server, we want these drives accessible from Windows and the internal network. Samba allows us to expose the drives, or more specifically their folders to the LAN.

We need to edit the smb.conf file, so run

sudo nano /etc/samba/smb.conf

Here’s an example of an entry for one of our shared drives/folders. In this case assume “share-name” to be the name you want to see in your network browser. The folder-path should point to your mounted drive and or any folders you want to expose

[share-name]
   path = /path/folder-path
   read only = yes
   browseable = yes

In this case, we’re stating the share can be browsed for (i.e. it’ll be visible in Windows Explorer when we access the server using \\mynas). In this case I’ve also set the folder to be read only. We add an entry for each folder/drive we want to expose.

Users and Permissions

As I have family members each having their own user space on the NAS, I need to create users – this is simply a case of using

sudo useradd username

replacing username with the name of the user. We can list all users using

cut -d: -f1 /etc/passwd

Now the useradd command simply adds a user, this does not create home folders or other private space for the user. As my WHS was set up with folders for each user, we don’t need to use the Linux /home folders, but just for completeness. When we added the users Ubuntu adds information to the /etc/passwd file to show where the user’s home folder is, but it didn’t create the folders, so we’ll create the home directories manually. Create a home folder for each of your user’s using

sudo mkdir /home/username

Now let’s change the ownership of the folders to each user

sudo chown -R username /home/username

to prove everything is working as we want we can run the command

ls -l

from the home folder and it’ll list the owners of each folder

Obviously at this point we can go back to the smb.conf and expose the user’s home folders to Windows and the LAN.

Once we’ve created the Samba folder configurations for the user’s folder, we’ll probably want to look at setting permissions on the folders. We go back to editing smb.conf

[share-name]
   path = /path/folder-path
   read only = yes
   browseable = yes
   write list = bob
   create mask = 0755

We can set up a write list of all user’s who can write to the share (we can use @group, replacing the word group with the actual group name of users as well). Notice the write list will give write access regardless of the read only option.

We can also define a create mask for files and directories (using standard Linux bitwise flags).

How about having the NAS start-up and shutdown automatically

It’d be cool if we can try and conserve energy a little by turning the NAS off when its less likely to be used and back on when it’s most likely to be used.

I haven’t yet implemented this, but have tried the steps in the following document and they worked perfectly, so check out

https://www.mythtv.org/wiki/ACPI_Wakeup#Using_.2Fsys.2Fclass.2Frtc.2Frtc0.2Fwakealarm

Changing the host name

A slight detour here, but when I installed Ubuntu I chose a host name, which on reflection I wanted to change to match the name the old WHS had as this allowed the family to use the new NAS as if it was the old one (i.e. not have to get them to change to using the new NAS name etc.).

We can find the host name using

hostname

and we can also use the same command to set a new hostname using

sudo hostname MyHostName

this is a temporary change, so to make this permanent (i.e. exist after a reboot) we use

sudo nano /etc/hostname
sudo nano /etc/hosts

References

https://help.ubuntu.com/community/Samba/SambaServerGuide
http://askubuntu.com/questions/113733/how-do-i-correctly-mount-a-ntfs-partition-in-etc-fstab
http://askubuntu.com/questions/205841/how-do-i-mount-a-folder-from-another-partition