Monthly Archives: June 2016

Embedded NoSQL with LiteDB

I was looking for a simple file based object/document database and came across LiteDB. This gives similar functionality to MongoDB.

If you’re looking for good documentation on LiteDB, I would suggest going to Getting Started. I’ll undoubtedly duplicate some/much of what’s written there in this post which is mainly aimed at reminding me how to get up and running with LiteDB.

Getting started

It’s so easy to get started with LiteDB. Let’s first define an object model for some data that we might wish to store.

public class Artist
{
   public string Name { get; set; }
   public IList<string> Members { get; set; }
}

public class Album
{
   public int Id { get; set; }
   public Artist Artist { get; set; }
   public string Name { get; set; }
   public string Genre { get; set; }
}

We need an Id property or a property marked with a BsonId attribute on our POCO object. Whilst we can get away without this for some operations, updates (for one will fail) without the Id property/BsonId.

To use LiteDB, simply install the nuget package LiteDB and now here’s the bare minimum to create/open a LiteDB database file and get a reference to the collection for our CRUD operations

using (var db = new LiteDatabase("Albums.db"))
{
   var albums = db.GetCollection<Album>("albums");
   // now we can carry out CRUD operations on the data
}

Easy enough. The LiteCollection returned from GetCollection allows us to to work on our data in a very simple manner.

For example, let’s insert a new album

albums.Insert(
   new Album
   {
      Artist = new Artist
      {
         Name = "Led Zeppelin",
         Members = new List<string>
         {
            "Jimmy", "Robert", "JP", "John"
         }
      },
      Name = "Physical Graffiti",
      Genre = "Rock"
   });

How about retrieving all the data, we can use

var results = albums.FindAll();

We can also query for specific data using a predicate, for example

var r = albums.Find(a => a.Artist.Name == "Alice Cooper");

Note: in this example, the Artist.Name property has not been indexed, so performance would be improved by setting an index on the data.

To update we need to get the instance from LiteDB (or at least know the Id) and then makes the changes as follows

var zep = albums.Find(a => a.Artist.Name == "Led Zeppelin").First();
zep.Artist.Members[2] = "John Paul Jones";
albums.Update(zep);

Obviously in the above we’re assuming there’s at least one item (by calling First() as obviously there might be zero, one or multiple returns), the key is how we simply call the Update method.

Deleting all items from a database can be achieved, obviously by deleting the DB file or using

albums.Delete(Query.All());

or we can delete an individual item by calling

// use the Id property 
albums.Delete(album.Id); 

// or

// use a query type syntax
albums.Delete(x => x.Artist.Name == "Led Zeppelin"); 

// or

// using the Query syntax similar to deleting all items
albums.Delete(Query.EQ("Artist.Name", new BsonValue("Led Zeppelin")));

Obviously the Query syntax seems a little over the top for most things, but offers more query like syntax if required.

And there’s more

Okay I’m not intending to document everything in this single post but I have to just touch on transactions. It’s great to see the ability to use ACID transactions for use with LiteDB.

So to ensure we only commit when all operations are successful we simply use

var albums = db.GetCollection<Album>("albums");

db.BeginTrans();
// multiple operations against LiteDB
db.Commit();

Multiple Inserts

It seems (from the documentation etc.) that when we carry out an insert, LiteDB creates an “auto-transaction” around the insert for us (see Transactions and Concurrency. As such we should be aware that if we’re creating many inserts (for example from a list of objects) then it’s best from a performance point of view to not call insert multiple times.

Instead either using the IEnumerable overload of the Insert method or wrap all inserts within a transaction (I’m assuming this would also work – not yet tested).

This make sense, but can be easily forgotten.

Powershell ForEach-Object gotcha!

Are you serious Powershell !!!!?

Okay, now I’ve got that out of the way, I wrote a little Powershell command to delete the *.deleteme files as well as the folders they refer to that Nuget occasionally seems to leave behind when updating packages.

So I developed the script on the command line and all looked good so decided to turn it into a function to add to $profile.

The function kept failing with the error “Supply values for the following paraneters: Process[0]”.

This appears because my foreach looked like this

foreach 
{ 
   // some functionality
}

and even though we think that the curly brace is the block (as it is in C#) it appears we need to place the first curly brace on the same line as the foreach, thus

foreach { 
   // some functionality
}

Here's the script, when it was completed

[code language="csharp"]
function Delete-DeleteMeAndFolder
{
   ls "*.deleteme" | 
   foreach { 
      rm -Recurse -Force $_.Name.Replace(".deleteme", "")
      del $_.Name
   }
}

Running Powershell commands from you application

So, I’ve written some Powershell cmdlet’s and occasionally, might want to either use them or use Powershell built-in cmdlet’s in one of my applications. Let’s look at making this work.

Hosting Powershell

To host Powershell we include the System.Management.Automation reference (locally or from Nuget) and we can simply use

using(var ps = Powershell.Create())
{
   // add command(s) and invoke them
}

to create a Powershell host.

Calling built-in commands/cmdlet’s

As you can see, creating the Powershell host was easy enough, but now we want to invoke a command. We can write something like

ps.AddCommand("Get-Process");

foreach (var r in ps.Invoke())
{
   Console.WriteLine(r);
}

Invoke will return a collection of PSObjects, from each of these objects we can get member info, properties etc. but also the actual object returned from GetProcess, in this case a Process object. From this we can do the following (if required)

foreach (var r in ps.Invoke())
{
   var process = (Process) r.BaseObject;
   Console.WriteLine(process.ProcessName);
}

Passing arguments into a command

When adding a command, we don’t include arguments within the command string, i.e. ps.AddCommand(“Import-Module MyModule.dll”) is wrong. Instead we pass the arguments using the AddArgument method or we supply key/value pair arguments/parameters using AddParameter, so for example

ps.AddCommand("Import-Module")                 
  .AddArgument("HelloModule.dll");

// or

ps.AddCommand("Import-Module")                 
  .AddParameter("Name", "HelloModule.dll");

So the parameter is obviously the switch name without the hyphen/switch prefix and the second value is the value for the switch.

Importing and using our own Cmdlet’s

So here’s a simply Cmdlet in MyModule.dll

[Cmdlet(VerbsCommon.Get, "Hello")]
public class GetHello : Cmdlet
{
   protected override void ProcessRecord()
   {
      WriteObject("Hello World");
   }
}

My assumption was (incorrectly) that running something like the code below, would import my module then run the Cmdlet Get-Hello

ps.AddCommand("Import-Module")                 
  .AddArgument("HelloModule.dll");

ps.Invoke();

ps.AddCommand("Get-Hello");

foreach (var r in ps.Invoke())
{
   Console.WriteLine(r);
}

in fact ProcessRecord for our Cmdlet doesn’t appear to get called (although BeginProcessing does get called) and therefore r is not going to contain any result even though it would appear everything worked (i.e. no exceptions).

What seems to happen is that the Invoke method doesn’t (as such) clear/reset the command pipeline and instead we need to run the code Commands.Clear(), as below

ps.AddCommand("Import-Module")                 
  .AddArgument("HelloModule.dll");

ps.Invoke();
ps.Commands.Clear();
// the rest of the code

An alternative to the above, if one is simply executing multiple commands which have no reliance on new modules or a shared instance of Powershell, might be to create a Poweshell object, add a command and invoke it and then create another Powershell instance and run a command and so on.

With a Powershell 3 compatible System.Management.Automation we can use the following. AddStatement method

ps.AddCommand("Import-Module")                 
  .AddArgument("HelloModule.dll");
  .AddStatement()
  .AddCommand("Get-Hello");

foreach (var r in ps.Invoke())
{
   Console.WriteLine(r);
}

Importing modules into a runspace

To share the importing of modules among Powershell host instances, we could, instead look to create a runspace (which is basically an environment space if you like) and import the module into the runspace, doing something like this

var initial = InitialSessionState.CreateDefault();
initial.ImportPSModule(new [] { "HelloModule.dll" });

var runSpace = RunspaceFactory.CreateRunspace(initial);
runSpace.Open();

using(var ps = PowerShell.Create())
{
   ps.Runspace = runSpace;

   ps.AddCommand("Get-Hello");

   foreach (var r in ps.Invoke())
   {
      Console.WriteLine(r);
   }
}

In the above code, we import our modules into the initial session, from this we create our runspace and then we associated that with the our Powershell host(s) and reuse as required.

References

Windows PowerShell Host Quickstart

Environment variables in Powershell

This just demonstrates how, when you’re used to the standard Windows command prompt, Powershell can (at times) bite you.

So I was getting some Java code configured and noticed the ant script listed a different JAVA_HOME to the one I expected. In a standard Windows command prompt I would type

echo %JAVA_HOME%

and ofcourse, expect to see the folder location. So this is what I did in Powershell and instead I just got the result %JAVA_HOME% written out.

Powershell does things differently. In this case to output environment variables we need to use

echo $Env:JAVA_HOME

Doh!

To set an environment variable in Powershell we simply type

$env:JAVA_HOME="C:\Program Files\Java\jdk1.6.0_45"

References

See Windows PowerShell Tip of the Week