Category Archives: C#

Using ThreadStatic and ThreadLocal

First off these are two different objects altogether, ThreadStatic is actually ThreadStaticAttribute and you mark a static variable with the attribute. Whereas ThreadLocal is a generic class.

So why are we looking at both in the one post?

Both ThreadStatic and ThreadLocal are used to allow us to declare thread specific values/variables.

ThreadStatic

A static variable marked with the ThreadStatic attribute is not shared between threads, therefore each thread gets it’s own instance of the static variable.

Let’s look at some code

[ThreadStatic]
static int value;

static void Main()
{
   Task t1 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T1: " + value);
   });
   Task t2 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T2: " + value);
   });
   Task t3 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T3: " + value);
   });

   Task.WaitAll(t1, t2, t3);
}

The output from this (obviously the ordering may be different) is

T3: 1
T1: 1
T2: 1

One thing to watch out for is that if we initialize the ThreadStatic variable, for example if we write the following

[ThreadStatic]
static int value = 10;

you need to be aware that this is initialized only on the thread it’s declared on, all the threads which use value will get a variable initialised with it’s default value, i.e. 0.

For example, if we change the code very slightly to get

[ThreadStatic]
static int value = 10;

static void Main()
{
   Task t1 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T1: " + value);
   });
   Task t2 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T2: " + value);
   });
   Task t3 = Task.Run(() =>
   {
      value++;
      Console.WriteLine("T3: " + value);
   });

   Console.WriteLine("Main thread: " + value);
   
   Task.WaitAll(t1, t2, t3);
}

The output will look something like

Main thread: 10
T2: 1
T1: 1
T3: 1

Finally as, by definition each variable is per thread, operations upon the instance of the variable are thread safe.

ThreadLocal

Like the ThreadStatic attribute, the ThreadLocal class allows us to declare a variable which is not shared between threads, one of the extra capabilities of this class is that we can initialize each instance of the variable as the class the supplied factory method to create and/or initialize the value to be returned. Also, unlike ThreadStatic which only works on static fields, ThreadLocal can be applied to static or instance variables.

Let’s look at the code

static void Main()
{
   ThreadLocal<int> local = new ThreadLocal<int>(() =>
   {
      return 10;
   });

   Task t1 = Task.Run(() =>
   {
      local.Value++;
      Console.WriteLine("T1: " + local.Value);
   });
   Task t2 = Task.Run(() =>
   {
      local.Value++;
      Console.WriteLine("T2: " + local.Value);
   });
   Task t3 = Task.Run(() =>
   {
      local.Value++;
      Console.WriteLine("T3: " + local.Value);
   });

   Task.WaitAll(t1, t2, t3);
   local.Dispose();
}

The output order may be different, but the output will read something like

T2: 11
T3: 11
T1: 11

As you can see, each thread altered their own instance of the thread local variable and more over we were able to set the default value to 10.

The ThreadLocal class implements IDisposable, so we should Dispose of it when we’ve finished with it.

Apart from Dispose all methods and protected members are thread safe.

Waiting for a specified number of threads using the Barrier Class

First off, if you are looking for the definitive online resource on threading in C#, don’t waste your time here. Go to Threading in C# by Joseph Albahari. This is without doubt the best resource anywhere on C# threading, in my opinion (for what it’s worth).

This said, I’m still going to create this post for my own reference

The Barrier Class

The Barrier class is a synchronization class which allows us to block until a set number of threads have signaled the Barrier. Each thread will signal and wait and thus be blocked by the barrier until the set number of signals has been reach at which time they will all be being unblocked, allowing them to continue. Unlike the CountdownEvent, when the Barrier has been signalled the specified number of times it resets and can block again and again awaiting for specified number of threads to signal.

Where would a synchronization class be without a real world analogy – so, let’s assume we are waiting for a known number of race horses (our threads) to arrive at a start barrier (our Barrier class) and once they all arrive we can release them. Only when they’ve all reached the end line (the Barrier class again) do we output the result of the race.

Let’s look at some code (note: this sample can just be dropped straight into a console app’s Program class)

static Barrier barrier = new Barrier(5, b => 
         Console.WriteLine("--- All horse have reached the barrier ---"));
static Random random = new Random();

static void Main()
{
   Task horse1 = Task.Run(() => Race("Horse1", random.Next(1, 1000)));
   Task horse2 = Task.Run(() => Race("Horse2", random.Next(1, 1000)));
   Task horse3 = Task.Run(() => Race("Horse3", random.Next(1, 1000)));
   Task horse4 = Task.Run(() => Race("Horse4", random.Next(1, 1000)));
   Task horse5 = Task.Run(() => Race("Horse5", random.Next(1, 1000)));

   // this is solely here to stop the app closing until all threads have completed
   Task.WaitAll(horse1, horse2, horse3, horse4, horse5);
}

static void Race(string horse, int speed)
{
   Console.WriteLine(horse + " is at the start gate");
   barrier.SignalAndWait();

   // wait a random amount of time before the horse reaches the finish line
   Task.Delay(speed);
   Console.WriteLine(horse + " reached finishing line");
   barrier.SignalAndWait();
}

Maybe I’ve taken the horse race analogy a little too far :)

Notice that the Barrier constructor allows us to add an action which is executed after each phase, i.e. when the specified number of threads have signalled the barrier.

We can add participants to the barrier at runtime, so if we were initially waiting for three threads and these created another three threads (for example), we could then notify the barrier that we want to add the three more threads using the AddParticipant or AddParticipants methods. Likewise we could reduce the participant count using RemoveParticipant or RemoveParticipants.

As you can see from the code above, when a thread wishes to signal it’s completed a phase of it’s processing it calls the SignalAndWait method on the barrier. With, as the name suggests, signals the Barrier class then waits until the Barrier releases it.

More Moq

In the previous post we touched on the fundamentals of using Moq, now we’ll delve a little deeper.

Please note, MOQ version 4.20 has introduced a SponsoreLink which appears to send data to some third party. See discussions on GitHub.

Verifications

So we can verify that a method or property has been called using the following

var mock = new Mock<IFeed>();

FeedViewModel vm = new FeedViewModel(mock.Object);
vm.Update();

mock.Verify(f => f.Update());

This assumes that the vm.Update() calls the IFeed.Update().

But what if we are passing a mock into a view model and we want to verify a method on the mock is called but we don’t care about the specific arguments passed into the method. We can do the following (this example uses the IEventAggregator in Caliburn Micro)

var eventMock = new Mock<IEventAggregator>();

PostListViewModel vm = new PostListViewModel(eventMock.Object);

eventMock.Verify(e => e.Subscribe(It.IsAny<PostListViewModel>()), Times.Once);

In the above example, PostListViewModel’s constructor is expected to call the Subscribe method on the IEventAggregator. The actual implementation passes this into the Subscribe method, but we’ll ignore the argument, except that we’re expecting it to be of type PostListViewModel. The It.IsAny() handles this and the Times.Once simply verifies the Subscribe method was called just once.

Fundamental Moq

Moq is a fabulously simple yet powerful mocking framework.

Please note, MOQ version 4.20 has introduced a SponsoreLink which appears to send data to some third party. See discussions on GitHub.

For those unaware of what a mocking framework are and what they have to offer check out Wikipedia’s Mock Object post.

Here’s the fundamentals for starting to use Moq…

Let’s start with a simple interface for us to mock out

public interface IFeed
{
   string Name { get; set; }
   bool Update(int firstNPosts);
}

Now to mock this interface we can simply create a mock object based on the interface, as follows

IMock<IFeed> mock = new Mock<IFeed>();

obviously this is of little use unless we can set up some behaviours, i.e. what happens when a property or method etc. is called. So in the case of this interface we might want to return a known string for Name like this

IMock<IFeed> mock = new Mock<IFeed>();

mock.Setup(f => f.Name).Returns("SomeUniqueName");

// now use the mock in a unit test
IFeed feed = mock.Object;
Assert.Equal("SomeUniqueName", feed.Name);

The code mock.Object gets the actual interface from the mock, i.e. in this case an IFeed.

For methods we do much the same thing

IMock<IFeed> mock = new Mock<IFeed>();

mock.Setup(f => f.Update(10)).Returns(true);

// now use the mock in a unit test
IFeed feed = mock.Object;
Assert.True(feed.Update(10));

The examples above are a little contrived, after all we obviously are not really going to test that the mock we setup, but instead want to test something that uses the mock.

For example, what about if we have a FeedViewModel which takes an IFeed for display via WPF, something like the following

public class FeedViewModel : PropertyChangedBase
{
   private IFeed feed;
   private bool showProgress;
   private string error;

   public FeedViewModel(IFeed feed)
   {
      this.feed = feed;
   }

   public string Name { get { return feed.Name; } }

   public void Update()
   {
      ShowProgress = true; 
      bool updated = feed.Update(10);
      if(updated)
      {
         Error = "Failed to update";
      }
      ShowProgress = false;
   }

   public bool ShowProgress
   {
      get { return showProgress; }
      set { this.RaiseAndSetIfChanged(ref showProgress, value); }
   }

   public string Error
   {
      get { return error; }
      set { this.RaiseAndSetIfChanged(ref error, value); }
   }
}

Now we can test our FeedViewModel by creating a mock of the IFeed and passing into the FeedViewModel, allowing us to set up mock calls and check our expectations in tests.

The previous examples showed how we can use mocks to setup return values or just take the place of methods when they are called. We may, however, also wish to verify that our code did correctly call our mock objects one or more times – we can do this by marking the set up as Verifiable and then Verify the mocks at the end of the unit test, as per the following

Mock<IFeed> mock = new Mock<IFeed>();
mock.Setup(f => f.Posts).Returns(posts).Verifiable();

FeedViewModel model = new FeedViewModel(mock.Object);
Assert.Equal(posts.Count, model.PostCount);

mock.Verify();

In this example posts is a list of Post objects. Our view model has a property PostCount which uses the list of posts. We want to verify that the Posts property on our (new extended IFeed) was actually called. So we mark it as Verifiable() and at the end of the test we use mock.Verify(); to verify it was actually called.

We can be a little more explicit with our verifications, for example if we expect a method to be called twice, we can verify this as follows

Mock<IFeed> mock = new Mock<IFeed>();
mock.Setup(f => f.Update()).Returns(true);

FeedViewModel model = new FeedViewModel(mock.Object);
// do something on the model which will call the feed Update method

mock.Verify(f => f.Update(), Times.Exactly(2));

So this will fail verification (with a MockException) if the Update() method is not called exactly 2 times.

We also may wish to raise events on the mocked object to see what happens in our object under test, for example assume we have a StatusChanged event on our IFeed which the FeedViewModel should subscribe to and change an Updating property upon the relevant event arguments

Mock<IFeed> mock = new Mock<IFeed>();
FeedViewModel model = new FeedViewModel(mock.Object);

mock.Raise(f => f.StatusChanged += null, 
       new StatusChangedEventArgs(StatusChange.UpdateStarted));
Assert.True(model.Updating);

mock.Raise(f => f.StatusChanged += null, 
        new StatusChangedEventArgs(StatusChange.UpdateEnded));
Assert.False(model.Updating);

In this code we’re raising StatusChanged events on the IFeed mock and expecting to see the view model’s Updating property change.

Dynamic Objects

The IDynamicMetaObjectProvider interface represents an object that can be late bound.

Microsoft supplies to implementations of this interface out of the box. DynamicObject and ExpandoObject.

DynamicObject

The DynamicObject is mean’t to be used as a base class for the user’s own dynamic object and hence has a protected constructor. The idea is that we can create our own dynamic object and handle any attempted access to member properties or methods etc. at runtime.

For example here’s an implementation of a dynamic object which adds (at runtime) a method which takes no arguments and returns a string

public class MyObject : DynamicObject
{
   public override bool TryGetMember(
            GetMemberBinder binder, out object result)
   {
      result = null;
      if (binder.Name == "GetName")
      {
         result = new Func<string>(() => "Hello World");
         return true;
      }
      return false;
   }
}

If we wanted to handle a property named GetName we’d simply return a string, not a function. The TryGetMember is used for methods with no arguments or get properties, when arguments are supplied to a method we should override TryInvokeMember.

ExpandoObject

Unlike DynamicObject an ExpandoObject can be instantiated and then we can add methods and properties to it via it’s IDictionary interfaces.

So for example, let’s create an ExpandoObject and add a new method

dynamic d = new ExpandoObject();

((IDictionary<string, object>)d).Add("GetName", new Func<string>(() => "Hello World"));

Console.WriteLine(d.GetName());

As you can see we must ofcourse declare our variable as dynamic otherwise the use of GetName will cause the compilation error

System.Dynamic.ExpandoObject’ does not contain a definition for ‘GetName’ and no extension method ‘GetName’ accepting a first argument of type ‘System.Dynamic.ExpandoObject’ could be found (are you missing a using directive or an assembly reference?)

We also need to cast the instance to IDictionary<string, object> as access to the Add method needs to go through an explicit interface.

So as you see, we can add to the ExpandoObject at runtime, unlike a DynamicObject derived class where only the derived class would be able to add to the object at runtime (obviously assuming we don’t add methods to allow this ourselves).

Dynamically loading assemblies into Caliburn Micro

I want to be able to load the views/viewmodel assemblies into Caliburn Micro dynamically, i.e. instead of adding references to them to the project….

Well, we’ve got two ways of loading assemblies (that I’m currently aware of). The first is via the Bootstrapper and the second can be handled pretty much anywhere. Both examples below are shown using the DLL name, but ofcourse we could take this further via configuration files, build out own attributes and so on, but I leave that decision to the user.

So through Bootstrapper simply overload SelectAssemblies, for example

protected override IEnumerable<Assembly> SelectAssemblies()
{
   string path = Path.GetDirectoryName(
                    Assembly.GetExecutingAssembly().Location);
   return base.SelectAssemblies().Concat(
         new Assembly[] 
         { 
            Assembly.LoadFile(Path.Combine(path, "MyViews.dll"))
         });
}

However if you’re using a plugin approach, possibly where you allow the user to select a plugin DLL or the likes, then at the point you want to load the assembly you could write something like the following

string path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
Assembly a = Assembly.LoadFile(Path.Combine(path, "Rabbit.Controls.dll"));
AssemblySource.Instance.Add(a);

The key here is the AssemblySource singleton and adding assemblies to the IObservableCollection Instance.

C# code for accessing the network through a proxy server

How to setup proxy authentication and details

ICredentials credentials = CredentialCache.DefaultCredentials;
IWebProxy proxy = new WebProxy(proxyServerHost, proxyServerPort);
proxy.Credentials = credentials;

The code (above) creates a set of credentials, in this case we get the user’s DefaultCredentials and ultimately places the credentials into the IWebProxy.

The IWebProxy is then instantiated with the host name and the port of the proxy server.

Let’s use the proxy

Now to use the web proxy obviously requires the library/framework to support proxies. So this is just one example using the .NET WebClient class.

using (WebClient wc = new WebClient())
{
   if(proxy != null)
   {
      wc.Proxy = proxy;
   }
   byte[] data = wc.DownloadData(url);
}

Code Contracts

Elevator pitch

Code contracts allow us to define preconditions, post conditions and the likes in our code. These are constructs I first read about when doing a (very) little bit of Eiffel development.

Visual Studio additions

Before we get into Code Contracts let’s just note that they are available in .NET 4.5 but we need to enable their usage and here we need to install a couple of additions to Visual Studio.

So if you’re running Visual Studio 2012 goto Tools | Extensions and Updates and search online for Code Contract Tools – install this. Then find Code Contracts Editor Extensions VS2012 and install this. You’ll have to restart Visual Studio to see the new installs working. So go ahead and do that.

If Visual Studio 2013 I just install Code Contract Tools.

This will install some addition static analysis tools etc. which we’ll look at later, for now let’s code…

Using Code Contracts

Predconditions

Often in our code we’ll have methods which check the arguments passed to a method or possibly check class wide variables have been set etc. Usually the code might take the following form

public void Analyze(string data)
{
   if(data == null)
      throw new ArgumentNullException("data");

   // do something
}

Well first off we can now replace this with a Code Contract precondition. The precondition ofcourse states that for this method to run, this or these preconditions must be met.

public void Analyze(string data)
{
   Contract.Requires<ArgumentNullException>(line != null)
}

We could have specified the same precondition without the ArgumentNullException and a ContractException would be thrown instead.

Postconditions

A postcondition specifies the state of a method when it ends. For example, we can declare that we’ll never return a null by declaring the following contract

public static string Analyze(string line)
{
   Contract.Requires<ArgumentNullException>(line != null);
   Contract.Ensures(Contract.Result<string>() != null);
   
   // do something
   
   return result;
}

Now if we actually did attempt to return null we’d get a ContractException.

Assumptions

So, we can define pre and postconditions, but also we can define assumptions. Basically let’s assume we have our Analyze method (above) but without the pre or postconditions and we want to declare an assumption that the result from Analyze will not be null, we could write

public string Analyze(string data)
{
   // do something

   return result;
}

static void Main(string[] args)
{
   string result = Analyze("Some Text");

   Contract.Assume(result != null);

   // do something with result
}

Obviously if result is set to null, our contract is violated and the ContractException will be thrown.

Object Invariants

An object invariant basically defines the state of an object. So for example, let’s assume we have a Person class as below

public class Person
{
   public Person(string firstName, string lastName)
   {
      FirstName = firstName;
      LastName = lastName;
   }

   public string FirstName { get; set; }
   public string LastName { get; set; }
}

It may be that we want to define a contract which states that FirstName and LastName are not null. We could add preconditions to the constructor – but what if we also wanted to define a contract that says FirstName and LastName are never null, not just within the contructor’s scope.

Now we can add a ContractInvariantMethod as below

public class Person
{
   public Person(string firstName, string lastName)
   {
      FirstName = firstName;
      LastName = lastName;
   }

   public string FirstName { get; set; }
   public string LastName { get; set; }

   [ContractInvariantMethod]
   private void ContractInvariant()
   {
      Contract.Invariant(FirstName != null && LastName != null);
   }
}

Note: We could have had the left and right parts of the && in different Contract.Invariant declarations if we wished, so basically we can have multiple Contract.Invariant in this method.

The method name can be anything you like and can only contain Contract.Invariant statements. A compile time error will occur if you have other code in this method.

Now when we create a Person object if either FirstName or LastName are set to null a ContractException will occur, but better still let’s assume we add a new method to the class which sets one of these variables to null. The ContractInvariantMethod will be called and the contract will be violated leading to a ContactException.

Contracts on interfaces

An interesting use of contracts is with interfaces. Basically if we define an interface it would be nice if we could declare the contract definitions against the interface. Then any implementation could “inherit” the contract definitions.

So let’s look at our Person class again but now add an interface and define some contracts on the interface

[ContractClass(typeof(IPersonContract))]
public interface IPerson
{
   string FirstName { get; set; }
   string LastName { get; set; }			
}

[ContractClassFor(typeof(IPerson))]
internal abstract class IPersonContract : IPerson
{
   public string FirstName
   {
      get 
      { 
         Contract.Ensures(Contract.Result<string>() != null);
         return default(string); 
      }
      set { Contract.Requires(value != null); }
   }

   public string LastName
   {
      get 
      { 
         Contract.Ensures(Contract.Result<string>() != null);
         return default(string); 
      }
      set { Contract.Requires(value != null); }
   }
}

public class Person : IPerson
{
   public Person(string firstName, string lastName)
   {
      FirstName = firstName;
      LastName = lastName;
   }

   public string FirstName { get; set; }
   public string LastName { get; set; }
}

So the interface is a standard interface except that we are declaring (using the ContractClass attribute) that the IPersonContract class includes contract definitions for the interfaces. Likewise on the IPersonContract abstract class. Yes, it’s mean’t to be abstract and implement the interface. Now when we return values, just use default(<type>) so all will compile and add your pre or postconditions. Unfortunately it doesn’t look like we can add a ContractInvariantMethod method to this abstract class (or at least we can but it doesn’t appear to do anything in the current release of .NET).

ContractAbbreviator

The ContractAbbreviatorAttribute is used on a method to denote it contains contract data which, when the method is called from another method in essence runs the contract checks in that method – eh ? It’s a bit like creating a C style macro and having it inlined in a method – eh ?

It’s a was to reuse contracts.

Okay let’s see an example

public class Person
{
   public string FirstName { get; set; }
   public string LastName { get; set; }

   [ContractAbbreviator]
   private void Validate()
   {
      Contract.Ensures(FirstName != null);
      Contract.Ensures(LastName != null);
   }

   public void Amend()
   {
      Validate();
   }
}

Now when we run Amend (with FirstName and/or LastName set to null) the exception occurs from the Amend method, as if the Contract.Ensure’s replaced the Validate call.

Based upon the above information it would come as no surprise to find out that when the ContractAbbreviator attribute is used on a method (and ofcourse the method is called from another method), the compiler does indeed place the contract code into the calling method and replaces the instruments within the Validate method (in the above example) with a nop. When the attribute is not included we really are calling another method.

Obviously this means we can reuse the Amend method now in more than one method (or more specifically the contracts contained within the method), so we’ve created a reusable set of contracts. If you remove the ContractAbbreviator attribute then run as previously suggested, the validation failure will occur in the Validate method.

Visual Studio Plugin

At the start of this post we installed a couple of code contract specific plugins (in VS2012 and only one in VS2013). These added the property editor into the project properties for enabling etc. Code Contracts. But they also installed editor warnings.

If you check the “Perform Static Contract Checking” option in the Code Contract property page, then rebuild your project. Static analysis will take place and may display warnings. For example if your code looked like this

public void Analyze(string data)
{
   Contract.Requires<ArgumentNullException>(line != null)
}

static void Main(string[] args)
{
   Analyze(null);
}

We would get warnings stating CodeContracts: requires is false: line != null. Along with a red squiggly underline of the code Analyze(null) and a tooltip popup showing the warning message.

So giving us an indication early on of possible contract violations.

References

Code Contracts User Manual
Code Contracts

Some Linq Patterns

What follows is a list of a few patterns for specific tasks using LINQ

Combining two lists

We can combine two lists, where duplicates are removed using

var a = new List {0, 2, 3, 4, 5, 6};
var b = new List { 0, 1, 3, 4, 6, 6 };

var union = a.Union(b);
[/code]

Find the common items in both lists

var a = new List<int> {0, 2, 3, 4, 5, 6};
var b = new List<int> { 0, 1, 3, 4, 6, 6 };

var intersection = a.Intersect(b);

we can also use the lambda expression join

var l1 = new List<int> {1, 2, 4, 5};
var l2 = new List<int> { 0, 2, 4, 6 };

var r = from a in l1 join b in l2 on a equals b select a;

Find the differences between two lists

This will find the items in the first list which are not in the second list

var listA = new List<int> {0, 2, 3, 4, 5, 6};
var listB = new List<int> { 0, 1, 3, 4, 6, 6 };

var differences = from a in listA where listB.All(i => i != a) select a;

Combining every item in one list with every item in another list

Here we’re going to do a cross join, where the output enumerable is a made up of each element from the first list, combined with each element of the second list

var l1 = new List<int> {1, 2};
var l2 = new List<int> { 0, 2 };

var crossjoin = from a in l1
       from b in l2
       select new {a, b};

The output would be

1 0
1 2
2 0
2 2

I’ll add more as and when I remember to add them.

Creating a custom Linq Provider

To create a custom LINQ provider we need to implement the interfaces, IQueryable or IOrderedQueryable and IQueryProvider.

This is a summary of several posts/blogs I’ve read on the subject and source code available on github etc. See References below.

IQueryable<T>

To implement a minimal LINQ provider we need to implement IQueryable<>. This interface inherits from IEnumerable. The actual IQueryable methods are minimal

ElementType
Expression
Provider

IOrderedQueryable<T>

If we want to support the sorting query operators, then we need to implement IOrderedQueryable<T>.

IOrderedQueryable inherits from IEnumerable, IEnumerable<T>, IOrderedQueryable, IQueryable, and IQueryable<T>.

We can implement an IOrderedQueryable that’s reusable for most situations, as follows

public class Queryable<T> : IOrderedQueryable<T>
{
   public Queryable(IQueryContext queryContext)
   {
      Initialize(new QueryProvider(queryContext), null);
   }

   public Queryable(IQueryProvider provider)
   {
     Initialize(provider, null);
   }

   internal Queryable(IQueryProvider provider, Expression expression)
   {
      Initialize(provider, expression);
   }

   private void Initialize(IQueryProvider provider, Expression expression)
   {
      if (provider == null)
         throw new ArgumentNullException("provider");
      if (expression != null && !typeof(IQueryable<T>).
             IsAssignableFrom(expression.Type))
         throw new ArgumentException(
              String.Format("Not assignable from {0}", expression.Type), "expression");

      Provider = provider;
      Expression = expression ?? Expression.Constant(this);
   }

   public IEnumerator<T> GetEnumerator()
   {
      return (Provider.Execute<IEnumerable<T>>(Expression)).GetEnumerator();
   }

   System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
   {
      return (Provider.Execute<System.Collections.IEnumerable>(Expression)).GetEnumerator();
   }

   public Type ElementType
   {
      get { return typeof(T); }
   }

   public Expression Expression { get; private set; }
   public IQueryProvider Provider { get; private set; }
}

Note: The IQueryContext is not part of the LINQ interfaces but one I’ve created to allow me to abstract the actual “custom” part of the code and allow maximum reuse of default functionality.

IQueryProvider

The IQueryProvider basically requires two methods (a generic and non-generic version of both) to be implemented. Obviously the generic versions are for strongly typed and the non-generic are loosely typed.

CreateQuery

The CreateQuery method is used to create a new instance of an IQueryable based upon the supplied expression tree.

Execute

The Execute method is used to actually execute a query expression, i.e. in a custom provider we will get the data from the datasource, for example a webservice or database etc. and it’s in the Execute method that we both get the data and do any conversion to, say SQL or the likes (if the LINQ query ultimately queries some other system.

An example of an implementation for an IQueryProvider might look like this…

public class QueryProvider : IQueryProvider
{
   private readonly IQueryContext queryContext;

   public QueryProvider(IQueryContext queryContext)
   {
      this.queryContext = queryContext;
   }

   public virtual IQueryable CreateQuery(Expression expression)
   {
      Type elementType = TypeSystem.GetElementType(expression.Type);
      try
      {
         return                
            (IQueryable)Activator.CreateInstance(typeof(Queryable<>).
                   MakeGenericType(elementType), new object[] { this, expression });
      }
      catch (TargetInvocationException e)
      {
         throw e.InnerException;
      }
   }

   public virtual IQueryable<T> CreateQuery<T>(Expression expression)
   {
      return new Queryable<T>(this, expression);
   }

   object IQueryProvider.Execute(Expression expression)
   {
      return queryContext.Execute(expression, false);
   }

   T IQueryProvider.Execute<T>(Expression expression)
   {
      return (T)queryContext.Execute(expression, 
                 (typeof(T).Name == "IEnumerable`1"));
   }
}

Note: Again, the IQueryContext is not part of the LINQ interfaces but one I’ve created to allow me to abstract the actual “custom” part of the code and allow maximum reuse of default functionality.

The TypeSystem class is taken from the post Walkthrough: Creating an IQueryable LINQ Provider

IQueryContext

The above shows some LINQ interfaces and some sample implementations. I’ve pointed out that IQueryContext is not part of LINQ but is instead something I’ve created (based upon reading various other implementations) to allow me to abstract the actual LINQ provider code specific to my provider’s implementation. Ofcourse we could have derived from QueryProvider, but for now we “plug-in” the data context instead. To change the implementation to derive from QueryProvider simple remove the IQueryContext (also from the Queryable implementation) and override the Execute methods.

For now I’ll continue this post using the IQueryContext, so here’s the interface

public interface IQueryContext
{
   object Execute(Expression expression, bool isEnumerable);
}

Implementing the IQueryContext

Whether the implementation of the actual Execute code (on the IQueryProvider) is abstracted into the IQueryContext or implemented within an alternate implementation of IQueryProvider. This is where the fun of actually “running” the LINQ query against your custom provider takes place.

When writing something like this…

var ctx = new Queryable<string>(new CustomContext())

var query = from s in ctx where s.StartsWith("T") select s;

what we are really doing is creating the query. Hence the CreateQuery methods on the IQueryProvider are called, but the actual data source or whatever supplies the data for your custom LINQ provider should not be called until we reach the execution phase, this is known as deferred execution. The execution phase takes place when we enumerate over the query or call methods, such as Count() etc. against a query.

foreach (var q in query)
{
   Console.WriteLine(q);
}

So at the point that we GetEnumerator implicitly via the foreach loop, that’s when the Execute method is called.

Executing the query

So, the Execute method is called and we will have an expression tree defined by LINQ supplied as an argument. We now need to translate that query into something useful, i.e. turn it into an SQL query for a custom database LINQ provider and then get the data for this query, or get data from a webservice and allow the query to be executed against the returned data etc.

As the actual decoding of the Expression is a fairly large subject in itself, I’ll leave that for another post, but suffice to say, there’s a lot we need to implement to duplicate some if not all of the “standard” LINQ query operators etc.