Monthly Archives: December 2017

Record & Playback HTTP requests using Fiddler

I’ve written about the Scotch library previously, but I have scenarios where I cannot intercept HTTP requests directly within my code. i.e. SOAP generated client code handles this for us.

So what we want to do is use Fiddler initially to intercept responses for our service calls, save these and then replay them offline.

Fiddler comes with a great feature which allows us to intercept calls and automatically respond with previously saved data call AutoResponder.

Just select the AutoResponder tab in Fiddler.

All we need to do now, is add a rule, here we can create rules for specific URL patterns, i.e. just intercept some calls or all URLs. We can then assign a file (likely our previously saved intercept response files) and now we have Fiddler acts as the service by responding to requests with our locally saved files.

Don’t forget to “Enable rules” when you’ve saved all your rules and you probably want “Unmatched requests passthrough” also checked to allow other HTTP requests to pass through the rules.

If you’re using a .NET client application, remember you’ll need the following within your App.config,

<system.net>
   <defaultProxy>
      <proxy bypassonlocal="false" usesystemdefault="true" />
   </defaultProxy>
</system.net>

Here’s an example of a simple bit of .NET code that can be used to check everything is working, just add autoresponder rules for the URL myurl/test

var client = new HttpClient();
var r = client.GetStringAsync(new Uri(
   "http://myurl/test")).Result;

References

AutoResponder Reference

Trying to get my head around CQRS

Let’s start by decoding the acronym CQRS – this stands for Command and Query Responsibility Segregation. From this we can see the intent is to segregate our code into command and query responsibilities.

So what’s a command in this context and what’s a query?

  • A command is “any method that mutates state” (analogous to a setter type operation)
  • A query is “any method that returns a value” (analogous to a getter type operation)

It’s probably pretty obvious, but we can liken the command to the CRUD operations, CREATE, UPDATE and DELETE, whereas a RETRIEVE would map to a query (probably self-explanatory).

There’s nothing too complicated about this except in terms of how we’d code such things. The idea would be to split our code into two classes whereby we might have something like this

public class EmployeeCommands
{
   public void CreateEmployee(int employeeId /* other params */) {}
   public void UpdateEmployee(int employeeId /* other params */) {}
   public void DeleteEmployee(int employeeId) {}
}

public class EmployeeQueries
{
   public Employee GetEmployee(int employeeId) { /* implementation */ }
}

If adhering explicitly to a CQRS pattern then we can also think of queries returning state but never changing it and commands as changing state and always returning void (i.e. not returning anything). This can become too restrictive in some situations but obviously one should have a valid reason for deviating from this pattern otherwise you’re not really following the CQRS way.

This is basically the core design principle behind creating a CQRS architecture. One of the key concepts here is that the separation of command and query means we can also do away with a common domain or data model for the commands or queries. Conceptually this separation gives us more freedom to not only design our data models but also in scenarios whereby our reads might come from a different locations/store to our writes, we’re again separating these pieces.

Another interesting side affect of a separation of command and query is, generally a command will warrant a different set of security requirements/permissions to a query. This is not to say we do not require security on queries, especially when sensitive data exists, but obviously we might be far more restrictive on who can amend a data object than who can view one. With this separation we can declare our permissions on our read/write data sources differently if we want.

Separating reads and writes!?

I previously stated that we might have our reads coming from one location and our writes from another, this may sound (initially at least) slightly strange. However, several documents on the concepts behind CQRS mention this scenario.

Notice, I said might, obviously this isn’t a requirement for us to be using a CQRS archtecture.

It’s not unusual to have a data store, such as a database to exist along with one or more data caches. The idea being writes are sent to the DB, the DB updates and the caches are then updated to keep in sync with the DB, but when we query data it’s (depending upon requirements) far faster to get data from the cache and return this. Hence we have a decoupling of the write and the read, so to speak, which means we might maintain a single DB but maintain multiple caches at different locations to ensure the fastest access/reads.

You may have thought – “if we have a situation whereby the DB is written to but the cache then gets updated, isn’t there a possible scenario where the cache’s data is not yet ‘consistent’ with the DB’s data?”

This is indeed the case of this separation, we’re really (sort of) doing something that a lot of No-SQL data stores do, which is have “eventual consistency”. This is eventual consistency, not at the DB level (like MongoDB) but as at an architectural level. In other words our DB will be immediately “consistent” but the cache’s will be “eventually consistent”.

From my understanding, the above is basically what we, as developers, need to do or how we need to think in terms of implementing CQRS.

Most CQRS documents also go on to talk about architectural design. Such as, we see CRQS documents talk about Event Stores as part of the overall design. An event store is used to store each transaction/change in data as single entities and can be used to regenerate the whole database from each of these entities. However I do not want to go into architectural choices/options as CQRS in this post.

References

CQRS Journey
CQRS Journey Source
CQRS, Task Based UIs, Event Sourcing agh!
Building an Event Storage

C# 7 Tuples

C#/.NET has had support for the Tuple class (a reference type) since .NET 4, but with C# 7 tuples now gets some “syntactic sugar” (and a new ValueTuple struct) to make them part of the language.

Previous to C# 7 we’d write code like this 

var tuple = new Tuple<string, int>("Four", 4);

var firstItem = tuple.Item1;

Now, in a style more like F#, we can write the following

var tuple = ("Four", 4);

var firstItem = tuple.Item1;

This code makes the creation of tuples slightly simpler but we can also make the use of the “items” within a tuple more readable by using names. So we could rewrite the above in a number of ways, examples of each supplied below

(string name, int number) tuple1 = ("Four", 4);
var tuple2 = (name: "Four", number: 4);
var (name, number) =  ("Four", 4);

var firstItem1 = tuple1.name;
var firstItem2 = tuple2.name;
var firstItem3 = name;

Whilst not quite as nice as F# we also now have something similar to pattern matching on tuples, so for example we might be interested in creating a switch statement based upon the numeric (second item) in the above code, we can write

switch (tuple)
{
   case var t when t.number == 4:
      Console.WriteLine("Found 4");
      break;
}

Again, similar to F# we also can discard items/params using the underscore _, for example

var (_, number) = CreateTuple();

In this example we assume the CreateTuple returns a tuple type with two items. The first is simply ignored (or discarded).

The Gherkin language

Gherkin is a DSL used within BDD development. It’s used along with Cucumber which processes the DSL or in the case of .NET we can use tools such as SpecFlow (which I posted about a long time back, see Starting out with SpecFlow) to help generate files and code.

Gherkin allows us to create the equivalent of use cases in a human readable form, using a simple set of keywords and syntax which can then be use to generate a series of method calls to undertake some action and assertion.

Getting started

We can use a standard text editor to create Gherkin’s feature files, but if you prefer syntax highlighting and intellisense (although this language we’re using is pretty simple) then install SpecFlow into Visual Studio or add a Gherkin syntax highlighter into VSCode (for example) or just use your preferred text editor.

I’m going to create a feature file (with the .feature extension) using the SpecFlow item template, so we have a starting point which we can then work through. Here’s the file generated code

Feature: Add a project
	In order to avoid silly mistakes
	As a math idiot
	I want to be told the sum of two numbers

@mytag
Scenario: Add two numbers
	Given I have entered 50 into the calculator
	And I have entered 70 into the calculator
	When I press add
	Then the result should be 120 on the screen

What we have here is a Feature which is meant to describe a single piece of functionality within an application. The feature name should be on the same line as the Feature: keyword.

In this example, we’ve defined a feature which indicates our application will have some way to add a project within our application. We can now add an optional description, which is exactly what the SpecFlow template did for us. The Description may span multiple lines (as can be seen above with the lines under the Feature line being the description) and should be a brief explanation of the specific feature or use case. Whilst the aim is to be brief, it should include acceptance criteria and any relevant information such as user permissions, roles or rules around the feature.

Let’s change our feature text to be a little meaningful to our use case.

Feature: Add a project

	Any user should be able to create/add a new project
	as long as the following rules are met

	1. No duplicate project names can exist
	2. No empty project names should be allowed

I’m sure with a little thought I can come up with more rules, but you get the idea.

The description of the feature is a useful piece of documentation and can be used as a specification/acceptance criteria.

Looking at the generated feature code, we can see that SpecFlow also added @mytag. Tags allow us to group scenarios. In terms of our end tests, this can be seen as a way of grouping features, scenarios etc. Multiple tags may be applied to a single feature or scenario etc. for example

@project @mvp
Feature: Add a project

I don’t need any tags for the feature I’m implementing here, so I’ll delete that line of code.

The Scenario is where we define each specific scenario of a feature and the steps to be taken/expected. The Scenario takes a similar form to a Feature, i.e. Scenario: and then a description of the context.

Following the Scenario line, we then begin to define the steps that make up our scenario, using the keywords Given, When, Then, And and But.

In unit testing usage, we can view Given as a precondition or setup. When, And and But as actions (where But is seen as a negation) and this leaves Then as an assertion.

Let’s just change to use some of these steps in a more meaningful manner within the context of our Scenario

Scenario: Add a project to the project list
	Given I have added a valid project name
	When I press the OK button
	Then the list of projects should now include my newly added project

Eventually, if we generate code from this feature file, each of these steps would get turned into a set of methods which could be used as follows

  • Given becomes an action to initialize the code to the expected context
  • When becomes an action to set-up any variables, etc.
  • Then becomes an assertion or validate any expectations

Multiple When‘s can be defined using the And keyword, for example imagine our Scenario looked like this

Scenario: Add a project to the project list
	Given I have added a valid project name
	When I press the OK button
	And I checked the Allow checkbox
	Then the list of projects should now include my newly added project

Now in addition to the When I press the OK button step I would also get another When created, as the And keyword simply becomes another When action. In essence the And duplicates the previous keyword.

We can also include the But keyword. As we’ve seen And, this is really another way of defining a additional When steps but in a more human readable way, But works in the same way as the And keyword by simply creating another When step in generated code, however But should be viewed as a negation step, for example

Scenario: Add a project to the project list
	Given I have added a valid project name
	When I press the OK button
	But the Do Not Allow checkbox is unchecked
	Then the list of projects should now include my newly added project

Finally, as stated earlier, Then can be viewed as a place to write our assertions or simply check if the results matches our expectations. We can again use And after the Then to create multiple then steps and thus assert multiple expectations, for example

Scenario: Add a project to the project list
	Given I have added a valid project name
	When I press the OK button
	But the Do Not Allow checkbox is unchecked
	Then the list of projects should now include my newly added project
        And the list of project should increase by 1

More keywords

In the previous section we covered the core keywords of Gherkin for defining our features and scenarios. But Gherkin also includes the following

Background, Scenario Outline and Examples.

The Background keyword is used to define reusable Given steps, i.e. if all our scenarios end up requiring the application to be in edit mode we might declare a background before any scenarios, such as this

Background:
	Given the projects list is in edit mode
	And the user clicks the Add button

we’ve now created a sort of, top level scenario which is run before each Scenario.

The Scenario Outline keywords allow us to define a sort of scenario function or template. So, if we have multiple scenarios which only differ in terms of data being used, then we can create a Scenario Outline and replace the specific data points with variables.

For example let’s assume we have scenarios which actually define multiple project names to fulfil the feature’s two rules (we outlined in the feature). Let’s assume we always have a project named “Default” within the application and therefore we cannot duplicate this project name. We also cannot enter a “” project name.

If we write these as two scenarios, then we might end up with the following 

Scenario: Add a project to the project list with an empty name
	Given the project name ""
	When I press the OK button
	Then the project should not be added

Scenario: Add a project to the project list with a duplicate name
	Given the project name "Default"
	When I press the OK button
	Then the project should not be added

If we include values within quotation marks or include numbers within our steps, then these will become arguments to the methods generated for these steps. This obviously offers us a way to reuse such steps or use example data etc.

Using a Scenario Outline these could be instead defined as

Scenario Outline: Add a project to the project list with an invalid name
	Given the project name <project-name>
	When I press the OK button
	Then the project should not be added

	Examples: 
	| project-name |
	| ""           |
	| "Default"    |

The <> acts as placeholders and the string within can be viewed as a variable name. We then define Examples which becomes our data inputs to the scenario.

Gherkin also includes the # for use to start a new line and mark it as a comment, multiple lines may be commented out using triple quotation marks, such as “””. Here’s a example of usage

# This is a comment
	
Scenario Outline: Add a project to the project list  with an invalid name
	Given the project name <project-name>
	"""
	Given the project name <project-id>
	"""
	When I press the OK button
	Then the project should not be added

	Examples: 
	| project-name |
	| ""           |
	| "Default"    |

It should be noted that after listing these different keywords and their uses you can also create a scenario that’s a Given followed by a Then, in other words a setup step followed by an assertion, if this is all you need.

For example

Scenario: Add a project
	Given A valid project name
        Then the project list should increase by 1

SpecFlow specifics

On top of the standard Gherkin keywords, SpecFlow adds a few bits.

The @ignore tag is used by SpecFlow to generate ignored test methods.

SpecFlow has also add Scenario Template as a synonym to Scenario Outline. Like wise Scenarios is a alternate to Examples.

Code generation

We’re not going to delve into Cucumber or the SpecFlow generated code except to point out that if you define more than one step within a scenario with the same text, this will generate a call to the same method in code. So whilst you might read a scenario as if it’s a new context or the likes, ultimately the code generated will execute the same method.

References

Gherkin Reference
Using Gherkin Language In SpecFlow