Category Archives: Unit Testing

Unit testing your MAUI project

Note: I found this didn’t work correctly on Visual Studio for Mac, I’ll update the post further if I do get it working.

This post is pretty much a duplicate of Adding xUnit Test to your .NET MAUI Project but just simplified for me to quickly repeat the steps which allow me to write unit tests against my MAUI project code.

So, you want to unit test some code in your MAUI project. It’s not quite as simple as just creating a test project then referencing the MAUI project. Here are the steps to create an NUnit test project with .NET 7 (as my MAUI project has been updated to .NET 7).

  • Add a new NUnit Test Project to the solution via right mouse click on the solution, Add | Project and select NUnit Test Project
  • Open the MAUI project (csproj) file and prepend the net7.0 to the TargetFrameworks so it looks like this
    <TargetFrameworks>net7.0;net7.0-android;net7.0-ios;net7.0-maccatalyst</TargetFrameworks>
    
  • Replace <OutputType>Exe</OutputType> with the following
    <OutputType Condition="'$(TargetFramework)' != 'net7.0'">Exe</OutputType>
    

    Yes, this is TargetFramework singular. You may need to reload the project.

  • Now in your unit test project you can reference this MAUI project and write your tests

So, why did we carry out these steps?

Our test project was targeting .NET 7.0 but our MAUI project was targeting different platform implementations of the .NET 7 frameworks, i.e those for Android etc. We need the MAUI project to build a .NET 7.0 compatible version hence added the net7.0 to the TargetFrameworks.

The change to add the OutputType ensures that we only build an EXE output for those other frameworks, and therefore for .NET 7.0 we’ll have a DLL to reference instead in our tests.

Now we can build and run our unit tests.

Unit testing with Swift

Swift comes with its own unit testing framework XCTest, let’s take a look at what we need to do to enable unit testing and how we use this library.

Test Package Dependencies

If you created your package via the Wwift package manager your project will be laid out with Sources and Tests folders.

Here’s an example of a library package, notice the .testTarget section

targets: [
   .target(
      name: "MyLib",
      dependencies: []),
   .testTarget(
      name: "MyLibTests",
      dependencies: ["MyLib"]),
]

Creating a test fixture

Next up we need to import the XCTest library and derive our test fixture/class from XCTestCase, for example

final class MyLibTests: XCTestCase {
   // tests go here
}

Creating a test

Tests are written in test methods within the test case subclass. Test methods have no parameters, no return value and the name should begin with lowercase prefix test, so for example

final class MyLibTests: XCTestCase {
   func testIsValid() {
     // test setup and asserts
   }
}

Asserting our tests

The XCTest library prefixes its assertion method with XCT. Hence, we have XCTAssertEqual for example to assert that two values are equal

final class MyLibTests: XCTestCase {
   func testIsValid() {
      let a = 1
      let b = 1
      XCTAssertEqual(a, b)
   }
}

Unit testing Haskel code with HUnit

As with most languages nowadays, we want to have some unit testing libraries/frameworks. Obviously Haskell is no different and we have tools such as HUnit.

Let’s first create a simple little bit of code to test, here’s my Calculator.hs file (abridged just to show the code I intend to write tests for)

module Modules.Calculator where

factorial :: (Integral a) => a -> a  
factorial 0 = 1  
factorial n = n * factorial (n - 1) 

We’ll need to have installed the HUnit package, if you haven’t already then run

cabal install --lib QuickCheck HUnit

We’ll store our tests in a folder named test off of our root folder (the name of the folder can be altered if you prefer).

Here’s our test code (my file is named CalculatorTests.hs)

module Main (main) where

import Test.HUnit
import System.Exit

import Modules.Calculator as Calc

testZeroCase = TestCase(assertEqual "Factorial 1" (1) (Calc.factorial 1))
testNonZeroCase = TestCase(assertEqual "Factorial 10" (2) (Calc.factorial 10))

main :: IO ()
main = do
    counts <- runTestTT ( test [
        testZeroCase,
        testNonZeroCase
        ])
    if (errors counts + failures counts == 0)
        then exitSuccess
        else exitFailure

In the above code we import our library code and have two tests, the first tests our factorial code with a 0 value, the second tests a non-zero. In this instance I’ve purposefully written a breaking test.

Before we run these and/or fix the tests let’s see what we’re doing with the test code. We create a function for the TestCase and within the TestCase we have our assertion along with a message prefix (the string), next we have the expected value and finally the actual value or in this case the function call which returns the actual value.

In main we basically create the test runner and supply an array of the test functions to be run, finally we exit the runner with success or failure.

Now before we can run our tests we need to declare a Test-Suite within the .cabal file. We add the following to the file

Test-Suite test-Calculator
  type:                exitcode-stdio-1.0
  hs-source-dirs:      test
                       .
  default-language:    Haskell2010
  main-is:             CalculatorTests.hs
  other-modules:       Modules.Calculator
  build-depends:       base >=4.14 && <4.15, HUnit

This tells cabal the folder to look for the tests (test in this case) along with . for the current folder (otherwise our tests won’t find out Calculator module). We then state what the main application for the cabal to run for the tests, include other modules etc.

Now run

cabal test

If all goes well you’ll get something like the following

Running 1 test suites...
Test suite test-Calculator: RUNNING...
### Failure in: 1
test\CalculatorTests.hs:9
Factorial 10
expected: 2
 but got: 3628800
Cases: 2  Tried: 2  Errors: 0  Failures: 1
Test suite test-Calculator: FAIL
Test suite logged to:
D:\Development\somefolder\test\HaskellBasics-0.1.0.0-test-HaskellBasics.log
0 of 1 test suites (0 of 1 test cases) passed.
cabal.exe: Tests failed for test:test-Calculator from
Calculator-0.1.0.0.

Obviously the test with the prefix message “Factorial 10” failed with the expected value 2 but the actual value 3628800, so we can fix that test case so it looks like this

testNonZeroCase = TestCase(assertEqual "Factorial 10" (3628800) (Calc.factorial 10))

and now all our tests will pass.

I’ll leave it to the reader to look into the other assertions etc. For example, checkout Test.HUnit.Base

Other References

HUnit: A unit testing framework for Haskell

Where have you been hiding dotMemoryUnit?

As a subscriber to all the JetBrains tools, I’m surprised that I didn’t know about dotMemoryUnit until it was mentioned on a recent episode of dotnetrocks. I’ve looked into memory type unit testing in the past as it’s an area (along with performance) that I find particularly of interest as part of a CI build.

So here it is MONITOR .NET MEMORY USAGE WITH UNIT TESTS.

Prerequisites

Simply add the NuGet package JetBrains.DotMemoryUnit to your test project and you’ll get the library for writing dotMemory test code, if you want to run dotMemory unit tests from the command line (or from your build/CI server) you’ll need to download the runner from https://www.jetbrains.com/dotmemory/unit/.

To run the command line test runner you’ll need to run dotMemoryUnit.exe passing in your test runner application and the DLL to be tested through dotMemoryUnit. For example

dotMemoryUnit "C:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console.exe" MyDotMemoryTests.dll

If you’re using Resharper the test runner supports dotMemory Unit, but you’ll need to select the option Run under dotMemory Unit or you’ll see the message

DotMemoryUnitException : The test was run without the support for dotMemory Unit. To safely run tests with or without (depending on your needs) the support for dotMemory Unit:

in Resharper’s test runner output window.

Getting started

The key thing to bare in mind is that dotMemoryUnit is not a unit testing framework and not an assertion framework, so you’ll still write NUnit, xUnit, MSTest etc. (unsupported test frameworks can be integrated with dotMemoryUnit, see Working with Unsupported Unit Testing Frameworks) code and you can still use the assertions with these libraries, or the likes of Should, Shouldly, FluentAssertions etc.

dotMemoryUnit simply allows us to add code to investigate memory usage, so here’s an example

[Test]
public void FirstTest()
{
   var o = new MyObject();

   dotMemory.Check(memory =>
      Assert.That(
         memory.GetObjects(
            where => where.Type.Is<MyObject>())
         .ObjectsCount, Is.EqualTo(0)));

   GC.KeepAlive(o);
}

In this example, we create an object and then use dotMemory to carry out a memory “check”. The dotMemory.Check call is a checkpoint (or snapshot) and returns a MemoryCheckpoint object which can be used to compare two checkpoints and/or we can carry out a query against the memory within the lambda in the checkpoint call.

Here, you can see, we get the memory then search of objects of type MyObject. We then assert whether this object count is zero. In the above it’s probable that the object has not been disposed of and hence this test will fail, however we have the GC.KeepAlive(o) in place to ensure that the object is not garbage collected during the test run.

See Checking for Objects

Beware that when running dotMemoryUnit, a .dmw file is created (by default) for failing tests and these are placed in your %USERPROFILE%\AppData\Local\Temp\dotMemoryUnitWorkspace folder. There is an auto delete policy when running dotMemoryUnit but these files are often MB’s in size. We can change the default locaiton on a per test basis using the DotMemoryUnitAttribute, i.e.

Note: The dmw file is viewable for the dotMemory application which is part of the JetBrains suite of applications.

[DotMemoryUnit(Directory = "c:\\Temp\\dotMemoryUnit")]
public void EnsureMemoryFreed()
{
// asserts
}

We’ve seen the use of the dotMemory.Check method. We can also use the dotMemoryApi class which gives a slightly different interface, for example the following code creates two snapshots and then checks what objects of type MyObject were created between the snapshots, for example

[Test]
public void AllocateFourMyObjects()
{
   var snap1 = dotMemoryApi.GetSnapshot();

   var f1 = new MyObject();
   var f2 = new MyObject();
   var f3 = new MyObject();
   var f4 = new MyObject();

   var snap2 = dotMemoryApi.GetSnapshot();

   Assert.That(
      snap2.GetObjects(w => w.Type.Is<MyObject>()).ObjectsCount -
      snap1.GetObjects(w => w.Type.Is<MyObject>()).ObjectsCount, 
      Is.EqualTo(4));
}

We can also use the AssertTrafficAttribute (Note: this seemed very slow via Resharper’s GUI whereas the console app was much faster). In a similar way to the above, although in this example we’re saying assert that the allocated object count of types MyObject does not exceed 4, i.e. in the previous code we states that we expect exactly 4 allocations, so be aware the difference here.

[
Test,
AssertTraffic(AllocatedObjectsCount = 4, Types = new [] { typeof(MyObject)})
]
public void AllocateFourMyObjects()
{
   var f1 = new MyObject();
   var f2 = new MyObject();
   var f3 = new MyObject();
   var f4 = new MyObject();
}

Finally, we can also use snapshot and the dotMemoryApi.GetTrafficBetween method do so something similar to the last two examples

[
Test, DotMemoryUnit(CollectAllocations = true)
]
public void AllocateFourMyObjects()
{
   var snap1 = dotMemoryApi.GetSnapshot();

   var f1 = new MyObject();
   var f2 = new MyObject();
   var f3 = new MyObject();
   var f4 = new MyObject();

   var snap2 = dotMemoryApi.GetSnapshot();

   var traffic = dotMemoryApi.GetTrafficBetween(snap1, snap2);
   var o = traffic.Where(w => w.Type.Is<MyObject>());
   Assert.AreEqual(4, o.AllocatedMemory.ObjectsCount);
}

To use memory allocations we need to set the DotMemoryUnitAttribute CollectAllocations to true. For the AssertTrafficAttribute this happens automatically.

That’s if for now. Go and experiment.

References

dotMemoryUnit Help

Fixie, convention based unit testing

I’m always looking around at frameworks/libraries etc. to see if they enhance my development experience or offer new tools/capabilities that could be of use. I’m not wholly sure how I came across Fixie, but I wanted to check it out to see what it had to offer.

Introduction

Fixie is a convention based unit testing library, think NUnit or xUnit but without attributes.

It’s amusing that after re-reading the sentence above, I totally missed the irony that Fixie is really more of a throw back to jUnit with it’s convention based testing than the .NET equivalents with their attributes.

I’m not usually a big fan of convention based libraries, but there’s no doubt that if you’re happy with the convention(s) then code can look pretty clean.

The coolest thing with Fixie (well it’s cool to me) is that you simply add the nuget package to your solution, but don’t need to add a using clause that references Fixie (unless you wish to switch from the default conventions).

Fixie is immediately (or near as immediately) recognized by the Visual Studio Test Runner, but Resharper’s test runner, sadly, does not work with it “out of the box” (I believe there is a Resharper test runner, but I’ve not tried it and it didn’t appear in their extension manager, so I’m not sure what the state is of this at the time of writing.

Default Convention

The default convention is that your test class name is suffixed with the word Tests and methods can be named anything, but must return either void or async and be public, i.e.

public class MatrixTests
{
   public void AddMatrix()
   {
      // test based code
   }

   public async Task SubtractMatrix()
   {
      // test based code
   }
}

I’ve purposefully not written any assertion code because, Fixie does not supply any. Such code should be supplied using the likes of Should, Shouldly, FluentAssertions etc.

Custom Convention

Fixie gives us the tools to define our own custom convention if we want to change from the default. This includes the ability to not only change the naming conventions but also to include attributes to mark tests (like NUnit/xUnit uses). In fact, with this capability Fixie can be used to run NUnit or xUnit style tests (as we’ll see later). For now let’s just create our own test attribute

AttributeUsage(AttributeTargets.Method)]
public class TestAttribute : Attribute
{
}

So this attribute is only declared on methods. To implement a new convention we simple create a class derived from Fixies.Convention, for example

public class TestConvention : Convention
{
   public TestieConvention()
   {
      Methods.HasOrInherits<TestAttribute>();
   }
}

so now our tests might look like this

public class MatrixTests
{
   [Test]
   public void AddMatrix()
   {
      // test based code
   }

   [Test]
   public async Task SubtractMatrix()
   {
      // test based code
   }
}

So in essence, using Fixie, we could create our own test framework (of sorts) and using (as already mentioned) Should, Shouldly, Fluent Assertions etc. libraries to handle asserts. However, I’m sure that’s ultimately not what this functionality is for, but instead allows us to define our own preferred conventions.

See also Custom Conventions

NUnit & xUnit Convention

I mentioned that we could use Fixie to execute against other testing frameworks and on the Fixie github repository, we can find two such convention classes (and support classes) to allow Fixie to run NUnit or xUnit style tests.

See NUnit Style and xUnit Style.

I doubt the intention here is to replace NUnit or xUnit with Fixie as obviously you’d need to support far more capabilities, but these convention repositories give a great overview of what can be done on the Fixie convention front.