Mutable values and closures

The real title of this post should be The mutable variable ‘count’ is used in an invalid way. Mutable variables may not be captured by closures. Consider eliminating this use of mutation or using a heap-allocated mutable reference cell via ‘ref’ and ‘!’ but I thought it was a little long for a title.

Setting the scene

I was writing a simple C# extension method for the System.String class which has the following method signature

public static void Split(this string s, char[] seperator, StringSplitOptions options, Action<string, int, int> action)

The idea being that instead of creating an array of strings for each delimited string, I would instead pass the original string to an Action delegate with the start of a delimited field and the length of the field. This would help save on potentially wasteful creation of memory when handling large amounts of data.

I decided it’d be cool to write the unit tests for this method using F# (to help increase my knowledge of F#) – review Unit testing in F# to see how I set up the tests etc.

So the intention was to create a simple unit test, which just checked the number of times the Action delegate was called, which should be the same number of times as the number of delimited words within the string. So here’s a sample of what I would have written in C#

[Fact]
public void ExpectTheSameNumberOfCallsToTheActionDelegateAsDelimitedWords()
{
   string test = "The, quick, brown, fox";

   int count = 0;
   test.Split(new []{','}, StringSplitOptions.None, (s, i, j) => count++);

   Assert.Equal(4, count);
}

So I implemented the following F# text, very much along the lines of the above

[<Fact>]
let ``Expect the same number of calls to the action delegate as delimited words`` () =
   let mutable count = 0
   "The, quick, brown, fox".Split([|','|], StringSplitOptions.None, 
            fun s a b -> count <- count + 1)
   count |> should equal 4

at this point Visual Studio shows a squiggly red line under the code fun s a b -> count <- count + 1 with the message The mutable variable ‘x’ is used in an invalid way. Mutable variables may not be captured by closures. Consider eliminating this use of mutation or using a heap-allocated mutable reference cell via ‘ref’ and ‘!’. What’s going on ?

Searching for answers

Credit where credit is due, I searched around and found the web and found the following StackOverflow question “The mutable variable ‘i’ is used in an invalid way.?” which describes a solution to this problem.

Basically you cannot use mutable values inside a closure and thus we need to change to a ref type. So the following code now works

[<Fact>]
let ``Expect the same number of calls to the action delegate as delimited words`` () =
   let count = ref 0
   "The, quick, brown, fox".Split([|','|], StringSplitOptions.None, 
                fun s a b -> count := !count + 1)
   !count |> should equal 4

The changes here are that the mutable value count is now a ref type, the code to increment the count changes to use the ref assignment operator := and we also use the reference cell dereferencing operator !. The last line also uses the same. Alternatively we could have written fun s a b -> count := count.Value + 1 and the last line as count.Value |> should equal 4.

So why can’t we use the mutable type within a closure, after all it works fine in C#. The StackOverflow question (listed above) has a telling comment. Mutable values are stored on the stack not the heap. Therefore when the anonymous function exits, values stored on the stack are discarded during the stack unwind.

So how can C# handle this code ?

In C# we can do this because the Common Language Runtime (CLR) automatically places variables on the heap if they’re used as part of a closure as detailed in the post Capturing Mutables in F#.

References

The following are some references I located either whilst trying to solve this problem or as part of my research into the reasons for this after fixing the code.

The mutable variable ‘i’ is used in an invalid way.?
Mutable variables cannot be captured by closures” is giving me a bad day
Capturing Mutables in F#
The Truth About Value Types