Testing list contents with xUnit and Assert.Collection

Martijn Storck

xUnit has gained widespread popularity among .NET developers as a favored unit testing tool. Personally, I prefer using xUnit along with its basic Xunit.Assert assertion library, rather than alternative options like FluentAssertions. While the reasons for this preference are worthy of a separate discussion, one challenge with the xUnit assertion library is its limited documentation. The sparse documentation means that much of its power, including lesser-known assertions, goes untapped. This article delves into the usage of one such assertion to create clear and meaningful collection tests in C#.

Testing a list of events

In this example the test subject is an Event Sourcing aggregate called Project, which has a public method that returns a list of unsaved Event objects. In this test we want to validate that:

  • the list has the correct number of events; and
  • the events are of the expected derived type; and
  • the events have the correct version number in the Version field; and
  • the events are in the correct order.

Here is an example of how this test could be written with only basic xUnit assertions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

[Fact]
public void APoorExample()
{
    List<Event> events = _project.UnsavedEvents();
    
    Assert.Equal(4, events.Count);
    
    Assert.IsType<ProjectCreatedEvent>(events[0]);
    Assert.IsType<PhraseCreatedEvent>(events[1]);
    Assert.IsType<TranslationUpdatedEvent>(events[2]);
    Assert.IsType<TranslationUpdatedEvent>(events[3]);

    for (var i = 0; i < 4; i++)
    {
        Assert.Equal(i, events[i].Version);
    }
}

Here is what happens:

  • On line 4, a list of events is retrieved from the test subject.
  • On line 6, the length of the list is asserted.
  • On lines 8-11, the types of the events are asserted.
  • On lines 13-16, the version numbers are identified using an unnecessary for loop.

If any of the assertions in this test fail, a very basic error message will be displayed in the test output. For example, when removing one of the translation events:

Xunit.Sdk.EqualException
Assert.Equal() Failure
Expected: 4
Actual:   3
   at APoorExample() in ProjectTest.cs:line 6

This provides no helpful information about the actual contents of the list at this point in the test, meaning we’d need to fire up the debugger to figure out what is going on. If the Version field for an event is off, the error message is equally unhelpful:

Xunit.Sdk.EqualException
Assert.Equal() Failure
Expected: 1
Actual:   0
   at APoorExample() in ProjectTest.cs:line 15

We’d need to look at the code first to see what’s actually being tested here and, again, debug the test to figure out what’s going on.

Introduction Assert.Collection

Let’s rewrite this test in a way that has more readable code and provides better error output. The only documentation for Asset.Collection that I could find is in the XML documentation:

/// <summary>
/// Verifies that a collection contains exactly a given number of elements, which meet
/// the criteria provided by the element inspectors.
/// </summary>
/// <typeparam name="T">The type of the object to be verified</typeparam>
/// <param name="collection">The collection to be inspected</param>
/// <param name="elementInspectors">The element inspectors, which inspect each element in turn. The
/// total number of element inspectors must exactly match the number of elements in the collection.</param>
public static void Collection<T>(
    IEnumerable<T> collection,
    params Action<T>[] elementInspectors)

To test our list, we need to specify an inspector for each element. As follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

[Fact]
public void ABetterExample()
{
    Assert.Collection(_project.UnsavedEvents(),
        e =>
        {
            Assert.IsType<ProjectCreatedEvent>(e);
            Assert.Equal(0, e.Version);
        },
        e =>
        {
            Assert.IsType<PhraseCreatedEvent>(e);
            Assert.Equal(1, e.Version);
        },
        e =>
        {
            Assert.IsType<TranslationUpdatedEvent>(e);
            Assert.Equal(2, e.Version);
        },
        e =>
        {
            Assert.IsType<TranslationUpdatedEvent>(e);
            Assert.Equal(3, e.Version);
        });
}

This test is slightly longer than the original version, but we’ll get to that in a bit. What’s nice is that the code cleanly groups the assertions per list element. The umbrella assertion, Assert.Collection, verifies that the number of items in the list is equal to the number of inspectors and that all the assertions pass.

Now, let’s look at some error messages. When the list is shorter than expected:

Xunit.Sdk.CollectionException
Assert.Collection() Failure
Collection: [
  ProjectCreatedEvent { Type = "ProjectCreatedEvent", Version = 0, ... },
  PhraseCreatedEvent { Type = "PhraseCreatedEvent", Version = 1, ... },
  TranslationUpdatedEvent { Type = "TranslationUpdatedEvent", Version = 2, ... }]
Expected item count: 4
Actual item count:   3
   at ABetterExample() in ProjectTest.cs:line 4

It shows the actual contents of the list and a clear error, which is the unexpected item count. How about an event type mismatch?

Xunit.Sdk.CollectionException
Assert.Collection() Failure
Collection: [
  ProjectCreatedEvent { Type = "ProjectCreatedEvent", Version = 0, ... },
  PhraseCreatedEvent { Type = "PhraseCreatedEvent", Version = 1, ... },
  PhraseCreatedEvent { Type = "PhraseCreatedEvent", Version = 2, ... },
  TranslationUpdatedEvent { Type = "TranslationUpdatedEvent", Version = 3, ... }]
Error during comparison of item at index 2
Inner exception: Assert.IsType() Failure
        Expected: TranslationUpdatedEvent
        Actual:   PhraseCreatedEvent
   at ProjectTest.<>c.<ABetterExample>b__6_2(Event e) in ProjectTest.cs:line 17

Again, it shows us the contents of the list, but this time it mentions the item index where the assertion failed along with the output of that assertion. For bonus points the backtrace points to the correct line number in our code where the problem occurred. Verbose error messages like these usually allow developers to fix behavior without having to reach for the debugger.

Best practice testing

In my personal opinion, it is more effective to test a single aspect per test case, as opposed to multiple aspects in a single test case. By splitting our tests into separate cases for event versions and event types, we can streamline the code and make its intent clearer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

[Fact]
public void Project_WithUnsavedEvents_HasEventVersions() =>
    Assert.Collection(_project.UnsavedEvents(),
        e => Assert.Equal(0, e.Version),
        e => Assert.Equal(1, e.Version),
        e => Assert.Equal(2, e.Version),
        e => Assert.Equal(3, e.Version));

[Fact]
public void Project_WithUnsavedEvents_HasEventTypes() =>
    Assert.Collection(_project.UnsavedEvents(),
        e => Assert.IsType<ProjectCreatedEvent>(e),
        e => Assert.IsType<PhraseCreatedEvent>(e),
        e => Assert.IsType<TranslationUpdatedEvent>(e),
        e => Assert.IsType<TranslationUpdatedEvent>(e));

This is the most concise example to date, even reducing the line count by two compared to the original. If the length of the list holds significant semantic importance, a simple additional test case could be created to explicitly assert it:

1
2
3

[Fact]
public void Project_WithUnsavedEvents_HasCorrectCount() =>
    Assert.Equal(4, _project.UnsavedEvents().Count);

Afbeelding van ikaika op Freepik