MultipleOutputFilesPerTest.md

Multiple output files per test

Contents

• Introduction
• Scenarios
• Catch2
• doctest
• Approval Tests

Introduction

ApprovalTests uses the name of the current test to determine the names of output files that it writes. This means that by default, there is only one approval file per test case.

However, sometimes it is useful to be able to verify multiple files in one test case, or have a file per OS or other environment configuration.

Scenarios

Here are some examples of files you might want.

 

Multiple data inputs:

In this scenario, your test creates 3 files, all of which are being checked when you run the test.

TestProteinGeneration.createImage.protein1.approved.png
TestProteinGeneration.createImage.protein2.approved.png
TestProteinGeneration.createImage.protein3.approved.png

 

Multiple outputs:

In this scenario, the code under test creates three different types of files, all of which are being checked.

TestProtein.processInput.logOutput.approved.txt
TestProtein.processInput.calculationResults.approved.txt
TestProtein.processInput.renderedResult.approved.png

 

Multiple environments:

In this scenario, your test only creates one file, and which one it is checked against is dependent on which OS the test is running on.

TestQtDialog.loginScreen.onMacOSX.approved.png
TestQtDialog.loginScreen.onWindows.approved.png
TestQtDialog.loginScreen.onLinux.approved.png

 

Here are a few ways to do that.

Catch2

You can have a file-per-subsection.

You can either do these dynamically, e.g. in a for-loop:

TEST_CASE("MultipleOutputFiles-DataDriven")
{
    // This is an example of how to write multiple different files in a single test.
    // Note: For data as small as this, in practice we would recommend passing the
    // greetings container in to Approvals::verifyAll(), with a lambda to format the output,
    // in order to write all data to a single file.
    std::vector<Greeting> greetings{ Greeting(British), Greeting(American), Greeting(French) };
    for(auto greeting: greetings)
    {
        SECTION(greeting.getNationality())
        {
            ApprovalTests::Approvals::verify(greeting.getGreeting());
        }
    }
}

^{snippet source / anchor}

Or hard-coded, with multiple sections:

TEST_CASE("MultipleOutputFiles-ForOneObject")
{
    Greeting object_under_test;
    SECTION("British")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(British));
    }
    SECTION("American")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(American));
    }
    SECTION("French")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(French));
    }
}

^{snippet source / anchor}

Note: Catch2 sub-sections continue to run even if the previous one failed. This is useful, as it allows you to approve all the files in one test run.

doctest

You can have a file-per-subcase.

Note: unlike Catch, doctest sub-cases must have static strings for names, so if you want to name things dynamically, you will have to use the native Approval Tests mechanism - see below.

You can have hard-coded, with multiple sections:

TEST_CASE("MultipleOutputFiles-ForOneObject")
{
    Greeting object_under_test;
    SUBCASE("British")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(British));
    }
    SUBCASE("American")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(American));
    }
    SUBCASE("French")
    {
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(French));
    }
}

^{snippet source / anchor}

Approval Tests

Approval Tests also allows for multiple files per test, via the NamerFactory. This works for all supported test frameworks.

You can either do these dynamically, e.g. in a for-loop:

TEST_CASE("ApprovalTests-MultipleOutputFiles-DataDriven")
{
    // This is an example of how to write multiple different files in a single test.
    // Note: For data as small as this, in practice we would recommend passing the
    // greetings container in to Approvals::verifyAll(), with a lambda to format the output,
    // in order to write all data to a single file.
    std::vector<Greeting> greetings{ Greeting(British), Greeting(American), Greeting(French) };
    for(auto greeting: greetings)
    {
        auto section = ApprovalTests::NamerFactory::appendToOutputFilename(greeting.getNationality());
        ApprovalTests::Approvals::verify(greeting.getGreeting());
    }
}

^{snippet source / anchor}

Or hard-coded, with multiple sections:

TEST_CASE("ApprovalTests-MultipleOutputFiles-ForOneObject")
{
    Greeting object_under_test;
    {
        auto section = ApprovalTests::NamerFactory::appendToOutputFilename("British");
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(British));
    }
    {
        auto section = ApprovalTests::NamerFactory::appendToOutputFilename("American");
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(American));
    }
    {
        auto section = ApprovalTests::NamerFactory::appendToOutputFilename("French");
        ApprovalTests::Approvals::verify(object_under_test.getGreetingFor(French));
    }
}

^{snippet source / anchor}

---

Back to User Guide