Test case fixture isolation with runtests.

[corriander]

This is a similar issue to #24, but my query is about using runtests and a simple naming scheme/pattern to invoke test fixtures on a per test case basis. I'll explain by example...

My intent was to keep all my test-related stuff in a single schema (e.g. tests), with tests named something like

testcase1_test_a_thing()
testcase1_test_more()
testcase2_test_another_thing()

with fixtures:

setup_testcase1_insert_test_data()
startup_testcase2_alter_view()

Then invoke runtests('tests', '^testcase2') to just run the relevant fixtures and tests (i.e. a startup fixture followed by a single test using this example). From what I can see, this is a non-starter due to the way all fixtures are identified by findfuncs by prefix, irrespective of any pattern provided to runtests.

Am I barking up the right tree here? This seems a reasonable approach to me but I may be influenced by other testing frameworks. In principle it looks simple to restrict fixtures by name pattern as well but how to do this cleanly is less obvious... This leads me to wonder about alternative approaches (either for implementing this behaviour or my use case). At the moment I'm leaning towards severely restricting/dropping the use of text fixtures because they'd need to be universally applicable, which means I might as well not use runtests either.

Any suggestions/comments?

[theory]

Yeah, I believe that the :pattern argument to runtests() only applies the pattern to test functions, not to fixture functions. It looks like this:

CREATE OR REPLACE FUNCTION runtests( NAME, TEXT )
RETURNS SETOF TEXT AS $$
    SELECT * FROM _runner(
        findfuncs( $1, '^startup' ),
        findfuncs( $1, '^shutdown' ),
        findfuncs( $1, '^setup' ),
        findfuncs( $1, '^teardown' ),
        findfuncs( $1, $2, '^(startup|shutdown|setup|teardown)' )
    );
$$ LANGUAGE sql;

I think it'd be reasonable to add a variant that allows patterns for the fixtures, too. Something like:

CREATE OR REPLACE FUNCTION runtests( NAME, TEXT, TEXT, TEXT, TEXT, TEXT )
RETURNS SETOF TEXT AS $$
    SELECT * FROM _runner(
        findfuncs( $1, COALESCE($2, '^startup' ) ),
        findfuncs( $1, COALESCE($6, '^shutdown') ),
        findfuncs( $1, COALESCE($3, '^setup') ),
        findfuncs( $1, COALESCE($5, '^teardown') ),
        findfuncs( $1, $4, '^(startup|shutdown|setup|teardown)' )
    );
$$ LANGUAGE sql;