Deprecation notice: This library is no longer supported.
Please visit wheredoc for an improved implementation (though not backward-compatible).
Provides data-table support in JavaScript tests.
Mostly ready for prime time, though still some things to do.
- Cucumber
scenario-outlines - Spock
where:blocks - JP Castro's (@jphpsf)DRYing Up Your JavaScript Jasmine Tests With the Data Provider Pattern
- Richard Rodger's (@rjrodger) mstring
- data-driven, an extension for mocha.js
JSON
- jasmine (v2.0.0 on browser)
- jasmine-node which uses v1.3.1
- mocha
- nodeunit
- QUnit
- tape @substack's event-driven TDD-flavored TAP project for testling
See the tests...
npm install where.js
or
git clone https://github.com/dfkaye/where.js.git
Running where.js adds a where() method to the global namespace -
(here's why, if you're curious...)
node.js:
require('where.js');
assert(typeof where === 'function');
// => true
assert(typeof global.where === 'function');
// => true
browsers:
<script src="[path/to]/where.js"></script>
assert(typeof where === 'function');
// => true
assert(typeof window.where === 'function');
// => true
Easier to read and modify this
it('description', function () {
where(function(){
/***
a | b | c
1 | 2 | 3
4 | 3 | 7
6 | 6 | 12
***/
expect(a + b).toBe(c);
});
});
than this
it('description', function () {
[[1, 2, 3],
[4, 3, 7],
[6, 6, 12]].forEach(function(row, r, rows) {
expect(Number(row[0]) + Number(row[1])).toBe(Number(row[2]));
});
});
where.js merges methods and lessons learned from my earlier jasmine-where and jasmine-intercept projects, which are now deprecated as of this release.
In imitation of Richard Rodger's mstring,
where() accepts a function and inspects its string value, converts the
commented data-table into an array of values, and applies the first row data as
argument names in a new Function().
The function's body should contain an expectation or assertion statement.
If your JavaScript runtime supports ES65 strict mode, an undeclared variable in the function body will result in an error.
where() accepts a second argument, an context specifier, that allows you to
inject other information or references into the test function, namely any
references to your test library (jasmine, QUnit, etc.) and methods, variables
that are defined outside of the current test (in setup or before calls, for
example), and flags for logging or intercepting results to the console.
See more details about the context argument.
A data table inside a where() function should be enclosed between start /***
and end ***/ 3-asterisk comments.
where(function(){
/*** // start
a | b | c
1 | 2 | 2
4 | 3 | 4
6 | 6 | 6
***/ // end
expect(Math.max(a, b)).toBe(c);
});
aside where.js is a testing module and is not intended to be run through a pre-production minifer such as google closure or uglify, both of which remove the comments.
If you're writing tests in CoffeeScript, see the CoffeeScript format section for slightly different syntax (though more aesthetically appealing, I must admit).
Similar to Cucumber and Spock, where.js data tables must contain at least two rows. The first row must contain the names to be used as variables in the expectation. The remaining rows must contain data values for each name. Values must be separated by the pipe | character.
For example, a, b, and c are named as variables in the table, and used in
the expectation - without having to be declared explicitly with the var
keyword.
it('should pass with correct data and expectation', function () {
where(function(){
/***
a | b | c
1 | 2 | 2
4 | 3 | 4
6 | 6 | 6
***/
expect(Math.max(a, b)).toBe(c);
});
});
Tables may also optionally contain left and right edge borders, similar to Cucumber and Fit:
it('should pass with left and right table borders', function () {
where(function(){
/***
| a | b | c |
| 1 | 2 | 2 |
| 4 | 3 | 4 |
| 6 | 6 | 6 |
***/
expect(Math.max(a, b)).toBe(c);
});
});
For empty cells, borders are required.
Table rows may contain line comments.
where(function(){
/***
| a | b | c |
| 1 | 2 | 2 | // should pass
| 4 | 3 | x | // should fail
***/
expect(Math.max(a, b)).toBe(c);
});
A commented row is ignored.
where(function(){
/***
| a | b | c |
| 1 | 2 | 2 | // should pass
//| 4 | 3 | x | (should not execute)
***/
expect(Math.max(a, b)).toBe(c);
});
it('should handle quoted strings', function () {
where(function () {
/***
a | b | c | d | e | f | g | h | normal
'' | "" | "''" | '""' | ' ' | ' faff ' | 'undefined' | | a b
***/
expect(a).toBe('\'\'');
expect(b).toBe('\"\"');
expect(c).toBe('\"\'\'\"');
expect(d).toBe('\'"\"\'');
expect(e).toBe('\' \'');
expect(f).toBe('\' faff \'');
expect(g).toBe('\'undefined\'');
expect(h).toBe(''); // empty boxes are empty strings
expect(normal).toBe('a b'); // unquoted values are strings
});
});
it('should throw when empty data missing separators', function() {
expect(function() {
where(function() {
/***
a
// interpreted as missing row, not as row with empty data
***/
expect(a).toBe('');
});
}).toThrow();
});
it('should not throw when empty data is separated', function() {
where(function() {
/***
| a |
| | // interpreted as row with empty data
***/
expect(a).toBe('');
});
});
That enables you to type Math.max(a, b) to avoid re-typing coercions such as
Math.max(Number(a), Number(b)).
That also means every test can use strict equality, so you don't need to rely on regular expression matchers when getting started.
However, where Math is involved there is usually an octal, signed, comma, or
precision bug waiting. where.js handles all but precision automatically. You
can get precision into your tests by adding another column, as seen in the test
created to verify that numeric conversions work:
where(function(){
/***
a | b | c | p
0 | 1 | 1 | 1
0.0 | 1.0 | 1 | 1
-1 | +1 | 0 | 1
+1.1 | -1.2 | -0.1 | 2
08 | 08 | 16 | 2
6 | 4 | 10.0 | 3
8.030 | -2.045 | 5.985 | 4
1,000.67 | 1345 | 2345.67 | 6
5 | 5 | 10 | 1
***/
/*
* using precision for famous .1 + .2 == 0.30000000000000004
* and 5.985 vs 5.98499999999999999999999999 bugs
*/
var s = (a + b).toPrecision(p);
/*
* toPrecision() returns a string, so the prefixed '+' uses implicit
* conversion to number.
*/
expect(+s).toBe(c);
});
where(function () {
/***
date | dataString | number | numberString
1973-01-01 | '2014-01-01' | 2 | '4'
***/
expect(date).toBe('1973-01-01');
expect(dataString).toBe('\'2014-01-01\'');
expect(number).toBe(2);
expect(numberString).toBe('\'4\'');
});
Truthy/falsy values are also automatically converted as per this test:
where(function () {
/***
a | b | c | d
null | undefined | false | true
***/
expect(a).toBe(null);
expect(b).toBe(undefined);
expect(c).toBe(false);
expect(d).toBe(true);
});
A passing where() test has no effect on the test runner's default reporter
output.
When an expectation fails, the data-table labels plus the row of values for the
current expectation are added to the current failing item. Every failed
expectation in a where() test will appear similar to:
[a | b | c] :
[1 | 2 | x] (Error: Expected 2 to be 'x'.)
where() returns a results object with arrays for labels and values
derived from the data table, passing tests, and failing tests.
The following snip shows how to refer to each array in the results:
it('should pass with correct data and expectation', function () {
var results = where(function(){
/***
a | b | c
1 | 2 | 2
4 | 3 | 4
6 | 6 | 6
***/
expect(Math.max(a, b)).toBe(c);
});
expect(results.data.labels.join(',')).toBe("a,b,c");
expect(results.data.values.length).toBe(3);
expect(results.passing.length).toBe(3);
expect(results.failing.length).toBe(0);
});
THIS IS THE CRITICAL "LIBRARY-AGNOSTIC" PIECE OF THE PUZZLE.
where() accepts up to two arguments. The first is the function containing the
data table and assertions.
The second, optional but recommended, argument is a context or configuration
object that allows you to specify the test strategy (or library) in use, and
pass non-global items to the test function (a library's assert or expect or test
method, e.g.).
The following snip shows that context is itself made available for inspection
inside the test function:
where(function(){
/***
a | b | c
0 | 0 | 0
***/
expect(context.expect).toBe(expect);
}, { expect: expect }); // <= context
Normal logging occurs mainly with failing tests. You can set a nonce property on
the context specifier as log: 1 to enable logging of all test output to the
console.
where(function(){
/***
a | b | c
1 | 2 | 2
4 | 3 | 4
6 | 6 | 6
***/
expect(Math.max(a, b)).toBe(c);
}, { log: 1, expect: expect });
Failed tests or errors will appear as
[a | b | c] :
[1 | 2 | x] (Error: Expected 2 to be 'x'.)
Passing tests will appear as
[a | b | c] :
[1 | 2 | 2] (Passed)
Not all test libraries support "fail" in the same way - that is, while some
tests "should" fail, a library may report the test as failed even though the
failure is expected. To prevent expected failures from appearing as failed in
a test library's reporter (make a fail into a pass), you can set a nonce
property on the context specifier as intercept: 1 and verify the number of
expected failures in the results returned by where():
var results = where(function(){
/***
a | b | c
1 | 2 | 2
4 | 3 | x
6 | 6 | 6
***/
expect(Math.max(a, b)).toBe(c);
}, { intercept: 1, expect: expect });
expect(results.data.labels.join(',')).toBe("a,b,c");
expect(results.data.values.length).toBe(3);
expect(results.passing.length).toBe(2);
expect(results.failing.length).toBe(1);
ISSUE: In logging-event-based libraries like QUnit and tape, pure
interception is not always successful - an assertion that fails inside a where
clause will almost always be reported as failed. This behavior will be visited
more thoroughly in the refactoring of the strategy API (see next section...).
where.js comes with several strategies pre-defined for each testing library supported initially. The default strategy is a basic try+catch used by mocha. You do not need to specify a strategy when using mocha.
A strategy can be specified in one of two ways:
- use
{ strategy: '<quoted-name>' }- if the name is available as a global variable (this is true for jasmine)
- if you do not need a reference to that object (that is true for nodeunit).
- use
{ <name>: <named-object-reference> }- when the name is not a global, and/or...
- when your strategy needs a reference to the object (this is true for QUnit and for tape).
The default strategy is a basic try+catch used by mocha. You do not need to specify a strategy when using mocha.
{ strategy: 'mocha' }{ mocha: mocha }(use this whenmochais defined elsewhere in your tests and you wish to use it within the test function itself)
However, unless you are using should.js, you must specify which assertion
method your test relies on:
{ expect: expect }{ assert: assert }{ expect: chai.expect }{ assert: chai.assert }
(Both mocha's should.js and chai's should.js add a should method to
Object.prototype, brilliantly making every object assertable - except those,
not surprisingly, created by the Object.create() method.)
The mocha assert browser tests in this repo rely on
assert.js, "a port of the Node.js standard
assertion library for the browser."
To use the jasmine strategy, add the following to your node suite:
require('[path/to/]/strategy/jasmine-strategy.js');
Because jasmine also uses a try+catch approach, you do not need to specify jasmine as the test strategy. However, it is recommended if you want to intercept failing test messages:
{ strategy: 'jasmine' }when jasmine is defined globally (node, browsers){ jasmine: jasmine }when you need to use jasmine within the where function.
To use the nodeunit strategy, add the following to your node suite:
require('[path/to/]/strategy/nodeunit-strategy.js');
The nodeunit strategy does not use a reference to the nodeunit object, so you need only use the strategy string approach in your tests:
{ strategy: 'nodeunit' }
When you need to use nodeunit within the where() function specify nodeunit with:
{ nodeunit: nodeunit }
To use the QUnit strategy, add the following to your node suite:
require('[path/to/]/strategy/qunit-strategy.js');
When using QUnit, specify the QUnit strategy as follows:
{ QUnit: QUnit }(QUnit is defined globally in both node and browsers)
The QUnit tests in this repo rely on qunit-tap, "A TAP Output Producer Plugin for QUnit."
To use the tape strategy, add the following to your node suite:
require('[path/to/]/strategy/tape-strategy.js');
For use with tape, specify the tape strategy as follows:
{ tape: [test | t] }
where test or t refers to the test function passed in to each tape test:
var test = require('tape');
test('should pass tape context', function(t) { // t is test, test is tape...
var results = where(function(){
/***
| a | b | c |
| 0 | 0 | 0 |
***/
// use the context reference to 'tape' to test itself
tape.equal(tape, context.tape, 'should find tape');
}, { tape: t }); // <= context
t.equal(results.passing.length, 1);
t.end();
});
A copy of my dom-console library is
included in the browser suite for tape, and can be found in the
test/util folder. The dom-console merely writes console.log()
statements to a list in the DOM.
You can define your own strategy for a different library than those supported by where.js.
- call
where.strategy(name, fn);nameis the name of your target test runner (not the expectation library).fnis an initializer or seed function that should return another function ~ which in turn is to be called on each row of where-data.
The following boilerplate shows how to do this.
where.strategy(name, function nameStrategy(context) {
// vars and references
return function nameTest(fnTest, test, value) {
// pre-process blocks here
fnTest.apply({}, [context].concat(value));
// post-process
if (error) {
test.result = 'there was an error';
}
};
});
The 'nameStrategy' function is the initializer, and accepts a context argument
that refers to the context specifier object in where() calls.
The returned 'nameTest' function should accept these three parameters:
fnTest, the wrapped test function that executes once per each row of where-datatest, the current test iteration, for collecting resultsvalues, an array containing the labels used as variables in each test
The post-process block is where to capture failing results and add them to the
test object's .result property.
Beyond this, it's best to look at the various strategy implementations to see which approach would suit your needs. These are located in the strategy directory.
As of Jan 2014, the 3-asterisk comment sequences and line comments are the only items not directly translatable to CoffeeScript.
One workaround is to use CoffeeScript's mechanism for
embedded JavaScript by placing backtick
( ` ) characters before the start /*** and after the end ***/ comment
sequences.
where ->
`/***
a | b | c
1 | 2 | 2
4 | 3 | 4
6 | 6 | 6
***/`
expect(Math.max(a, b)).toBe c
You can also use the triple-quote multi-line string supported by CoffeeScript.
where ->
"""
a | b | c
1 | 2 | 3
4 | 5 | 9
"""
expect(a + b).toBe(c)
which compiles to
where(function() {
"a | b | c\n1 | 2 | 3\n4 | 5 | 9";
return expect(a + b).toBe(c);
});
Thanks to Jason Karns, for this suggestion ~ this is a nicer approach.
To intercept expected failures (using jasmine in this case), this CoffeeScript:
results = where ->
"""
a | b | c
1 | 2 | 3
4 | 5 | 6
"""
expect(a + b).toBe(c)
, { jasmine: jasmine, expect: expect, intercept: 1 }
expect(results.failing.length).toBe(1)
expect(results.passing.length).toBe(1)
compiles to:
var results;
results = where(function() {
"a | b | c\n1 | 2 | 3\n4 | 5 | 6";
return expect(a + b).toBe(c);
}, {
jasmine: jasmine,
expect: expect,
intercept: 1
});
expect(results.failing.length).toBe(1);
expect(results.passing.length).toBe(1);
which is good enough for now.
Line comments inside the multi-line comment are not removed by the CoffeeScript
compiler, but where() will parse for this situation.
In other words, this Coffeescript:
where ->
"""
a | b | c
# 1 | 2 | 3 # should be removed
4 | 5 | 9
"""
expect(a + b).toBe(c)
compiles to
where(function() {
"a | b | c\n# 1 | 2 | 3 # should be removed\n4 | 5 | 9";
return expect(a + b).toBe(c);
});
and where() will replace the characters from # to \n to:
"a | b | c\n 4 | 5 | 9";
The goal of making where.js run in "any" test framework required making numerous versions of test suites. Here's how they stack up:
-
where.js core functionality tests using jasmine-node
npm testnode jasmine-node --verbose ./test/where/where.spec.js
-
framework strategy comparison tests on node.js
npm run jasminenpm run mochanpm run nodeunitnpm run qunitnpm run tape
I'm using Toby Ho's (@airportyh) MAGNIFICENT testem to drive tests in node.js and in multiple browsers.
The core tests use jasmine-2.0.0 (which requires testem v0.6.3 or later) in browsers, and Misko Hevery's (@mhevery) jasmine-node (which uses jasmine 1.3.1 internally) on node.js.
The testem.json file defines launchers for the different frameworks for both
node.js and browser suites:
testem -l where(core tests using jasmine)testem -l jasminetestem -l mochatestem -l nodeunittestem -l qunittestem -l tape(runs browserify on the tape suite)
The package.json file defines scripts to call the testem launchers (with the
appropriate browser test page for each):
npm run testem(core tests using jasmine)npm run testem-jasminenpm run testem-mochanpm run testem-nodeunitnpm run testem-qunitnpm run testem-tape
The browser-suites rely on browser versions of each library stored in the
vendor directory (mocha, expect.js, assert.js, should,
chai, qunit, qunit-tap, and jasmine-2.0.0), rather than your system's /testem
directory, so they can be viewed as static pages.
You can view them directly on rawgit:
Adding a custom adapter for nodeunit to work in testem took half a day to get completely wrong, then a second half-day to get correct. The file is at vendor/testem/nodeunit-adapter.js
I started with jasmine-where back when (October 2013) as a self-assessment and
found both v1.3.1 and v2.0.0-rc3 were a bit tricky but not difficult to
intercept. With jasmine 2.0.0 (first stable in late December) a fundamental
feature was no longer exposed (the currentSpec) which led to - shockingly -
simplifying my own jasmine-where and jasmine-intercept projects. Thinking I was
just not that great at this anymore, I took on the whole where.js concept in
January 2014 to invalidate that bit of self-doubt in a hurry.
what I found out
Test libraries make assumptions. Not all test libraries are designed with the same "lifecycle" in mind, meaning they do not expose hooks at identical stages. When adding message interception support, I found in more than one case that there is no 'after-test-but-before-result' hook that would allow me to scrub expected failures from the reporter.
That indicates some library output reporters are tightly coupled with their test runners. Which means some libraries themselves may not benefit from their own stated purpose - to ease test-driven development.
In addition, not all browsers handle the test lifecycle the same way. This was not a surprise so much as annoyance. For example, in the QUnit suite, IE, Chrome and Opera report two failures whereas Firefox reports only one. However, these failures are expected so we really want to see NO failures because we're trying to intercept them before QUnit sends them to the reporter. QUnit also reported the failures first in the console, regardless of their occurrence in the test suite. (That could be a side-effect of using qunit-tap, however.)
QUnit was surprisingly easy to hook some information, but impossible to trap completely. You can force a failing result in QUnit with this:
QUnit.push(false, 'actual', 'expected', 'force fail message');
But QUnit's legacy lifecycle doesn't expose a way for testers to intercept messages to the HTML reporter (which is just visual rather than logical inspection anyway). So I recommend using it with another reporting utility like qunit-tap so that you can ignore the "should fail" results in the HTML reporter.
Same goes for Tape except that there is no HTML reporter for it, as it is a test library for node.js projects. To run tape in the browser, you need browserify. That's a minor benefit if you prefer the terse syntax of tape. If your emphasis is on browser capabilities rather than node.js, tape should not be your first choice.
Mocha was even easier to work with, probably due to TJ's (@tjholowaychuk) early decision to de-couple the assertion system completely from the test runner and reporter. Taking assert, expect and should to the browser required only one library to be ported - assert.js. Working with chai was even easier - no port needed for the browser.
The browser version that ships with nodeunit is based on an earlier version of
that library. After trying the make command on it in my Windows setup and
finding yet again that some people just can't be bothered to test anything on
Windows, I've updated the key methods with the proper versions in the core
library. That's in the vendor/nodeunit directory.
- clean up the procedural long-method setup code in the where() function
- clean up the procedural verbose documentation