Run all tests:
npm test
To run all tests on PostgreSQL as in the OurBigBook Web, first setup the PostgreSQL database similarly to local run as identical to deployment as possible:
and then run with:
createdb ourbigbook_cli
psql -c "CREATE ROLE ourbigbook_user with login password 'a'"
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook_cli TO ourbigbook_user'
npm run test-pg
List all tests:
as per: stackoverflow.com/questions/41380137/list-all-mocha-tests-without-executing-them/58573986#58573986.
node node_modules/mocha-list-tests/mocha-list-tests.js main.js
Run just one test by name:
as per: stackoverflow.com/questions/10832031/how-to-run-a-single-test-with-mocha todo: what if the test name is a substring? You will want this Bash alias:
which allos you to just:
npm test -- -g 'one paragraph'
npmtg() ( npm test -- -g "$*" )
npmtg one paragraph
Run all tests that don't start with
This works because
cli:
:
npm test -- -g '^(?!cli:)'
-g
takes JavaScript regular expressions, so we can use negative lookahead, see also: stackoverflow.com/questions/26908288/with-mocha-how-do-i-run-all-tests-that-dont-have-slow-in-the-nameThere are two types of test in our test suite:
- tests that call the
ourbigbook.convert
JavaScript API directly.These tests don't actually create files in the filesystem, and just mock the filesystem instead with a dictionary.Database access is not mocked however, we just use Sqlite's fantastic in-memory mode.Whenever possible, these tests check their results just from the abstract syntax tree tree returned by the API, which is cleaner than parsing the HTML. But sometimes HTML parsing is inevitable. - tests that call the
ourbigbook
executable itself:- their titles are prefixed with
cli:
- they tend to be a lot slower than the API test
- can test functionality that is done outside of the
ourbigbook.convert
JavaScript API, notably stuff prevent in ourbigbook, so they are more end to end - don't do any mocking, and could therefore be more representative.However, as of 2022, we have basically eliminated all the hard database access mocking and are using the main database methods directly.So all that has to be mocked is basically stuff done in the ourbigbook executable itself.This means that except for more specific options, the key functionality of ourbigbook, which is to convert multiple paths, can be done very well in a non executable test.The only major difference is that instead of passing command line arguments like in
ourbigbook .
to convert multiple files in a directory, you have to useconvert_before
andconvert_before_norender
and specify conversion order one by one.This test robustness is new as of 2022, and many tests were previously written with executable that would now also work as unit tests, and we generally want that to be the case to make the tests go faster. - work by creating an actual physical filesystem under
out/test/<normalized-test-title>
with the OurBigBook files and other files likeourbigbook.json
, and then running the executable on that directory.npm test
first deletes theout/test
directory before running the tests. After running, the generated files are kept so you can inspect them to help debug any issues. - all these tests check their results by parsing the HTML and searching for elements, since here we don't have access to the abstract syntax tree. It wouldn't be impossible to obtain it however, as it is likely already JSON serializable.
- their titles are prefixed with
Step debug during a test run. Add the statement:
to where you want to break in the code, and then run:
where
debugger;
npm run testi -- -g 'p with id before'
i
in testi
stands for inspect
from node inspect
. Also consider the alias:
npmtgi() ( npm run testi -- -g "$*" )
Note however that this does not work for tests that run the
but not working, related: stackoverflow.com/questions/23612087/gulp-target-to-debug-mocha-tests So for now, we are just printing the command being run as in:
so you can just re-run it manually with
This works since the
ourbigbook
executable itself, since those spawn a separate process. TODO how to do it? Tried along:
const out = child_process.spawnSync('node', ['inspect', 'ourbigbook'].concat(options.args), {
cwd: tmpdir,
input: options.stdin,
stdio: 'inherit',
});
cmd: cd out/test/executable-ourbigbook.json-outputOutOfTree && ourbigbook --split-headers .
node inspect
as in:
cd out/test/executable-ourbigbook.json-outputoutoftree && node inspect "../../../ourbigbook" --split-headers .
tmp
directory is not deleted in case of failure.