The Kuma test suite¶
Kuma has a fairly comprehensive Python test suite. Changes should not break tests. Only change a test if there is a good reason to change the expected behavior. New code should come with tests.
Commands should be run inside the development environment, after
with Docker, or
vagrant ssh with Vagrant.
Before you run the tests, you have to build assets:
Running the test suite¶
If you followed the steps in the installation docs, then all you should need to do to run the test suite is:
The default options for running the test are in
pytest.ini. This is a
good set of defaults.
If you ever need to change the defaults, you can do so at the command line by running what the Make task does behind the scenes:
Some helpful command line arguments to py.test (won’t work on
- Drop into pdb on test failure.
- Create a new test database.
- Shows local variables in tracebacks on errors.
- Exits on the first failure.
py.test --help for more arguments.
Running subsets of tests and specific tests¶
There are a bunch of ways to specify a subset of tests to run:
only tests marked with the ‘spam’ marker:
py.test -m spam
all the tests but those marked with the ‘spam’ marker:
py.test -m "not spam"
all the tests but the ones in
py.test --ignore kuma/core
all the tests that have “foobar” in their names:
py.test -k foobar
all the tests that don’t have “foobar” in their names:
py.test -k "not foobar"
tests in a certain directory:
See http://pytest.org/latest/usage.html for more examples.
Showing test coverage¶
While running the tests you can record which part of the code base is covered by test cases. To show the results at the end of the test run use this command:
To generate an HTML coverage report, use:
The test database¶
The test suite will create a new database named
whatever value you have for
sure the user has
ALL on the test database as well.
for the list of available markers.
To add a marker, add it to the
To use a marker, add a decorator to the class or function. Examples:
import pytest @pytest.mark.spam class SpamTests(TestCase): ... class OtherSpamTests(TestCase): @pytest.mark.spam def test_something(self): ...
Code should be written so that it can be tested, and then there should be tests for it.
When adding code to an app, tests should be added in that app that cover the
new functionality. All apps have a
tests module where tests should go. They
will be discovered automatically by the test runner as long as the look like a
If you’re expecting
reverse to return locales in the URL
instead of the default client for the
Unless the current behavior, and thus the test that verifies that behavior is correct, is demonstrably wrong, don’t change tests. Tests may be refactored as long as it’s clear that the result is the same.
On those rare, wonderful occasions when we get to remove code, we should remove the tests for it, as well.
If we liberate some functionality into a new package, the tests for that functionality should move to that package, too.