Disable automatic test execution for some tests unless explicitly requested
I have some very slow tests and many short unittests. I would like to be able to run only short unittests with a simple command nosetests
, and if I decide to run slow tests to be able to call them explicitly.
What I would like to do:
-
run unittests, but not slow tests
$ nosetests
No special command is used - anyone entering
nosetests
just out of curiosity will be executed in a few seconds. -
explicitly request slow tests:
$ nosetests --somehow-magicaly-invoke-slow-tests
No ambiguity - I want slow tests (or unittests and a slow test - it doesn't matter)
What I have tried:
I tried using nose.plugins.attrib
:
from nose.plugins.attrib import attr
import time
@attr('slow')
def test_slow_one():
time.sleep(5)
def test_unittest():
assert True
but actually this is almost the opposite of what I am trying to accomplish - I have to specify additional command line parameters to not run slow tests. Commands:
Run unittests, but not slow tests
$ nosetests -v -a '!slow'
test_a.test_unittest ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
explicitly request slow tests:
$ nosetests -v -a 'slow'
test_a.test_slow_one ... ok
----------------------------------------------------------------------
Ran 1 test in 5.005s
OK
And to make things worse when someone only runs
$ nosetests -v
test_a.test_slow_one ... ok
test_a.test_unittest ... ok
----------------------------------------------------------------------
Ran 2 tests in 5.007s
OK
all tests , including slow ones, will run.
Question:
Is there a way to disable some tests so that they are not called with a simple command nosetests
, but can be executed with some additional command line options?
Current "solution":
I just moved all the slow tests to separate files and named the files check_*.py
instead test_*.py
so nosetests
won't pick them up. When I want to run slow tests, I just give the entire file path check_*.py
, for example:
$ nosetests test/check_foo.py test/check_bar.py [...]
which is not very elegant.
source to share
You can use easily unittest.skipUnless()
.
Skipping single tests
Just beautify the methods of your test case that you want to conditionally skip with
@unittest.skipUnless(condition, reason)
For example, you can check the environment variable SLOW_TESTS
which you just don't install in your automated CI environment, but install if and when you want to run your slow tests locally:
import os
import time
import unittest
SLOW_TESTS = os.environ.get('SLOW_TESTS', False)
class TestFoo(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
@unittest.skipUnless(SLOW_TESTS, "slow test")
def test_something_slow(self):
time.sleep(5)
self.assertTrue(True)
Conclusion to run regularly:
$ nosetests -v
test_something_slow (test_foo.TestFoo) ... SKIP: slow test
test_upper (test_foo.TestFoo) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.002s
OK (SKIP=1)
Output when setting an environment variable:
$ SLOW_TESTS=1 nosetests -v
test_something_slow (test_foo.TestFoo) ... ok
test_upper (test_foo.TestFoo) ... ok
----------------------------------------------------------------------
Ran 2 tests in 5.003s
OK
(Note that the nose tester still speaks Ran 2 tests
even if he missed one. The skipped tests are indicated at the very end with (SKIP=n)
and in the test results with SKIP
or S
in non-verbose mode).
Of course, you can invert the behavior by using skipIf()
using an environment variable of the type FAST_TESTS
you set in the CI setup.
Skipping the entire test system
If you want to skip all tests in TestCase that you know to be slow or have heavy configuration, it might be more convenient to call TestCase.skipTest()
explicitly (you can also do this from a single test if you need finer grained control):
class TestFoo(unittest.TestCase):
def setUp(self):
if not SLOW_TESTS:
self.skipTest("slow test")
# some expensive setup
For more information on skipped tests, see Skipping tests and expected failures .
source to share