Run all tests marked with @Ignore
I want to make a Jenkins job to run an ant task to run all tests in my codebase that are tagged @Ignore because using annotations like @Category (IgnoredTest.class) doesn't work with our test run parallelization. After a lot of searching, it looks invalid, but I still have hope. Help?
JUnit 4.10
source to share
I'm not sure what's stopping you from doing "test run parallelization", but you can do it with a rule if you want to use a custom "ignore" annotation instead of JUnit. The reason for this is because JUnit processes @Ignored tests at the Runner level, especially in the BlockJUnit4ClassRunner.runChild()
(default). If you could find a way to use a custom Runner in Ant, you could come up with one to suit your needs quite easily, but I don't know if it's easy to do it in Ant.
As I mentioned above, you can easily use different annotations and rules to choose which methods to run. I have compiled a quick example of such a rule for github and also a that uses it . My little example uses a system property to switch, but you can also force it to switch to anything you can imagine that you can get your hands on here.
You can clone and run this example with:
git clone git@github.com:zzantozz/testbed tmp
cd tmp
mvn test -pl stackoverflow/9611070-toggleable-custom-ignore -q
mvn test -pl stackoverflow/9611070-toggleable-custom-ignore -q -D junit.invertIgnores
The only downside to this approach I can think of is that your tests will not be correctly marked as "ignored" because that is also done BlockJUnit4ClassRunner.runChild()
, and if you look at ParentRunner.runLeaf()
(to which runChild()
delegates) you will see that the notifier, which is that you need to report ignored tests is not conveyed far enough to be used by the Rule. Again, this is what you'll need a custom Runner for.
source to share
You can create a custom ant target that will remove the @Ignore annotation and add the @ignore annotation to any active @Test annotated method
the goal would be something like this:
<project name="anyname" default="test" basedir=".">
..
..
<target name="enable-ignored-test" depends="copy-code-to-replace-ignored">
<fileset id="fsTestCase" dir="." includes="**/*Test.java">
</fileset>
<replaceregexp flags="gm">
<regexp pattern="^@Ignore"/>
<substitution expression=""/>
<fileset refid="${fsTestCase}"/>
</replaceregexp>
<replaceregexp flags="gm">
<regexp pattern="@Test"/>
<substitution expression="@Ignore @Test"/>
<fileset refid="${fsTestCase}"/>
</replaceregexp>
</target>
<target name="run-ignored-tests" depends="enable-ignored-test,test" />
..
..
</project>
source to share