Be in multiple browsers in parallel
We are currently using Behat 3 to automate BDD tests for our site.
The current setup uses Jenkins to run Selenium , which attaches to Firefox and uses XVFB for rendering (this allows us to save screenshots if something goes wrong).
This is great for testing that the site (including JavaScript) is working and that the user can successfully complete each documented task.
I want to expand our testing capabilities, and one thing I would like to add is the ability to test multiple browsers. This is very important as we get random quirks that can break functionality.
Since the tests currently take a little over an hour to run (and we have 4 kits for this Jenkins site), I would prefer to run all browsers at the same time. If I can't find a way to do this at the same time, then I'll probably just set up multiple Behat profiles and run each one in sequence.
One thing I considered as a possible solution was Ghostlab . This will allow us to simultaneously test several browsers and several devices, including mobile ones. The problem is I can't seem to find a way to join this on Behat in a meaningful way.
I can launch one browser connected to Ghostlab, which will cause the same actions to be taken in all connected browsers, however, it was a browser other than the one controlled by Selenium to break, I don't know how we will capture this information.
TL; DR: Is there a way to run BDD (preferred Behat) tests in multiple browsers in parallel and grab information from any browser that doesn't work?
source to share
This is what the multiconfiguration jobs (or matrix jobs) are for Jenkins.
You set the job configuration once, but add one or more variables that must change each time you create the combination matrix (in your case, the matrix is one size: the browser).
Jenkins then runs one main assembly with multiple parallel rows - one for each combination in the matrix. Then you can clearly see the results for each combination.
This requires that your test job can be parameterized, i.e. you can choose at runtime where the browser should run rather than running all tests together in one job.
The Jenkins wiki has minimal documentation on this feature, but there are some good blog posts (and questions) out there on how to set it up.
The matrix job will use all available "executors" in Jenkins to run builds in parallel as much as possible.
There are two runners in the default Jenkins installation, but you can change this or extend Jenkins by adding further machine builds .
source to share