Selenium and Concurrent JUnit - WebDriver Instances
Setting up
So, basically I'm trying to run Selenium tests that run in parallel using JUnit.
I found this JUnit runner for this . It works really well, I love it.
However, I am having problems handling WebDriver instances.
What I want
Each WebDriver element must be instantiated once per class before the methods are executed @Test
.
Logically, I could use the class constructor for this. This is actually a requirement for my tests because I need to use @Parameters
so that I can instantiate the WebDriver appropriately (Chrome, FF, IE ...).
Problem
The problem is, I want the WebDriver instance to be cleaned up ( driver.quit()
) after the class ends , not after the method@Test
. But I cannot use @AfterClass
because I cannot make the WebDriver a static member since each instance of the class must use its own (otherwise the tests will run in the same browser).
Possible Solution
I found a possible suggestion here by Munnal Gosar. Following his recommendations, I changed the WebDriver instead static ThreadLocal<WebDriver>
and then instantiate it in each constructor using
// in the classes constructor
driver = new ThreadLocal<WebDriver>() {
@Override
protected WebDriver initialValue() {
return new FirefoxDriver(); /
}
};
Needle says I have replaced every driver.whatever
call driver.get().whatever
in my code.
Now, to solve the ultimate purpose of this, I also wrote a method @AfterClass
that will call driver.get().quit();
, which is now accepted by the compiler since the variable is static.
Testing this, however, results in unexpected behavior. I have a Selenium Grid setup with 2 nodes running on a remote machine. I had this setup as expected before, but now browsers are spamming and tests fail. (Whereas two browsers should be running instead of 8+ open)
A linked thread suggesting this solution noticed that it is not a good idea to manually handle threads if already using a framework like JUnit.
My question
What's the correct construction for this?
I could only think of
- Do what is suggested here.
- Write one @Test annotated method that does all the other methods and then uses @After to achieve the same as @AfterClass
- Store the constructor parameter in a member variable and consider the fact that I have to create a browser before executing each annotated method
@Test
(using@Before
to instantiate WebDriver and@After
to close the session)
I don't quite understand if there is option 3 in the possible problems. If I close the session after each method, then grid-Server can actually open a new session with a completely new class on that node until it completes the previous ones. Although the tests are independent of each other, I still feel this is a potential hazard.
Does anyone here actively use Selenium multi-threaded benchmark and can guide me on what is the correct design?
source to share
In general, I agree that:
it might be a bad idea to manually handle streams if already used as JUnit
But looking at the Parallelized
runner you talked about and the internal implementation @Parametrized
in junit 4.12 it is possible.
Each test case is scheduled for execution. By default junit runs test cases in a single thread. Parallelized
extends Parametrized
in such a way that the single-threaded test planner is replaced by the multi-threaded test planner, so to understand how this affects test case execution Parametrized
, we need to look inside the JUnit sources Parametrized
:
Looks like:
-
@Parametrized
the test case is split into a groupTestWithParameters
for each test parameter - for each
TestWithParameters
instanceRunner
is created and assigned for execution (in this case, theRunner
instance is specializedBlockJUnit4ClassRunnerWithParameters
)
Basically, each @Parametrized test case generates a bunch of test instances to run (one instance for each parameter) and each instance is scheduled independently , so in our case ( Parallelized
and a @Parametrized
test with instances WebDriver
as parameters), WebDriver
multiple independent tests will run in each separate type. in dedicated streams. And this is important because it allows us to store a specific instance WebDriver
in the scope of the current thread.
Please keep in mind that this behavior depends on the internal implementation details of junit 4.12 and may change (for example, see comments in RunnerScheduler
).
Take a look at the example below. It builds on the mentioned JUnit behavior and uses ThreadLocal
to store instances WebDriver
shared between test in the same case groups. The only trick with is ThreadLocal
to initialize it only once (in @Before) and destroy every instance created (in @AfterClass).
package example.junit;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.Iterator;
import java.util.List;
import org.apache.commons.lang3.StringUtils;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
/**
* Parallel Selenium WebDriver example for http://stackoverflow.com/questions/30353996/selenium-and-parallelized-junit-webdriver-instances
* Parallelized class is like http://hwellmann.blogspot.de/2009/12/running-parameterized-junit-tests-in.html
*/
@RunWith(Parallelized.class)
public class ParallelSeleniumTest {
/** Available driver types */
enum WebDriverType {
CHROME,
FIREFOX
}
/** Create WebDriver instances for specified type */
static class WebDriverFactory {
static WebDriver create(WebDriverType type) {
WebDriver driver;
switch (type) {
case FIREFOX:
driver = new FirefoxDriver();
break;
case CHROME:
driver = new ChromeDriver();
break;
default:
throw new IllegalStateException();
}
log(driver, "created");
return driver;
}
}
// for description how to user Parametrized
// see: https://github.com/junit-team/junit/wiki/Parameterized-tests
@Parameterized.Parameter
public WebDriverType currentDriverType;
// test case naming requires junit 4.11
@Parameterized.Parameters(name= "{0}")
public static Collection<Object[]> driverTypes() {
return Arrays.asList(new Object[][] {
{ WebDriverType.CHROME },
{ WebDriverType.FIREFOX }
});
}
private static ThreadLocal<WebDriver> currentDriver = new ThreadLocal<WebDriver>();
private static List<WebDriver> driversToCleanup = Collections.synchronizedList(new ArrayList<WebDriver>());
@BeforeClass
public static void initChromeVariables() {
System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
}
@Before
public void driverInit() {
if (currentDriver.get()==null) {
WebDriver driver = WebDriverFactory.create(currentDriverType);
driversToCleanup.add(driver);
currentDriver.set(driver);
}
}
private WebDriver getDriver() {
return currentDriver.get();
}
@Test
public void searchForChromeDriver() throws InterruptedException {
openAndSearch(getDriver(), "chromedriver");
}
@Test
public void searchForJunit() throws InterruptedException {
openAndSearch(getDriver(), "junit");
}
@Test
public void searchForStackoverflow() throws InterruptedException {
openAndSearch(getDriver(), "stackoverflow");
}
private void openAndSearch(WebDriver driver, String phraseToSearch) throws InterruptedException {
log(driver, "search for: "+phraseToSearch);
driver.get("http://www.google.com");
WebElement searchBox = driver.findElement(By.name("q"));
searchBox.sendKeys(phraseToSearch);
searchBox.submit();
Thread.sleep(3000);
}
@AfterClass
public static void driverCleanup() {
Iterator<WebDriver> iterator = driversToCleanup.iterator();
while (iterator.hasNext()) {
WebDriver driver = iterator.next();
log(driver, "about to quit");
driver.quit();
iterator.remove();
}
}
private static void log(WebDriver driver, String message) {
String driverShortName = StringUtils.substringAfterLast(driver.getClass().getName(), ".");
System.out.println(String.format("%15s, %15s: %s", Thread.currentThread().getName(), driverShortName, message));
}
}
It will open two browsers and simultaneously execute three test cases in each browser window.
The console will print something like:
pool-1-thread-1, ChromeDriver: created
pool-1-thread-1, ChromeDriver: search for: stackoverflow
pool-1-thread-2, FirefoxDriver: created
pool-1-thread-2, FirefoxDriver: search for: stackoverflow
pool-1-thread-1, ChromeDriver: search for: junit
pool-1-thread-2, FirefoxDriver: search for: junit
pool-1-thread-1, ChromeDriver: search for: chromedriver
pool-1-thread-2, FirefoxDriver: search for: chromedriver
main, ChromeDriver: about to quit
main, FirefoxDriver: about to quit
You can see that the drivers are created once per worker thread and destroyed at the end.
To summarize, we need something like @BeforeParameter
and @AfterParameter
in the context of a thread of execution, and a quick search shows that such an idea is already registered as a problem in Junit
source to share