Popup handling in Selenium phantomJS
I am creating a web crawler in python selenium phantomJS that visits links, takes a screenshot of the entire webpage and saves it to AWS s3. Unfortunately, on some sites I get stuck with the "Please subscribe to our newsletter" popup and despite my best efforts, I cannot find a solution to shut it down. This is how it looks:
From their documentation ( http://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.common.alert ) I tried alert, but it doesn't seem to work. I need to handle both HTML and JavaScript popups.
Here's a shorthand example of the code I'm working with:
from selenium import webdriver
from pyvirtualdisplay import Display
display = Display(visible=0, size=(1600, 1200)) # Xvfb headless display
display.start()
capabilities = {'handlesAlerts': 'True'}
driver = webdriver.PhantomJS(service_log_path=os.path.devnull, desired_capabilities=capabilities)
driver.get('https://www.everlane.com/products/mens-slim-fit-oxford-shirt2-grey?collection=mens-shirt-shop')
alert = driver.switch_to_alert()
alert.accept()
driver.save_screenshot()
I also tried:
driver.switch_to_default_content()
Keyboard "ESC", "ENTER", "F4" also do not work.
I can find the XPath element and click to close, but I need a general purpose solution because I am working with multiple sites and therefore multiple XPaths.
Any help is appreciated! I may not be the only one with this problem, so I hope others find it useful. Thank you.
No one has answered this question yet
Check out similar questions: