How do I make Symfony 2 Crawler accept cookies?

I am working on a crawler to crawl a website with a listing. I finished scanning and scraping, but I ended up in a road block.

The pagination on this website is cookie (or session) based, so I have to force my crawler to accept any cookies the website sets (submissions) and behave like a browser on further requests.

I am new to symphony and I cannot find a source explaining this clearly.

I found the following links while trying to find a solution. But they're not exactly what I'm looking for.

http://www.webtipblog.com/creating-a-crawler-in-symfony-2-using-the-domcrawler-and-client/

http://symfony.com/doc/current/cookbook/testing/simulating_authentication.html

Can anyone tell me how can I do this?

Many thanks.

+3


source to share





All Articles