R - web scraper with https login form
I am trying to submit a login form via R
so that I can run some web scrapers (which are not available if I am not logged in). I thought everything was working for me, but whenever I try to access the registration page, it returns to the login page (i.e., the message form doesn't work or doesn't recognize that I'm already signed in). Could it be cookies?
This is what I have used so far rcurl
:
loginurl = "https://shop.supervalu.ie/shopping/StartShopping/login.aspx"
dataurl = "https://shop.supervalu.ie/shopping/shopping/shop.aspx?catid=150300005&PDEPTID=8"
pars=list(
`ctl00$ctl00$cphContent$cphMain$ucntrlLogin$txtEmailAddress` = "email",
`ctl00$ctl00$cphContent$cphMain$ucntrlLogin$txtPassword` = "password",
`ctl00$ctl00$cphContent$cphMain$ucntrlLogin$btnLogin` = "LOGIN"
)
agent="Mozilla/5.0"
curl = getCurlHandle()
curlSetOpt(cookiejar="cookies.txt", useragent = agent, followlocation = TRUE, curl=curl)
html=postForm(loginurl, .params = pars, curl = curl)
html=getURL(dataurl, curl=curl)
+3
source to share
No one has answered this question yet
Check out similar questions: