Can't login to site using Python requests session module
I'm just getting started with web scraping. For my first project, I am trying to login to artofproblemsolving.com using request.Session () and access a different user account. Here is my code:
import requests
LOGIN_URL = 'https://www.artofproblemsolving.com/Forum/ucp.php?mode=login'
DATA_URL = 'https://www.artofproblemsolving.com/Forum/memberlist.php?mode=viewprofile&u=90586'
payload = {
'username': '{{my_username}}',
'password': '{{my_password}}'
}
with requests.Session() as s:
s.post(LOGIN_URL, data=payload)
r = s.get(DATA_URL)
print r.text
But when I run this in a terminal, the HTML output is on the login page, not the user profile I'm trying to grab. I double checked that LOGIN_URL is the POST action on the login form, that "username" and "password" are the names of the form elements, and that my login details are correct.
All help is appreciated, thanks!
source to share
I'm not sure if this is the direct cause of the problem. But there is one more parameter that the form sends for login.
Next to username
, the password
following parameters also pass:
'username': username,
'password': password,
'login': 'Login'.
'sid': sid, # You need to parse the login page to get sid
'redirect': 'index.php',
source to share