Resetting data from pogdesign.co.uk/cat/
I am trying to reverse some data from
http://www.pogdesign.co.uk/cat/
.
I want to get the channel and airtime of each program, but the problem is that they are not displayed by default. Only after manually adjusting the parameters and saving them will the channel and air time of each program appear.
As I understand it, after checking the "Network" section in Chrome developer tools, what really happens after I click "Save Settings" is that a POST request is sent with the appropriate data parameters (like 's_networks':'on'
etc ') , then a GET request is sent to get the html file with the channel and the displayed air time.
I tried to mimic this process (POST request then GET request) using both python requests
package and package mechanicalsoup
.
requests:
s = requests.Session()
s.post('http://www.pogdesign.co.uk/cat/', data = {'s_networks':'on'})
s.get('http://www.pogdesign.co.uk/cat/')
mechanicalsoup:
mcs = mechanicalsoup.Browser()
res_post = mcs.post('http://www.pogdesign.co.uk/cat/', data {'s_networks':'on'})
res_get = mcs.get('http://www.pogdesign.co.uk/cat/')
However, the response received does not contain the channel and airtime data.
The only difference I noticed is that the status code returned from the browser POST request is equal 302
, and the status code returned from my python requests is 200
.
source to share
This is because of the cookie that stores user information, you can try the following code
import requests
s = requests.Session()
data = {
"style": 3,
"timezone": "GMT",
"s_numbers": "on",
"s_epnames": "on",
"s_airtimes": "on",
"s_popups": "on",
"s_wunwatched": "on",
"s_sortbyname": "on",
"s_weekstyle": "on",
"s_24hr": "on",
"settings": None
}
cookies = { # you can get the cookie info from dev tool
"CAT_UID":'' ,
"PHPSESSID":'' ,
"_ga": '',
"_gid": '',
"_gat": ""
}
post = s.post('http://www.pogdesign.co.uk/cat/', data=data, cookies=cookies)
text = post.text
get = s.get('http://www.pogdesign.co.uk/cat/', cookies=cookies)
text1 = get.text
source to share