Estoy intentando conectarme a través de requests a la web Paf
Para ello esto es lo que hago.
url = "https://www.paf.es/player/authentication/authentication/login"
headers = {
'authority': 'www.paf.es',
'accept': 'application/json, text/plain, */*',
'x-requested-with': 'XMLHttpRequest',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36',
'content-type': 'application/x-www-form-urlencoded;charset=UTF-8',
'origin': 'https://www.paf.es',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.paf.es/',
'accept-language': 'es-ES,es;q=0.9,en;q=0.8',
'cookie': '__cfduid=d4247b3fce5d260d7c5257b5d65a572001592254859; com.paf.frontend.common.LocaleCookie=es_ES; com.paf.frontend.common.LifeCycleCookie=PROSPECT; com.paf.frontend.common.device=desktop; com.paf.frontend.common.tid=e594b8dd-9e06-4a82-a06a-a09bb66da49d; JSESSIONID=ldq5yQi8ECI61icXHNOseCcAelTpGUW2l_okMkxEL0zBJAQCD-jfu0021151447412; com.paf.frontend.common.journeycookie=paf-program-id=6331&paf-program-id-hash=6032d61e305edfbd5a1e6ab5d6d667f76098fbac4ce8a99c6dc68504a05642ed; BIGipServerprod01_pool=1067697930.22811.0000; __cf_bm=5eab9f3849498c5dbca21928e913480809882400-1592254860-1800-ATxl1o2pNsPocq+5ZfuH2vfeFL+kvZlCg2NgwQ4K3mBF/XfI0HcUPcuIjpAmy1MExYuswC3JLcWZQHLhQC2aDl8=; LaVisitorNew=Y; LaVisitorId=ol7l89e9rynpjs165thh765oc9ujt; LaSID=qugayn3m2tpvsmh1dgpuk36bkmbzh; _gcl_au=1.1.193127229.1592254863; _ga=GA1.2.484179826.1592254863; _gid=GA1.2.1016568931.1592254863; trackingParams={"_ga":{"value":"GA1.2.484179826.1592254863","expiration":1600030862876},"utm_nooverride":{"value":"1","expiration":1600030862876}}; _gat_UA-641842-15=1; _fbp=fb.1.1592254863595.468566668; com.paf.frontend.cookiesAccepted=true; com.paf.frontend.common.showMenu=account; JSESSIONID=XnC6Utz7YE878OEcJjxc8snsO2eBLXmslLBjJf5tYDJvA2cq9-aB!151447412; com.paf.frontend.common.LocaleCookie=es_ES; com.paf.frontend.common.LoginTime=1592263892354; com.paf.frontend.common.LoggedIn=true; __cfruid=921905b1799fb2de66fc3378b12a65be27868140-1592263892'
}
payload={
'loginSite': 'NCI',
'captchaToken': '',
'username': '[email protected]',
'password': 'password',
}
requests.post(url=url,headers=headers,data=json.dumps(payload))
Puesto que el servidor de la web es cloudflare, esto da error 403. Para solventarlo, en lugar delrequest normal utilizo la libreria cloudscraper. En esencia, es como un request session.
scraper=cloudscraper.create_scraper()
a=scraper.post(url=url,headers=headers,data=json.dumps(payload))
Resuelto, 200 y esta es la respuesta obtenida
a.content
b'{"serviceMessage":"login_successful","authenticationSuccessfulLocale":"es_ES","loginCount":85}'
a.headers
{'Date': 'Sun, 21 Jun 2020 17:53:37 GMT', 'Content-Type': 'application/json;charset=UTF-8',
'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'CF-Ray': '5a6f970f6b74ff14-MAD',
'Cache-Control': 'no-cache, no-store, no-transform, must-revalidate',
'Set-Cookie': 'JSESSIONID=oMXYA6XS8bGnYJsWOB1qWuIi-L4KnF5Dec6hWg4iInb5hq6yWZIQ!-1530641001; path=/; HttpOnly, com.paf.frontend.common.LocaleCookie=es_ES; path=/, com.paf.frontend.common.LoginTime=1592762017437; path=/, com.paf.frontend.common.LoggedIn=true; path=/, __cf_bm=a0ccff455f6db87a1ecbcd05bcbd6c33e944efe4-1592762017-1800-Aa/oxP4jXRZ26gckyxqkFqNfrtQ4bqQNHfF4uZdgvgSjcJA38uS8Tyz+EEFD6OAfEW61h6WiFd6z4VQv7RzBG3k=; path=/; expires=Sun, 21-Jun-20 18:23:37 GMT; domain=.paf.es; HttpOnly; Secure; SameSite=None',
'Strict-Transport-Security': 'max-age=15552000; includeSubDomains; preload', 'CF-Cache-Status': 'DYNAMIC',
'cf-request-id': '03799cbd9d0000ff14a9bf6200000001',
'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"',
'X-Content-Type-Options': 'nosniff', 'Vary': 'Accept-Encoding', 'Server': 'cloudflare',
'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400'}
A partir de aquí, la página hace varias consultas, y cuando intento replicar una serie de ellas es cuando obtengo el 401. En el siguiente pantallazo
Hasta la consulta notifications?norefresh no tengo ningún problema, pero cuando hago la marcada en amarillo, necesaria para obtener un token para poder seguir haciendo consultas, es cuando obtengo el 401.
urlu = "https://www.paf.es/player/player/public/playerinformation"
headers = {
'authority': 'www.paf.es',
'accept': 'application/json, text/plain, */*',
'path':'player/player/public/playerinformation',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36',
'x-requested-with': 'XMLHttpRequest',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.paf.es/my-paf',
'accept-language': 'es-ES,es;q=0.9,en;q=0.8',
'cookie': '__cfduid=d4247b3fce5d260d7c5257b5d65a572001592254859; com.paf.frontend.common.LocaleCookie=es_ES; _gcl_au=1.1.193127229.1592254863; _ga=GA1.2.484179826.1592254863; _fbp=fb.1.1592254863595.468566668; com.paf.frontend.cookiesAccepted=true; com.paf.frontend.common.LifeCycleCookie=HAS_LOGGED_IN; com.paf.frontend.common.device=desktop; BIGipServerprod01_pool=1050920714.22811.0000; _gid=GA1.2.1224451480.1592762649; com.paf.frontend.common.showMenu=account; __cfruid=e82d9eabd02b2dab828eb8dc8fa3a585671ac853-1592762678; __cf_bm=a26a133bbd4922137879e294f82d3002b753d382-1592772326-1800-AXt+QmNX9+Xr6/6dngMQgWKLqpBjRbsl1fKH5wb99CgD8KHE6U1cPSfPzq4j4TTVY46K+9pRGtrzT60GV+o5974=; _gali=forceLoginButton; JSESSIONID=2M_YpHqxlclwh930HKxIunh2uanUUlH60MvjhtkxnrbgB7SatGZeu0021-1115016185; com.paf.frontend.common.LoggedIn=true; com.paf.frontend.common.LoginTime=1592772552517; trackingParams={"_ga":{"value":"GA1.2.484179826.1592254863","expiration":1600548553385},"utm_nooverride":{"value":"1","expiration":1600548553385}}; _gat_UA-641842-15=1; __cfduid=d8748223ca6c38ac4af774df938e4ff471592763354'
}
b=scraper.get(url=urlu,headers=headers)
>401
b.content
b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Draft//EN">\n<HTML>\n<HEAD>\n<TITLE>Error 401--Unauthorized</TITLE>\n</HEAD>\n<BODY bgcolor="white">\n<FONT FACE=Helvetica><BR CLEAR=all>\n<TABLE border=0 cellspacing=5><TR><TD><BR CLEAR=all>\n<FONT FACE="Helvetica" COLOR="black" SIZE="3"><H2>Error 401--Unauthorized</H2>\n</FONT></TD></TR>\n</TABLE>\n<TABLE border=0 width=100% cellpadding=10><TR><TD VALIGN=top WIDTH=100% BGCOLOR=white><FONT FACE="Courier New"><FONT FACE="Helvetica" SIZE="3"><H3>From RFC 2068 <i>Hypertext Transfer Protocol -- HTTP/1.1</i>:</H3>\n</FONT><FONT FACE="Helvetica" SIZE="3"><H4>10.4.2 401 Unauthorized</H4>\n</FONT><P><FONT FACE="Courier New">The request requires user authentication. The response MUST include a WWW-Authenticate header field (section 14.46) containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field (section 14.8). If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity MAY include relevant diagnostic information. HTTP access authentication is explained in section 11.</FONT></P>\n</FONT></TD></TR>\n</TABLE>\n</BODY>\n</HTML>\n'
b.headers
{'Date': 'Sun, 21 Jun 2020 22:26:16 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'CF-Ray': '5a71267249cdd689-MAD', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains; preload', 'WWW-Authenticate': 'Basic realm="weblogic"', 'CF-Cache-Status': 'DYNAMIC', 'cf-request-id': '037a965b6a0000d6897b8cc200000001', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'X-Content-Type-Options': 'nosniff', 'Set-Cookie': '__cf_bm=d2add1f10df0866510c8b614b3946a34aedadd4d-1592778376-1800-AaSI30ASGwIA3BXp5S616TWB026u4JxsPM/Ct4Ktx9fDhPfZQOZAgCgYTiiu8JX99si6JmTSpnDzEZ5o2ynt0Vk=; path=/; expires=Sun, 21-Jun-20 22:56:16 GMT; domain=.paf.es; HttpOnly; Secure; SameSite=None', 'Vary': 'Accept-Encoding', 'Server': 'cloudflare', 'alt-svc': 'h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400'}
He probado:
scraper.get(url=urlu,headers=headers,auth=HTTPBasicAuth('user','pass'))
scraper.get(url=urlu,headers=headers,auth=HTTPDigestAuth('user','pass'))
scraper.get(url=urlu,headers=headers,data=json.dumps(payload)) #La misma que para el login
He probado añadiendo a los headers
'Authorization':'Basic Token' #Donde token es usuario:contraseña como string en base64
'Authorization':'Bearer Token' #Donde token es usuario:contraseña como string en base64
'Authorization':'Token Token' #Donde token es usuario:contraseña como string en base64
En resumen, no soy capaz, y en ninguna query de las anteriores a esa hay una respuesta que contenga algún token que pueda usar para conectarme.
Qué hago? Cómo establezco la conexión?