Enable javascript and cookies to continue python requests

by

To enable JavaScript and cookies in Python requests, you can use the ‘requests’ library along with the ‘requests.cookies’ module. Here’s how you can do it: 

    
import requests

url = "https://example.com"
    
# Enable JavaScript and cookies
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
}

response = requests.get(url, headers=headers)

# Extract cookies
cookies = response.cookies

# Use cookies in subsequent requests
headers_with_cookies = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
    "Accept-Language": "en-US,en;q=0.9",
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
    "Cookie": "; ".join([f"{cookie.name}={cookie.value}" for cookie in cookies])
}

# Make a request with enabled cookies
response_with_cookies = requests.get(url, headers=headers_with_cookies)

# Print the response content
print(response_with_cookies.content)

In the above example, we first make a GET request to the desired URL with additional headers that mimic a typical browser request. This is important to effectively enable JavaScript and cookies. We extract the cookies from the response using the ‘response.cookies’ attribute.

In the second GET request, we include the extracted cookies in the ‘Cookie’ header of the request to ensure that the subsequent request is made with the enabled cookies. The response content is then printed, which will now contain the expected data that requires JavaScript and cookies to be enabled.

Read more

Leave a comment