Chapter 1: Introduction to Cookie Management

In our previous discussion, we covered executing JavaScript code through Selenium. This time, we’ll delve into the management of cookies within this framework.

What exactly are cookies? Cookies are small data packets formatted as key-value pairs, often accompanied by additional attributes. These are stored on the client-side during a user's interaction with a website. Typically, cookies hold information regarding users, their preferences, and past actions. They play a crucial role in identifying users and personalizing their experience.

For instance, consider your experience on YouTube. If you watch several videos, your viewing preferences are saved, allowing the platform to recommend similar content during your next visit, even if you haven't logged in.

The Selenium WebDriver API includes several methods for cookie management:

• get_cookies
• get_cookie
• add_cookie
• delete_cookie
• delete_all_cookies

The get_cookies method retrieves a list of cookie data as dictionary objects for the ongoing session. This operates by performing a GET request to the session/:sessionId/cookie endpoint.

Here’s an example that displays the cookies for www.google.com:

# output

{'name': '1P_JAR', 'value': '2021-02-13-13', 'path': '/', 'domain': '.google.com', 'secure': True, 'httpOnly': False, 'expiry': 1615813217, 'sameSite': 'None'}

{'name': 'NID', 'value': '209=bQMfEPsjBsSxSTq-jnAv5tTk5pVQkksxSf3xf1YXdCGVnc4xDOK3vYgKSc9OLabwzTnKuTri6cJzAgOPVWKrC9KTfO03XXWPV-jKzE-K8pYS2R0pN5WQ5P0psxMBCo23BFuopiD1cgJvBMbz6fTWj66SXfAcce1tv3vUEonACYc', 'path': '/', 'domain': '.google.com', 'secure': True, 'httpOnly': True, 'expiry': 1629032417, 'sameSite': 'None'}

The NID cookie value can be extracted using the get_cookie method. In the previous example, we identified a cookie named NID. The following code snippet retrieves this cookie’s data:

# output

NID cookie value: 209=bQMfEPsjBsSxSTq-jnAv5tTk5pVQkksxSf3xf1YXdCGVnc4xDOK3vYgKSc9OLabwzTnKuTri6cJzAgOPVWKrC9KTfO03XXWPV-jKzE-K8pYS2R0pN5WQ5P0psxMBCo23BFuopiD1cgJvBMbz6fTWj66SXfAcce1tv3vUEonACYc

To add a new cookie to the current session, use the add_cookie method. It requires a POST request to the session/:sessionId/cookie endpoint. Ensure that you are on the correct domain for the cookie to be valid, or an InvalidCookieDomainException will be raised.

Here’s how to add a cookie:

# output

customCookie value: test

This snippet creates a dictionary representing the cookie, adds it to the active session using the add_cookie method, and then retrieves it by name using get_cookie.

To remove cookies, the delete_cookie method can be used to delete a specific cookie by name. If you wish to clear all cookies in the current session, delete_all_cookies is the method to call. The example below illustrates this process:

# output

Number of cookies before adding cookies: 2

Number of cookies after adding cookies: 5

Number of cookies after deleting a cookie: 4

Number of cookies after deleting all cookies: 0

Storing and Restoring Cookies with Pickle

The pickle module allows for binary serialization of Python objects, enabling you to save cookies and restore them in future sessions. The following example demonstrates adding two custom cookies to the session, serializing them, and saving them to a file named stored_cookies.pkl. After closing the session, a new one is opened, and the previously saved cookies are restored.

# output

Name: NID, Value: 209=rgPaCuCm3PgwaDHwxvOklaG-uo_im1dPTmX7nyWBTnjdwwinjzSMWYSpxPQ67LPIW08NNxdxcJKifTgwyitqK93t2YyoOscnTxnKzfQcEmZbivDwlZCPUQXNmjtX6dnahh21xqM6MszypI2aI-sI1pefjcfvp0lIkaIQftm2hgE

Name: 1P_JAR, Value: 2021-02-13-13

Name: cookie1, Value: test1

Name: cookie2, Value: test2

Key Takeaways

• Cookies are small key-value pairs stored in the web browser, typically holding user information, preferences, and past activities.
• Selenium provides various methods for cookie management, including get_cookie, get_cookies, add_cookie, delete_cookie, and delete_all_cookies.
• Ensure you are on the correct domain when adding cookies to avoid errors.
• Cookies in Selenium are represented as dictionary objects, allowing for easy serialization and restoration between sessions.

In our next installment, we will explore how to manage popups using Selenium.

Thank you for your attention.

Chapter 2: Practical Video Tutorials

To further enhance your understanding, refer to the following video resources:

The first video, Reuse cookies in Selenium with Python, provides a detailed overview of cookie management techniques in Selenium.

The second video, Selenium Webdriver - Cookies with Python WebDriver, dives into practical applications and examples of using cookies in Selenium.