How to set up a proxy using selenium in python

What is Proxy?

A proxy acts as a middleman between user requests and server responses. The main purposes of proxies are to protect user privacy and encapsulate data between various interactive systems.

Proxies are a means to get around websites that frequently ban IPs that generate too many requests. Proxy is especially helpful when clients' access to websites needs to be classified as either authorized or prohibited based on the content of those websites. By acting as a firewall between clients and web servers, a proxy can also add an extra layer of security.

Why Proxy Servers

Proxy servers help test a website whether it has a proper firewall and ensure no loopholes that may lead to data leakage, thereby ensuring user privacy.

The best usage for proxy servers is doing localization tests. Let's imagine a tester who wants to access a Banking webpage to verify that consumers from a particular country see the correct language and currency options.

It is automated testing activity. We use this proxy server mostly when a website is being checked from various locations.

What is Selenium?

The most popular tool for executing automated browser tests is Selenium. In short, Selenium allows developers to observe browser and website behaviour without launching and running a full browser instance.

Jason Huggins developed Selenium in the year 2004. Selenium is used in the process of automation of different websites and platforms.

By using Selenium, we can run a proxy in python language. And we can crosscheck various websites for any loopholes using this tool.

Selenium in python

We have flexibility in python for selenium usage. We can directly import the selenium package, which is available in python and use the methods in the selenium package.

For example, if we have to proxy a website using Selenium in python, we can import the web driver module from Selenium.

Syntax: From selenium import webdriver, it loads the web driver module from the selenium package into the python program.

How to set up a Proxy Server using Selenium?

There are a few steps to be followed for setting up a proxy server in Selenium:

Step 1: Import the webdriver from the selenium package in python.

Step 2: Defining the address of the proxy server

Step 3: Setting up Chrome option class and its object

Step 4: Communicating with the chrome option.

Step 5: Group the options to the object of chrome options.

A username and password are unnecessary on many free proxy servers, which implies that they lack authentication. A proxy for the unauthenticated server can be easily made using the abovementioned steps.

Setting an unauthenticated Proxy

Code for setting a proxy using Selenium

from selenium import webdriver


#Defining the proxy server id (IP: PORT)
site = "128.21.0.0:8080"   #Address of the website


#Seting up the object of chrome option class
chrome_options = WebDriver.ChromeOptions()


#Adding parameters to the object of chrome options
chrome_options.add_argument('--proxy-server=%s' % site)


#Comunicating with chrome 
driver = webdriver.Chrome(chrome_options= chrome_options)


#Gathering the information
driver.get("https://www.javatpoint.com/index.htm")

Create a method that accepts the proxy IP address as an argument so that you can test a website from various places by making this code reusable across several tests.

We can create such a function which sets the location bypassing the proxy address of the site in the form of arguments.

def check_location(self):
self.chrome.get(self.url)
search_ele = self.driver.find_element_by_id('city')
#Using assertion method for checking location
self.assertEqual('India', search_ele.text)

Authenticated Proxy Servers

There are now two ways to manage authenticated proxies. The best option will rely on the testers' needs. It depends on various elements, including the Selenium version employed and the browser tested. Automated tests that employ authenticated proxy servers may be difficult to use as Selenium lacks a built-in mechanism for passing proxy server credentials.

We cannot access authenticated proxy servers using Selenium as the selenium package in python does not support the passing of proxy along with proxy server credentials. So, it is impossible to set up an authenticated proxy server using Selenium in python.

However, we can integrate authenticated proxy servers with Selenium by using PhantomJS as a headless browser instead of using the Chrome WebDriver.

You may also add a browser extension that handles Selenium's authentication. Although this method is more difficult, it may be used with the most recent Selenium version, which may be necessary for some development teams.