Run Multiple Browsers in Parallel Selenium Python

In today’s fast-paced world of software development and testing, efficiency and speed are of paramount importance. When working with Selenium for web automation and testing, you might often find yourself needing to run tests across multiple browsers concurrently to reduce execution time and increase test coverage.

Selenium natively has no way of running multiple browsers in parallel. Fortunately for us, Python has various libraries which can be used for parallelism.

Prerequisites

Before we dive into the implementation, ensure you have the following prerequisites in place:

  1. Selenium: Install the Selenium WebDriver library using pip:
pip install selenium
  1. Web Drivers: Download the web drivers for the browsers you intend to use (e.g., Chrome, Firefox, Edge). Make sure they are compatible with your browser versions. If you are not familiar with this step, check out our tutorial on how to setup a web driver.
  2. multiprocessing: Python’s multiprocessing library comes bundled with Python, so you don’t need to install it separately. You will however, have to import it in your code.

How to run Multiple Browsers in Parallel Selenium Python

First, we import the necessary Python modules.

from selenium import webdriver
from multiprocessing import Pool

webdriver is imported from Selenium, which provides browser automation capabilities, and Pool is imported from the multiprocessing library, which allows us to create a pool of worker processes which is needed for the parallel browsers to work.

Next, we define the create_browser function. This function takes one argument, num, an integer which will help us differentiate between different browser instances (if needed). 

def create_browser(num):
    browser = webdriver.Chrome()
    browser.get('https://mail.google.com/')
    return browser

Within the create_browser function, we create a Chrome WebDriver instance (browser) using the previously configured options. We also navigate this browser to the Gmail login page (https://mail.google.com/).

Next, create a pool of worker processes using multiprocessing.Pool. The processes parameter specifies the maximum number of processes that can run concurrently. 

pool = Pool(processes=10)

Here, we set it to 10, meaning that up to 10 browsers will run simultaneously.

for i in range(0, 5):
    async_result = pool.apply_async(create_browser, args=(i))

Within a loop, we use pool.apply_async to asynchronously create five browser instances in parallel. For each iteration, a new process is started to execute the create_browser function with a unique num argument. The args=(i) part passes the i value from the loop to the create_browser function as its num argument.

Finally, after creating all the browser instances, we call pool.close() to prevent any new processes from being added to the pool. 

pool.close()
pool.join()

We also use pool.join() to wait for all the processes in the pool to finish their execution. This ensures that we don’t proceed further in the script until all browser instances have completed their tasks.

Here is the complete code:

from selenium import webdriver
from multiprocessing import Pool

def create_browser(num):
    browser = webdriver.Chrome()
    browser.get('https://mail.google.com/')
    return browser

pool = Pool(processes=10)  # Maximum number of browsers opened simultaneously

for i in range(0, 5):  # Create five browsers in parallel
    async_result = pool.apply_async(create_browser, args=(i))

pool.close()
pool.join()

You can further modify this code to open different types of browsers, such as FireFox, Safari, or Microsoft Edge.

This marks the end of the “Run Multiple Browsers in Parallel Selenium Python” Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.

Leave a Comment