Python Selenium web automation



Description

The Python Selenium library can be thought as a web automation library. If you’ve heard of pyautogui, a library that can automate the movements of a mouse and keyboard, Selenium is the web equivalent of this. Selenium is able to take the place of the user while accessing web pages.

Selenium has the ability to seek out elements in a web page and interact with them. Such as the ability to click on buttons or links in a web page. Selenium is a more advanced way of accessing the web page as compared to other web libraries like BeautifulSoup and requests.

Installation

Install selenium through the command prompt using the following statement.

pip install selenium

Starting out

In order to actually begin executing any selenium commands, we first need to create the browser object. Using this object, we can do almost anything to a web page. Selenium revolves around this browser object.

This browser object is created differently depending on the browser you want to use. For Firefox, you need the Firefox() function, where as for Chrome, you need the Chrome() function. To do this, we need to import webdriver first though.

Don’t worry, it’s part of selenium, so you don’t have to download it separately.

from selenium import webdriver

Now to create the browser object. If you wish to use Chrome, you’ll have to download the chromedriver.exe and link your Chrome() function to it, as shown below. You can download it using this link to Chrome’s website.

browser_obj = webdriver.Chrome(executable_path=filename)

Make sure you download the correct webdriver, as there the Chromedriver for each version is different. You can check your Chrome version by opening the Menu (the three dots in the upper right corner) then navigate to help and click on the “About Google Chrome” option.

Firefox can be used without any such hindrance however. Keep in mind though, you actually need to have the FireFox browser installed and setup to use it.

Regardless of which one you choose, the syntax remains the same, except for that one line above. Each browser however, has it’s separate quirks, which you’ll find out if you dig deep enough into selenium.

Inspecting the Web page

Before you actually begin writing any code, the first thing to do is scope out your target web page. For instance, you’re looking to automate your Facebook login. In order to automate it, you need to locate and identify the email and login fields.

You cannot achieve this without first inspecting their HTML code. It’s actually very simple. Right click on the email field, and click the Inspect option. This will open up the respective HTML text in front you.

All you have to do is identify some attribute or text that you can use to locate it. See below to understand.

Example

We’ll be using this site, CodersLegacy as a place to test selenium. Our goal is to find and click the button shown below using selenium. First, let’s find and connect to this page using it’s URL.

import selenium
from selenium import webdriver

url = "https://coderslegacy.com/libraries-in-python/python-selenium/"
driver = webdriver.Chrome('C:\\Users\Default\chromedriver.exe')
driver.get(url)

If this page opened up in front of you in a new browser window, you were successful. Now that we’ve established a connection, we can begin automating tasks.

X = driver.find_element_by_link_text('Click me')
X.click()

Adding these two lines is our first automation example. The driver.find_element function looks for an element that has the text “Click me”. There are numerous ways we might do this, such as locating the class. However, there may be other buttons sharing the same class, hence we need something unique to this button.

Example 2

A common use of selenium is automating logins. Let’s try automating our Facebook login.

After inspecting the Facebook login page, I see we are in need of three elements. First, the email field, which has the name attribute “email“. Secondly, the password field, which has the name attribute, “pass“. Finally, the login button itself, which we can access using it’s id, “loginbutton“.

Here’s our code.

import selenium
from selenium import webdriver

url = "https://facebook.com"
driver = webdriver.Chrome('C:\\Users\Shado\source\chromedriver.exe')
driver.get(url)

email = driver.find_element_by_name("email")
email.send_keys("example@hotmail.com")
passw = driver.find_element_by_name("pass")
passw.send_keys("psswrd")
X = driver.find_element_by_id('loginbutton')
X.click()

It’s up to you now to keep practicing, and trying to automate new things. Remember to scope out your target page first, identifying each element separately. Keep an open mind and be willing to learn new things.

Below is a list of all Methods for the Web driver object and web element.

List of Web Driver Methods

Method NameDescription
browser.find_element_by_class_name(name)
browser.find_elements_by_class_name(name)
returns element(s) with the class name, name.
browser.find_element_by_css_selector(selector)
browser.find_elements_by_css_selector(selector)
returns element(s) that match the CSS selector, selector.
browser.find_element_by_id(id)
browser.find_elements_by_id(id)
returns element(s) with the id, id.
browser.find_element_by_link_text(text)
browser.find_elements_by_link_text(text)
returns <a> element(s) that have the exact text as text.
browser.find_element_by_partial_link_text(text)
browser.find_elements_by_partial_link_text(text)
returns <a> element(s) that contain the text, text. It’s basically a partial match.
browser.find_element_by_name(name)
browser.find_elements_by_name(name)
returns element(s) with the name value of name.
browser.find_element_by_tag_name(name)
browser.find_elements_by_tag_name(name)
returns element(s) with the tag_names matching name.

List of Web element Methods

Attribute or MethodDescription
tag_nameReturns the tag name, such as <p> for a paragraph element.
get_attribute(name)Retrieves the value of the attribute called name.
textReturns the text within the element, like the text in a paragraph. <p> text </p>.
clear()For text field or text area elements, clears the text typed into it.
is_displayed()Returns True is element is visible on screen, other wise returns False.
is_enabled()Used for Input element. Returns True is the element is Enabled, else returns False.
is_selected()For checkbox or radio button elements, returns True if the element is selected; otherwise returns False.
locationReturns a dictionary with keys ‘x’ and ‘y’ for the position of the element on the web page.

This marks the end of the Python Selenium – web automation Article. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the article can be asked in the comments section below.

Here’s a link back to the main Python Libraries section where you can learn about other great libraries.

Leave a Reply

Your email address will not be published. Required fields are marked *