Python pyautogui - automating your computer

What is Python pyautogui?

PyAutoGUI is a cross-platform GUI automation Python module for human beings. It is used to programmatically control the mouse and keyboard as a human might. Because of this, the Python pyautogui library proves very useful when it comes to automating tasks on your computer.

Installation

Pyautogui has a few dependencies that need to be downloaded before we can use it. Windows Users will already have these dependencies pre-installed, so they are not required to install anything besides the pyautogui library itself.

Run the following commands in your command prompts for your respective Operating System.

Windows

pip install pyautogui

Mac

pip3 install pyobjc-core
pip3 install pyobjc
pip3 install pyautogui

Linux

pip3 install python3-xlib
pip3 install pyautogui

Screen Based Functions

size() Function:

This function gives you the size of the screen by returning the maximum x and y co-ordinates available.

print(pyautogui.size())

Output:

Size(width=1920, height=1080)

position() Function:

This function gives you the co-ordinates of the mouse cursor on the screen. Returns X and Y co-ordinates.

print(pyautogui.position())

Output:

Point (x = 587, y = 486)

onScreen Function()

The onScreen Function tell us whether a specific position exists on screen. Returns true if the position exists, otherwise False.

print(pyautogui.onScreen(400, 500))
print(pyautogui.onScreen(4000, 5000))

Output:

True
False

Mouse Functions

You can move the cursor using the moveTo() function which takes the x and y coordinates as input.

You should understand how the co-ordinate system for the display screen works in Python. The value of x_coordinate increases from left to right on the screen, and the value of y_coordinate increases from top to bottom. Basically, the value of both the x_coordinate and y_coordinate at the top left corner of the screen is 0.

pyautogui.moveTo(x, y, time_in_seconds)

The time_in_seconds is an important parameter to consider. This parameter decides how long the cursor will take to “move to” the location specified. Sometimes, while trying to automate a program or something, you want to be able to see your automated cursor move. If the value is really low, your cursor will be zipping around all over the place. Keeping it at a value like 1-2 will allow you to see it as it moves.

Moves the cursor relative to current position of pointer on screen.

pyautogui.moveRel(x, y, duration = time)

Normally if you pass an x or y value, the cursor moves you to that location. Like passing 100 and 100 will move the cursor to coordinates (100, 100) of the screen. With the moveRel() function however, movement is relative. So passing 100 and 100 as x and y values will move the cursor 100 pixels to the right and 100 pixels downwards.

Click Functions

The following functions deal with automating mouse clicks in a variety of ways. The names of the Functions are pretty self-explanatory.

pyautogui.rightClick(x, y)

pyautogui.doubleClick(x, y)

pyautogui.tripleClick(x, y)

pyautogui.middleClick(x, y)

The middleClick button usually refers to the scroll wheel, which can actually be clicked.

Don’t confuse these two functions for Mouse clicks. The mouseDown() function will keep the mouse “clicked down” until the mouseUp() function is called. May come useful in automating selections (where you click down and drag your mouse across the screen).

pyautogui.mouseDown(x=x, y=y, button='left')

pyautogui.mouseUp(x=x, y=y, button='left')

Dragging the Mouse

Using pyautogui.drag(), we can have the mouse hold itself down and drag itself across the screen, selecting everything in it’s path. It takes several parameters, the main ones being the three shown below.

pyautogui.drag(x= 100, y= 100, duration= 1)

One important thing to note here, is that the x and y values are relative offsets, not specfic coordinates on the screen.

This function is a better alternative to using mouseUp() and mouseDown() for scrolling, as it saves you an extra function call.

Scrolling the Mouse

pyautogui.scroll(amount_to_scroll, x=x_movement, y=y_movement)

This will store a PIL object containing the image in a variable.

Keyboard Automation

Pyautogui also performs many keyboard like commands. Once again, these are very self-explanatory. The Hotkey function first presses shift and then enter, exactly as a regular human might.

pyautogui.hotkey('shift', 'enter')

pyautogui.hotkey('ctrl', '2' )

If you just want single key presses instead, you can use the pyautogui.press() function as follows.

pyautogui.press('a')
pyautogui.press('enter')

To learn more about Keyboard Automation in PyAutoGUI, refer to it’s dedicated tutorial.

Prompt Functions

Below are a list of common prompt functions. Each of them brings up a unique prompt with a different GUI. The alert prompt brings up a window with an exclamation mark for instance.

pyautogui.confirm("Are you ready?")

pyautogui.alert("The program has crashed!")

pyautogui.prompt("Please enter your name: ")

This marks the end of the Python pyautogui Article. Any suggestions or contributions for CodersLegacy are more than welcome. Questions can be asked in the comment section below.

Link to the Python Libraries Section: link