The entire foundation of PyAutoGUI, and how it automates tasks, relies on Mouse and Click Automation. Without the mouse, Keyboard automation cannot be done effectively either, which makes it even more important.
PyAutoGUI offers us a variety of different mouse-related functions and functionality. From simulating the movement of the mouse, to automating it’s clicks, to emulating click-and-drag functionality, PyAutoGUI has it all.
PyAutoGUI Mouse Automation
Let us now discuss the various ways in which we can automate the mouse in PyAutoGUI.
Position of Mouse on Screen
In order to move our mouse around effectively, we need to know where it is currently located on screen. For this we have the position() function, which returns a Point object, with x and y values, representing the current co-ordinates the mouse is placed at.
print(pyautogui.position())Output:
Point(x = 587, y = 486)If you want to find out the size of the Screen, you can simply use the pyautogui.size() function, which returns a tuple with the width and height.
Moving the Mouse
You can move the mouse cursor around the screen, using the moveTo() function. This function takes the x and y values of the coordinate to which you want to move it to.
You should understand how the co-ordinate system for the display screen works in Python. The value of x_coordinate increases from left to right on the screen, and the value of y_coordinate increases from top to bottom. To summarise, the value of both the x_coordinate and y_coordinate at the top left corner of the screen is 0, and it’s it max in the bottom-right corner.
pyautogui.moveTo(x, y, time_in_seconds)The time_in_seconds is an important parameter to consider. This parameter decides the time taken for the cursor to “move to” the location specified. Sometimes when automating a program or task, you want to be able to see your automated cursor move. If the value is really low or 0, your cursor will be zipping around all over the place.
Keeping it at a value like 0.5 - 1 seconds will allow you to see it as it moves. This may also come in handy when trying to emulate human movement. Any basic anti-bot filter will notice the abnormal speed of the mouse, and likely block you.
Relative Mouse Movement
Sometimes we may not be concerned about where we want to move the mouse with respect to the screen. Maybe we just want to move the mouse by 100 pixels to the left, or to the right. This is called relative movement, where we move the mouse cursor relative to current position of pointer on screen.
pyautogui.moveRel(x, y, duration = time)Normally if you pass an x or y value, the cursor moves you to that location. Like passing 100 and 100 will move the cursor to coordinates (100, 100) of the screen. With the moveRel() function however, movement is relative. So passing 100 and 100 as x and y values will move the cursor 100 pixels to the right and 100 pixels downwards.
Automating Clicks on the Mouse
The following functions all deal with automating various types of mouse clicks . The names of the Functions are pretty self-explanatory.
pyautogui.rightClick() # Clicks the Right Mouse Buttonpyautogui.leftClick() # Clicks the Left Mouse Buttonpyautogui.doubleClick() # Double Clicks the Left Mouse Buttonpyautogui.tripleClick() # Triple Clicks the Left Mouse Buttonpyautogui.middleClick() # Clicks the Middle Mouse ButtonAlternatively, you can choose to pass in x and y values to these functions. This will cause the Mouse to first go to that location, then click on it.
pyautogui.leftClick(x, y) You can also you use the duration parameter here.
Another option you can take advantage of is “interval”. Especially with the double click functions, you will want maybe a 0.5 - 1 second delay between the clicks. Otherwise the double-click will occur instantly.
pyautogui.doubleClick(interval = 1, button = "right")By using the changing the button parameter, we can change it from double clicking the left button twice, to the right button.
Don’t confuse these two functions for Mouse clicks. The mouseDown() function will keep the mouse “clicked down” until the mouseUp() function is called. May come useful in automating selections (where you click down and drag your mouse across the screen).
pyautogui.mouseDown(x=x, y=y, button='left')pyautogui.mouseUp(x=x, y=y, button='left') Dragging the Mouse across the Screen
Using the drag() function, we can have the mouse hold itself down and drag itself across the screen, selecting everything in it’s path. It takes several parameters, the main ones being the three shown below.
pyautogui.drag(x= 100, y= 100, duration= 1)One important thing to note here, is that the x and y values here do not refer to a coordinate. They are offsets, meaning that the drag will occur 100 pixels to the right, and 100 pixels downwards.
What’s next?
Once you have learnt the PyAutoGUI basics, you should enhance your knowledge and skill at automation by learning image detection in PyAutoGUI. This allows you to automatically locate certain patterns/images on the screen, to which you can navigate.
This marks the end of the Python PyAutoGUI Mouse and Click Automation Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.
