Python Semaphore Tutorial (with Examples)

In this Python tutorial we will discuss how to use a Semaphore.

A Semaphore is a special type of variable or datatype that controls access to particular resource. We use Semaphores in multi-thread applications to maintain synchronization between these threads.

If you are familiar with the threading module and Lock Class, then you already know most of the syntax and logic for Semaphores. The acquire() and release() functions are common to both the Lock Class and Semaphore Class.

Why do we need Semaphores?

As long as threads run independently of each other (meaning they have no common resource or data), there is no need for any synchronization. But as soon as threads begin to interact with a common shared resource (such as a variable or file) we need to be aware of potential synchronization problems.

For example, if there is a shared resource that multiple threads are accessing, and one (or more) of these threads wants to modify to shared resource in some manner then a “race condition” might occur. This is a type of synchronization problem that has a chance of appearing in such cases, that can be solved using Semaphores.

Note: It is important to note that multiple threads can share a common resource without the need of Semaphores or Thread Locks as long as they are only reading from it. (not modifying it)

Let’s take a look at a little example where a Race Condition occurs.

In the below example, the shared resource is “x”. There are two threads accessing this value and modifying it by adding/subtracting from it.

from threading import Thread, Semaphore
from time import sleep

x = 0

def add_one():
    global x
    for i in range(1000000):
        x += 1

def subtract_one():
    global x
    for i in range(1000000):
        x -= 1      

thread1 = Thread(target=add_one)
thread2 = Thread(target=subtract_one)

thread1.start()
thread2.start()

thread1.join()
thread2.join()

print(x)

What do you expect the answer to be? It should be zero right? Since we are adding 1000000 and subtracting 1000000. But that is not the case.

Here is the actual output(s). We ran this code 3 times to give you an idea on what to expect.

104642
-53215
-209834

There is no “single” output as each thread contains several 1000000’s of instructions (keep in mind that incrementing a value is actually several instructions, not one). Hence there are a massive number of possible outputs depending on the sequence of events.

Implementing a Semaphore in Python

Forget about the previous code for a bit and let’s focus on how to create a Semaphore in Python.

Using the Semaphore Class from the threading module, we first need to create a “lock”. This lock will serve as a sort of “ticket”. There is only one such ticket, and only one thread can use it at a time. This ticket will allow a thread to enter it’s critical section (in which it will access the shared resource). Once it is done, it will “release” the ticket for the next person to use.

lock = Semaphore()

This lock can be acquired using the acquire() function. Similarly, it can be released using the release() function. By default, a thread will wait indefinitely for a lock if it has been acquired by another thread.

Below is a simple example showing how we can use Semaphores.

from threading import Thread, Semaphore
from time import sleep

lock = Semaphore()

def func(lock):
    print("Attempting to acquire Lock")
    lock.acquire()

    print("Acquired Lock")
    # Perform some action here
    sleep(1)

    lock.release()
    print("Released Lock")

thread = Thread(target=func, args=(lock,))
thread.start()
thread.join()

Here we passed the lock as an argument to the function that our new thread will execute.

There is another choice we have, which is to make the lock global. We have shown you how to do this in the below example.

from threading import Thread, Semaphore
from time import sleep

lock = Semaphore()

def func():
    global lock

    print("Attempting to acquire Lock")
    lock.acquire()

    print("Acquired Lock")
    # Perform some action here
    sleep(1)

    lock.release()
    print("Released Lock")

thread = Thread(target=func)
thread.start()
thread.join()

Both have the same effect, so use whichever you prefer.

There is another way we can use semaphores through the concept “counting semaphores”, also known as “non-binary semaphores”. We will discuss these at a later point in the tutorial.

Solving the Race Condition Problem

With our new found knowledge about Semaphores let us solve the earlier problem!

Here is the updated code. Try it yourself!

from threading import Thread, Semaphore
from time import sleep

lock = Semaphore()
x = 0

def add_one():
    global x, lock
    for i in range(1000000):
        lock.acquire()
        x = x + 1
        lock.release()

def subtract_one():
    global x, lock
    for i in range(1000000):
        lock.acquire()
        x = x - 1      
        lock.release()


thread2 = Thread(target=subtract_one)
thread1 = Thread(target=add_one)

thread1.start()
thread2.start()

thread1.join()
thread2.join()

print(x)

The output of this code will be “0”, no matter how many times you execute it.

Counting Semaphore in Python

To understand Counting Semaphores, let’s use the same “ticket analogy” from earlier.

So remember how normally we only have one ticket? Well, there is a way to actually create more than one. When creating the lock using Semaphore(), just pass in an integer “n”, which will create “n” number of tickets. By default, the value of “n” is one, so Semaphore() and Semaphore(1) are the same thing.

Counting Semaphores are often used in places where we only want a certain number of threads accessing a resource at a given time. Real-life examples include an Elevator (has a maximum people limit) or a Ticket counter (number of counters = number of people who can be served).

Semaphores with only one “ticket” are called binary semaphores, and those with more than one are called non-binary or counting semaphores.

Below we have a playground example where only 3 children are allowed in a playground at a time. There are 10 children in total, and all of them want a turn. We have included print statements in the appropriate places so that the output tells us exactly what happened.

from threading import Thread, Semaphore
from time import sleep
import sys

print = lambda x: sys.stdout.write("%s\n" % x)

playground = Semaphore(3)

def enter_playground(num):
    global playground
    
    print(f"Child {num} is waiting for his turn")
    playground.acquire()

    print(f"Child {num} is playing")
    sleep(3)

    print(f"Child {num} has left the playground")
    playground.release()

children = []

for i in range(10):
    children.append(Thread(target = enter_playground, args = (i,)))
    children[i].start()

for i in range(10):
    children[i].join()

Here is our output. You can see here that the first three children were able to enter the playground immediately. The other 7 children had to wait until the first three left, after which 3 other children came in and so on.

Child 0 is waiting for his turn
Child 0 is playing
Child 1 is waiting for his turn
Child 2 is waiting for his turn
Child 1 is playing
Child 2 is playing
Child 3 is waiting for his turn
Child 4 is waiting for his turn
Child 5 is waiting for his turn
Child 6 is waiting for his turn
Child 7 is waiting for his turn
Child 8 is waiting for his turn
Child 9 is waiting for his turn
Child 2 has left the playground
Child 0 has left the playground
Child 1 has left the playground
Child 3 is playing
Child 4 is playing
Child 5 is playing
Child 4 has left the playground
Child 5 has left the playground
Child 6 is playing
Child 3 has left the playground
Child 7 is playing
Child 8 is playing
Child 7 has left the playground
Child 6 has left the playground
Child 8 has left the playground
Child 9 is playing
Child 9 has left the playground

Do keep in mind that Counting Semaphores are prone to Race Conditions. You can always use multiple Semaphores (a mix of both types) or Thread Locks to fix this problem though. It all depends on your scenario, and may not even be necessary to begin with.

Why do we need Semaphores?

Implementing a Semaphore in Python

Solving the Race Condition Problem

Counting Semaphore in Python

Similar topics: