Using Arrays with Python Ctypes

In this Python tutorial we will discuss how to pass Arrays back and forth between our C and Python programs using the ctypes library.

Arrays in Python, and Arrays in C/C++ are created and handled in very different ways. Luckily, Ctypes exists as a medium between C and Python, thanks to which creating C-compatible arrays is possible in Python. Lets explore how we can do this.

Creating Arrays with Ctypes

Lets start off by trying to do something rather simple. Unfortunately when dealing when ctypes, even the simplest thing can become rather complex. (It’s worth it for the performance though).

We will be creating an array of integers in our Python file, and a function which sums the elements of an array and returns the result in our C program. We will pass this array to our C file, get the result and print it out in our Python file.

Lets begin!

First we need to create our C function.

    int sum(int *arr, int n) {
        int sum = 0;
        for (int i = 0; i < n; i++) {
            sum += arr[i];
        }
        return sum;
    }

This is a basic function that takes two parameters. A pointer to an array, and the size of the array. We then iterate through the array, calculate the sum and then return it.

Lets give our C program a quick compile now by running the following command in our command prompt:

gcc -fPIC -shared -o clibrary.so cpplibrary.c

Likewise, if we had a .cpp file we would instead do the following:

g++ -fPIC -shared -o cpplibrary.so cpplibrary.cpp

Note: These names are completely arbitrary. Only the extensions matter here.

Now that our shared library (the .so file) is generated, we can move back to our Python file. We used some fancy code using the os module to automatically generate the full file path for our shared file (under the assumption that both the python file and shared library are in the same directory). You can change this behavior if you run into any issues (just write the raw file path).

import ctypes
import os
import time

path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))

Now we need to create our Python array.

array = (ctypes.c_int * 10)()

In ctypes, the way to do this is to multiply a normal datatype (like c_int) by the number of elements you want in the array. The empty () brackets count as a constructor of sorts. Don’t forget to include it.

Next we will fill in some values into our array (from 1 to 10). This way we have basically converted our C-array pointer to a Python-list.

for i in range(10):
    array[i] = i

Finally, we will pass this array and its size into our sum() function from our clibrary.

x = clibrary.sum(array, len(array))
print(clibrary.sum(array, 10))

As you can see, we have the correct output here! 45 is the sum of numbers from 1 to 10.

Converting an existing list to Array

Here is a slight bonus trick that you can use to quickly convert an existing Python list into a Ctypes Array.

values = [5, 7, 8, 3, 6, 8, 9, 6, 3, 2]
array = (ctypes.c_int * 10)(*values)

x = clibrary.sum(array, len(array))
print(clibrary.sum(array, 10))

Note: The * asterisk next to values is being used to “unpack” the list into individual values.

Returning Arrays from C into Python

First we will create a function in our C program which returns an array of numbers.

#include <stdio.h>
#include <stdlib.h> 

int* getArray() {
    int *array = (int*) malloc(10 * sizeof(int));
    for (int i = 0; i <= 10; i++) {
        array[i] = i;
    }
    return array;
}

(Remember to recompile your library before proceeding)

Back to our Python file now. We will need to first explicitly define the return type of the getArray() function as a “integer pointer” using the restype attribute (part of the function signature in ctypes). (We usually need to do this with functions that return pointers in C)

import ctypes
import os

path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'lib.so'))

clibrary.getArray.restype = ctypes.POINTER(ctypes.c_int)
x = ctypes.POINTER(ctypes.c_int)
x = clibrary.getArray()

Next we will create a ctypes pointer which points to an integer, called “x”. We will store the return value from our getArray() function in here.

After we call getArray() and store the result in x, we now have a pointer to an array of numbers. But what now?

Well we can access it using regular Python list indexing. The below code shows us how we can iterate over it and print out its values.

for i in range(10):
    print(x[i])

The only slight problem here, is that we cannot determine from the pointer what the size of the array is. One way of overcoming this would be to wrap the array in a struct with an attribute for size, and then return the struct instead.

A similar concept that you might be interested in, is the handling of structs in Ctypes. You can create custom Classes in Python, which you can then pass into your C program with the help of Ctypes. Interested in learning how? Follow the link for more!

This marks the end of the Arrays with Python Ctypes Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.