In this Python tutorial, we will explore how to create, pass and return Structs between our C and Python program using the ctypes library.
The Python Ctypes Library can be quite difficult to use, and online tutorials on it are rather scarce, especially those about Structs and Classes. Hence, we will go through several different problems faced by users when dealing with Structs in Python ctypes. If you have any remaining questions, leave them down in the comments section.
Pass Struct from Python to C
First lets take a look at how to pass a “Struct” from Python into our C program using ctypes. There can be many reasons why we do this, such as passing in Custom objects (e.g coordinates) for further computation and processing, taking advantage of C’s superior processing speed.
A “struct” in Python is actually just a Class which we inherit from ctypes.Structure. It needs to be created with a special attribute called _fields_ which ctypes uses to extract information about the attributes inside the “struct”.
The _fields_ attribute contains 2-value tuple pairs. The first value represents the name of the attribute, and the second represents the datatype. Remember, we need to be using ctypes datatypes here, not regular Python datatypes.
class Point(ctypes.Structure):
_fields_ = [("x", ctypes.c_int),
("y", ctypes.c_int)]
We have created a simple “Point” class, with two attributes “x” and “y” which are integers. Our goal is to pass this in to a C function, and print out its value.
Lets write the code for our C library. It is also essential to declare an equivalent struct for “Point” in the C file.
#include <stdio.h>
struct Point {
int x;
int y;
};
void printPoint(struct Point p) {
printf("%d %d\n", p.x, p.y);
}
As you can see, our code is fairly straightforward. We have a single struct declaration for Point, followed by a single function which prints out the contents of a Point struct that it receives as a parameter.
We will now compile this into a shared library by running this command.
gcc -fPIC -shared -o clibrary.so clibrary.cLet’s move back over to our Python file now, where we will access this shared library and its function, passing our Structs back and forth using Ctypes.
import ctypes
import os
class Point(ctypes.Structure):
_fields_ = [("x", ctypes.c_int),
("y", ctypes.c_int)]
path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))
We just used some fancy code there to automatically try and locate the shared library’s file path. You can remove it and just write out the full file path to the shared library if you run into an error.
Now its time to call our function from our shared library.
p1 = Point(10, 20) # Create a Point Object
clibrary.printPoint(p1)
10 20As you can see from the output, our code is working perfectly.
Return Struct from C to Python
Now lets try and do the reverse of what we just did. We will now create a Point object inside the C file instead, and return it back to our Python program.
You might be tempted to define a function as shown below, which returns a regular struct.
struct Point getPoint() {
struct Point temp;
temp.x = 50;
temp.y = 100;
return temp;
}
However, this is actually incorrect and will not work properly if we attempt to call this from our Python program. (Note: If you are curious, we got an integer “50” as the return value, not a struct).
Instead what we need to do is return a “pointer” to a struct instead. If we redefine our function from earlier, we get the following:
struct Point *getPoint() {
struct Point *temp;
temp->x = 50;
temp->y = 10;
return temp;
}
Let’s now run a quick recompile of our C library.
gcc -fPIC -shared -o clibrary.so clibrary.cAnd now back to our Python file.
import ctypes
import os
class Point(ctypes.Structure):
_fields_ = [("x", ctypes.c_int),
("y", ctypes.c_int)]
path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))
clibrary.getPoint.restype = ctypes.POINTER(Point)
print(clibrary.getPoint().contents.x, clibrary.getPoint().contents.y)
50 100As you can see we have removed some of the old code, and added two new lines. The first one is used to explicitly declare that the return type of the getPoint() is a Pointer. The second line is responsible for actually calling this function and printing out the value of the memory location to which the Pointer points.
If you haven’t read my tutorial on ctypes Pointers, then you should definitely check it out! But basically in ctypes, to access the object which a pointer points to, we use the .contents attribute on the pointer.
print(clibrary.getPoint().contents)
If you execute the above line, you will get a Point object in the output. We can now treat it as a regular object and access it by using its attributes “x” and “y” as shown earlier.
We might need to store this pointer somewhere for future use. To do this, first create a pointer of the appropriate, and then store the return value of the function inside it.
clibrary.getPoint.restype = ctypes.POINTER(Point)
p = ctypes.POINTER(Point)
p = clibrary.getPoint()
print(p.contents.x, p.contents.y)
50 100Pretty cool right?
Dealing with advanced Structs in Ctypes
In the example earlier, we were using a fairly simple Struct with just two integer values. Lets explore a slightly more complex example this time where we deal with arrays and pointers inside our Python ctypes Struct.
This time we will be passing an struct which contains an array into our C program. We will also explore an extra concept, where we use nested structures. Basically we will create an array of “Points” where Point is the structure we defined earlier.
import ctypes
import os
class Point(ctypes.Structure):
_fields_ = [("x", ctypes.c_int),
("y", ctypes.c_int)]
class PointArray(ctypes.Structure):
_fields_ = [("points", Point * 3)]
As you can see from our above file, we have defined a new structure called “PointArray” which contains an array called “points” which consists of three “Points”.
Now lets head over to our C file, and define the same struct for PointArray. We will also modify our print() function a bit to print out all the points in a PointArray object.
#include <stdio.h>
struct Point {
int x;
int y;
};
struct PointArray {
struct Point points[3];
};
void printPointArray(struct PointArray pa) {
for (int i = 0; i < 3; i++) {
printf("%d %d\n", pa.points[i].x, pa.points[i].y);
}
}
After recompiling our c file, we will head back to our Python file. We have three new lines to add.
points = (Point(1, 1), Point(2, 3), Point(5, 10))
pa = PointArray(points)
clibrary.printPointArray(pa)
First we will create a points array which is basically just a tuple of three individual Points. We then pass this as a single parameter to our PointArray Class and create a new object “pa”. We then pass this object into our print function from earlier.
Note: We can reduce this to just two lines by passing in the array of Points directly into PointArray(). We simply broke it down a bit for the concept.
Here is the complete code, along with the output.
import ctypes
import os
class Point(ctypes.Structure):
_fields_ = [("x", ctypes.c_int),
("y", ctypes.c_int)]
class PointArray(ctypes.Structure):
_fields_ = [("points", Point * 3)]
path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))
points = (Point(1, 1), Point(2, 3), Point(5, 10))
pa = PointArray(points)
clibrary.printPointArray(pa)
1 1
2 3
5 10Ctypes Structs with Strings (Pointers)
One slightly annoying case to handle is when your Python Ctypes Structs contains pointers. Especially character pointers, which are even harder to handle. If you are not careful with how you allocate memory, then you are screwed. Be careful of dangling pointers, remember how dynamic memory allocation works, make sure that your memory is referenced at all times and you should be good.
Sounds hard? A bit, yeah. Lets go through quickly, and make this as simple as possible.
The first thing we will try, is creating a Student Class with a single attribute called “name”. We will create this in our Python file, send it over to our C file and print it out there.
import ctypes
import os
class Student(ctypes.Structure):
_fields_ = [("name", ctypes.c_char_p)]
path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))
s = Student(b"CodersLegacy")
clibrary.printStudentDetails(s)
Here we create a Student Class with a character pointer. We then initialize it using a bytes string (we can’t use normal strings directly). You can also use a variable instead of a string directlty.
name = "CodersLegacy"
s = Student(bytes(name, 'utf-8'))
clibrary.printStudentDetails(s)
Now lets go over to our C file and actually define the printStudentDetails() function.
#include <stdio.h>
#include <stdlib.h>
struct Student {
char* name;
};
void printStudentDetails(struct Student s) {
printf("%s", s.name);
}
As you can see this was all fairly straight forward. Try compiling the C file yourself, and running this code.
Returning a Struct with a Pointer
Now let’s try the reverse. Creating the Struct in C, and returning it to our Python program using Ctypes.
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
struct Student {
char *name;
};
struct Student *getStudent() {
struct Student *s = malloc(sizeof(struct Student));
s->name = strdup("CodersLegacy");
return s;
}
It is essential that all our memory is dynamically allocated here. If you do not, then these will remain as local variables, and when you return these pointers to Python and the function ends, then the local memory will be destroyed, and your Python program will receive dangling pointers.
We don’t want that, so we declare our memory dynamically which means that it exists until we explicitly free it.
And we need to do this for both the Struct, and the string inside it. (Because both objects are not primary datatypes, and cannot be passed by value)
import ctypes
import os
class Student(ctypes.Structure):
_fields_ = [("name", ctypes.c_char_p)]
path = os.getcwd()
clibrary = ctypes.CDLL(os.path.join(path, 'clibrary.so'))
Here you can see the same setup code from before. Now we need to call our function.
clibrary.getStudent.restype = ctypes.POINTER(Student)
s = clibrary.getStudent()
print(s.contents.name)
b'CodersLegacy'And that’s it! Just return the pointer into a variable, access its contents using contents attribute, and voila!
You will get a bytes object in return though. You can convert it to a string by using the decode() function as shown below.
print(name.decode('utf-8'))
CodersLegacyOne last thing. We need to actually free up the memory that we dynamically allocated. All you have to do is create a function in your C file, which takes your dynamically created objects and calls the free() function on them.
Like this:
void free_mem(struct Student* s) {
free(s->name);
free(s);
}
This marks the end of the Structs with Python Ctypes Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.