Nuitka is a Python library which can convert .py files to standalone executables. However, it often take alot of time to compile these EXE’s, especially when dealing with large libraries such as numpy, matplotlib, SciPy, etc. In this tutorial we will discuss how to speed up the compilation time for Nuitka when creating EXE’s.
Note: Some of the performance optimizations and techniques we are going to discuss are usually enabled by default, or are installed during the Nuitka setup. There is a chance that they were not enabled or setup for you however, so we will go through all such optimizations and how to manually enable them if needed.
Cache
A “cache” refers to as high-speed storage location which we use to store data, for the sake of improving performance. The data stored in a cache is data that we expect to need frequently. Instead of re-computing or re-fetching the data that we need, we first check to see if its in the cache.
In the context of Nuitka, the cache is used to speed-up compilation by re-using previously compiled build data (instead of regenerating it). Usually when you do a recompile after making changes, usually about 80-90% of the code remains the same, hence only 10-20% needs to be recomputed.
This can drastically speed up your compilation time when recompiling (the initial compile will still take the same amount of time).
Note: For the MSVC compilers and ClangCL setups, using the clcache is automatic and included in Nuitka. Try compiling some Python code, and observe the log carefully. If Nuitka has detected the cache, there will be a log in there about how you got some ‘cache hits’. (Cache hits are when the cache was used to speed things up)
For people on Mingw, Nuitka will automatically ask for and install the cache for you when you make your first compilation.
If you want (or need) to install the cache manually, here are the steps.
First head over to the website for ccache (the one that Nuitka recommends). Next download the appropriate version (Linux or Windows). Download this file somewhere, and note down the filepath to the folder as shown below.
C:\Users\CodersLegacy\Desktop\ccache-4.7.4-windows-x86_64\Next, add this folder to PATH. Nuitka will automatically (atleast on Windows) check the PATH for ccache. On Windows you can add this folder to PATH through the “Environment Variables” option.
And that’s all. Nuitka will begin using it automatically when compiling your Python EXE’s.
Anti-bloat Plugin
Anti-bloat is a setting in Nuitka which is typically enabled by default (on newer versions atleast). However, you can also explicitly activate it, just incase you are using an older version which may not have on by default or some other configuration problem.
Anti-bloat is a smarter way of checking for imports, and removing the ones which are not needed by your program. Less libraries to include and compile, means both a faster compilation and a smaller EXE size. It’s a win-win!
All you need to do is add the additional option when compiling:
--enable-plugin=anti-bloatIf you already have it enabled, Nuitka will show you a warning during the compilation process.
Ordered-Set
You will likely receive a warning by Nuitka when you begin compiling about installing the “Ordered-set” library for faster compilation.
Here is the warning:
Nuitka:WARNING: Using very slow fallback for ordered sets, please install 'ordered-set' PyPI package for best Python compile time
Nuitka:WARNING: performance.How much of an improvement this gives can vary, ranging from nothing at all to a moderate improvement. This depends on your code, the target system, and how often the set datatype is being used.
There is no harm in doing this though, and Nuitka recommends it, so go ahead!
pip install ordered-setThis marks the end of the “Speed up Nuitka Compilation time” Tutorial. Any suggestions or contributions for CodersLegacy are more than welcome. Questions regarding the tutorial content can be asked in the comments section below.
