Can I statically link Cython modules into an executable which embeds python?

2024/10/7 2:18:39

I currently have an executable compiled from C++ that embeds python. The embedded executable runs a python script which load several Cython modules. Both the Cython modules and the executable are linked against a shared library.

I want to move the shared library into the executable by statically linking the shared library against the executable.

Can I statically link the Cython modules into the executable which embeds python? What is the best way to handle this situation?

Answer

Yes it's possible, but if you have an hand on the python interpreter. What i'm going to describe have been done for python on IOS platform. You need to check more how to let python known about your module if you don't want to touch on the original python interpreter (Replace TEST everywhere with your own tag/libname)

One possible way to do it is:

  • Compile your own python with a dynload patch that prefer to not dlopen() your module, but use directly dlsym() to check if the module is already in memory.

  • Create an libTEST.a, including all the .o generated during the build process (not the .so). You can found it usually in the build/temp.*, and do something like this:

    ar rc libTEST.a build/temp.*/*.o
    ranlib libTEST.a
    
  • When compiling the main executable, you need to add a dependency to that new libTEST.a by appending in the compilation command line:

    -lTEST -L.

The result will give you an executable with all the symbol from your cython modules, and python will be able to search them in memory.

(As an example, I'm using an enhanced wrapper that redirect ld during compilation to not produce .so, and create a .a at the end. On the kivy-ios project, you can grab liblink that is used to produce .o, and biglink that is used to grab all the .o in directories and produce .a. You can see how it's used in build_kivy.sh)

https://en.xdnf.cn/q/70290.html

Related Q&A

How to Store Graphs?

Lets say i have a class Graph defined.graph iowa = {.....nodes:{edges.......}}and similar graph clusters.once the script is running its all in the object. But how do you store the Graph info (including…

regex for only numbers in string?

I cant find the regex for strings containing only whitespaces or integers. The string is an input from user on keyboard. It can contain everything but \n (but it doesnt matter I guess), but we can focu…

How to build a chi-square distribution table

I would like to generate a chi-square distribution table in python as a function of the probability level and degree of freedom.How to calculate the probability, given a known chi-value and degree of f…

Reset all weights of Keras model

I would like to be able to reset the weights of my entire Keras model so that I do not have to compile it again. Compiling the model is currently the main bottleneck of my code. Here is an example of w…

How to fix NaN or infinity issue for sparse matrix in python?

Im totally new to python. Ive used some code found online and I tried to work on it. So Im creating a text-document-matrix and I want to add some extra features before training a logistic regression mo…

Mutable default argument for a Python namedtuple

I came across a neat way of having namedtuples use default arguments from here.from collections import namedtuple Node = namedtuple(Node, val left right) Node.__new__.__defaults__ = (None, None, None) …

Windows notification with button using python

I need to make a program that alerts me with a windows notification, and I found out that this can be simply done with the following code. I dont care what library I use from win10toast import ToastNo…

numpy IndexError: too many indices for array when indexing matrix with another

I have a matrix a which I create like this:>>> a = np.matrix("1 2 3; 4 5 6; 7 8 9; 10 11 12")I have a matrix labels which I create like this:>>> labels = np.matrix("1;0…

how to use enum in swig with python?

I have a enum declaration as follows:typedef enum mail_ {Out = 0,Int = 1,Spam = 2 } mail;Function:mail status; int fill_mail_data(int i, &status);In the function above, status gets filled up and wi…

What does except Exception as e mean in python? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.Want to improve this question? Update the question so it focuses on one problem only by editing this post.Closed 4…