Pickling a Python Extension type defined as a C struct having PyObject* members

2024/11/14 23:21:37

I am running C++ code via Python and would like to pickle an extension type.

So I have a C++ struct (py_db_manager) containing pointers to a database object and a object manager object (both written in C++) that I wrapped with a python type object (t_db_manager). My problem is that this python type needs to know how to pickle the two pointers in order to send it to some child multicore processes. So I registered the type with the copy_reg module (this is equivalent to writing a reduce() method on the type. However, I'm not too sure what to put in it. Should I build a tuple with the PyObject* or just the integer pointers?. Can anyone help?

typedef struct
{PyObject_HEADPyObject* man_inst_ ;PyObject* db_inst_ ;}py_db_manager;`

Here's the Py_TypeObject

PyTypeObject t_db_manager = {PyObject_HEAD_INIT(0)               /* tp_head */0,                                  /* tp_internal */".py_db_manager",                  /* tp_name */sizeof(py_db_manager)};

And here's the code that would be in the reduce method:

PyObject *pickle_manager(PyObject *module, PyObject *args)
{py_db_manager *cpp_manager =0;PyObject *values = NULL,*tuple = NULL;char text[512];if (!PyArg_ParseTuple(args, "O!", &t_db_manager, &cpp_manager))goto error;sprintf(text,"man_inst_, db_inst_");if ((values = Py_BuildValue("(sii)", text,cpp_manager->man_inst_, cpp_manager->db_inst_)) == NULL)goto error;tuple = Py_BuildValue("(OO)", manager_constructor, values);error:Py_XDECREF(values);return tuple;
}
Answer

Because this will be passed to another process, pickling just integer pointers will not work as you would want it to. Different processes use different memory space therefore they don't see the same things.

So, to answer your question, you should pickle full objects and reconstruct them from the receiving end.

https://en.xdnf.cn/q/71496.html

Related Q&A

Generating random ID from list - jinja

I am trying to generate a random ID from a list of contacts (in Python, with jinja2) to display in an HTML template. So I have a list of contacts, and for the moment I display all of them in a few cell…

Unit testing Flask app running under uwsgi

I’m relatively new to python and am looking for a pythonic way to handle this practice. I’ve inherited a fairly trivial Python 2.7 Flask app that runs under uwsgi that I want to add some unit tests t…

fastest way to find the smallest positive real root of quartic polynomial 4 degree in python

[What I want] is to find the only one smallest positive real root of quartic function ax^4 + bx^3 + cx^2 + dx + e [Existing Method] My equation is for collision prediction, the maximum degree is quarti…

Split strings by 2nd space

Input :"The boy is running on the train"Output expected:["The boy", "boy is", "is running", "running on", "on the", "the train"]Wha…

Searching for a random python program generator

Im searching for a program that can generate random but valid python programs, similar to theRandom C program generator. I was trying to do this myself giving random input to the python tokenize.untoke…

Python tk framework

I have python code that generates the following error:objc[36554]: Class TKApplication is implemented in both /Library/Frameworks/Tk.framework/Versions/8.5/Tk and /System/Library/Frameworks/Tk.framewor…

SQLAlchemy relationship on many-to-many association table

I am trying to build a relationship to another many-to-many relationship, the code looks like this: from sqlalchemy import Column, Integer, ForeignKey, Table, ForeignKeyConstraint, create_engine from …

Python: interpolating in a triangular mesh

Is there any decent Pythonic way to interpolate in a triangular mesh, or would I need to implement that myself? That is to say, given a (X,Y) point well call P, and a mesh (vertices at (X,Y) with val…

Customizing pytest junitxml failure reports

I am trying to introspect test failures and include additional data into the junit xml test report. Specifically, this is a suite of functional tests on an external product, and I want to include the p…

python nltk keyword extraction from sentence

"First thing we do, lets kill all the lawyers." - William ShakespeareGiven the quote above, I would like to pull out "kill" and "lawyers" as the two prominent keywords to …