C++ class not recognized by Python 3 as a module via Boost.Python Embedding

2024/9/28 1:27:04

The following example from Boost.Python v1.56 shows how to embed the Python 3.4.2 interpreter into your own application. Unfortunately that example does not work out of the box on my configuration with MSVC2013 under Windows 8.1. And I have not found 1 working complete example about embedding, at least none that is younger than 10 years or so.

I receive the following error running it: ImportError: 'embedded_hello' is not a built-in module

The code is here: http://pastebin.com/shTtdxT8

Any hints what I can do to let this run? And in general how to expose a c++ class in Python and vice versa?

Answer

The code is compiling with a Python 2 header configuration. When compiling with a Python 3 header configuration, the boost/python/module_init.hpp would have declared the embedded_hello module's initialization function as PyInit_embedded_hello rather than initembedded_hello. I highly recommend verifying the proper header configuration, and performing a clean build of Boost.Python, as Boost.Python and modules built with the library need to use the same header configuration.

Additionally, when adding modules to the built-in table, the PyImport_AppendInittab() calls need to occur before Py_Initialize(). The PyImport_AppendInittab() documentation explicitly states:

Add a single module to the existing table of built-in modules. ... This should be called before Py_Initialize().

Boost.Python uses the BOOST_PYTHON_MODULE macro to define a Python module. Within the body of the module, the current scope is the module itself. Thus, when C++ types are exposed via type wrappers, such as when a C++ classes is exposed to Python via boost::python::class_, the resulting Python class will be within the module defined by BOOST_PYTHON_MODULE.

On the other hand, user-defined types declared in Python are first-class objects. From a C++ perspective, they can be treated as though they are a factory function. Hence, to use a Python defined class in C++, one needs to get a handle to the class object, then instantiate an instance of a class by calling the class object.


Here is a complete minimal example demonstrating embedding a Python 3 interpreter that:

  • Imports a module (example) that has been built directly into the binary and exposes a basic C++ class (spam_wrap) to Python (example.Spam) that has virtual function/dispatching with a default.
  • Demonstrates using the exposed Python class (example.Spam).
  • Derives from the exposed Python class (example.Spam) within Python (example.PySpam) and uses the resulting class.
#include <iostream>
#include <boost/python.hpp>/// @brief Mockup Spam model.
struct spam: boost::noncopyable
{virtual ~spam() {};virtual std::string hello() { return "Hello from C++"; }
};//@ brief Mockup Spam wrapper.
struct spam_wrap: spam,boost::python::wrapper<spam>
{virtual std::string hello(){
#if BOOST_WORKAROUND(BOOST_MSVC, <= 1300)return boost::python::call<std::string>(this->get_override("hello").ptr());
#elsereturn this->get_override("hello")();
#endif}std::string default_hello() { return this->spam::hello(); }
};/// @brief Python example module.
BOOST_PYTHON_MODULE(example)
{namespace python = boost::python;// Expose C++ spam_wrap as Python Spam class.python::class_<spam_wrap, boost::noncopyable>("Spam").def("hello", &spam::hello, &spam_wrap::default_hello);
}   int main()
{// Add example to built-in.PyImport_AppendInittab("example", &PyInit_example);// Start the interpreter.Py_Initialize();namespace python = boost::python;try{python::object main = python::import("__main__");python::object global = main.attr("__dict__");// Execute Python code, using the example module.exec("from example import Spam          \n""spam = Spam()                     \n""                                  \n""class PySpam(Spam):               \n""    def hello(self):              \n""        return 'Hello from Python'\n",     global, global);/// Check the instance of the Python object using the C++ class.// >>> spam_object = spampython::object spam_object = global["spam"];assert(python::extract<spam>(spam_object).check());// >>> result = spam_object.hello()python::object result = spam_object.attr("hello")();// >>> print(result)std::cout << python::extract<std::string>(result)() << std::endl;// >>> assert("Hello from C++" == result)assert("Hello from C++" == python::extract<std::string>(result)());/// Create an instance using PySpam class.  It too is a Python object.// >>> py_spam_type = PySpampython::object py_spam_type = global["PySpam"];// >>> py_spam_object = py_spam_type()python::object py_spam_object = py_spam_type();// >>> result = py_spam_object()result = py_spam_object.attr("hello")();// >>> print(result)std::cout << python::extract<std::string>(result)() << std::endl;// >>> assert("Hello from Python" == result)assert("Hello from Python" == python::extract<std::string>(result)());}catch (const python::error_already_set&){PyErr_Print();}
}

The program should run to completion without errors, resulting in the following output:

Hello from C++
Hello from Python
https://en.xdnf.cn/q/71393.html

Related Q&A

Python NET call C# method which has a return value and an out parameter

Im having the following static C# methodpublic static bool TryParse (string s, out double result)which I would like to call from Python using the Python NET package.import clr from System import Double…

ValueError: Length of passed values is 7, index implies 0

I am trying to get 1minute open, high, low, close, volume values from bitmex using ccxt. everything seems to be fine however im not sure how to fix this error. I know that the index is 7 because there …

What is pythons strategy to manage allocation/freeing of large variables?

As a follow-up to this question, it appears that there are different allocation/deallocation strategies for little and big variables in (C)Python. More precisely, there seems to be a boundary in the ob…

Why is cross_val_predict so much slower than fit for KNeighborsClassifier?

Running locally on a Jupyter notebook and using the MNIST dataset (28k entries, 28x28 pixels per image, the following takes 27 seconds. from sklearn.neighbors import KNeighborsClassifierknn_clf = KNeig…

Do I need to do any text cleaning for Spacy NER?

I am new to NER and Spacy. Trying to figure out what, if any, text cleaning needs to be done. Seems like some examples Ive found trim the leading and trailing whitespace and then muck with the start/st…

Hi , I have error related to object detection project

I have error related to simple object detection .output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()] IndexError: invalid index to scalar variable.import cv2.cv2 as cv import…

What is the fastest way to calculate / create powers of ten?

If as the input you provide the (integer) power, what is the fastest way to create the corresponding power of ten? Here are four alternatives I could come up with, and the fastest way seems to be usin…

How to disable date interpolation in matplotlib?

Despite trying some solutions available on SO and at Matplotlibs documentation, Im still unable to disable Matplotlibs creation of weekend dates on the x-axis.As you can see see below, it adds dates to…

Continuous error band with Plotly Express in Python [duplicate]

This question already has answers here:Plotly: How to make a figure with multiple lines and shaded area for standard deviations?(5 answers)Closed 2 years ago.I need to plot data with continuous error …

How to preprocess training set for VGG16 fine tuning in Keras?

I have fine tuned the Keras VGG16 model, but Im unsure about the preprocessing during the training phase.I create a train generator as follow:train_datagen = ImageDataGenerator(rescale=1./255) train_ge…