I have code to extract a numeric value from a python sequence, and it works well in most cases, but not for a numpy array.
When I try to extract an unsigned char, I do the following
unsigned char val = boost::python::extract<unsigned char>(sequence[n]);
where sequence is any python sequence and n is the index.I get the following error:
TypeError: No registered converter was able to produce a C++ rvalue of type
unsigned char from this Python object of type numpy.uint8
How can I successfully extract an unsigned char in C++? Do I have to write/register special converters for numpy types? I would rather use the same code that I use for other python sequences, and not have to write special code that uses the PyArrayObject*.
One can register a custom from-python converter with Boost.Python that handles conversions from NumPy array scalars, such as numpy.uint8, to C++ scalars, such as unsigned char. A custom from-python converter registration has three parts:
- A function that checks if a
PyObject is convertible. A return of NULL indicates that the PyObject cannot use the registered converter.
- A construct function that constructs the C++ type from a
PyObject. This function will only be called if converter(PyObject) does not return NULL.
- The C++ type that will be constructed.
Extracting the value from the NumPy array scalar requires a few NumPy C API calls:
import_array() must be called within the initialization of an extension module that is going to use the NumPy C API. Depending on how the extension(s) are using the NumPy C API, other requirements for importing may need to occur.PyArray_CheckScalar() checks if a PyObject is a NumPy array scalar.PyArray_DescrFromScalar() gets the data-type-descriptor object for an array scalar. The data-type-descriptor object contains information about how to interpret the underlying bytes. For example, its type_num data member contains an enum value that corresponds to a C-type.PyArray_ScalarAsCtype() can be used to extract the C-type value from a NumPy array scalar.
Here is a complete example demonstrating using a helper class, enable_numpy_scalar_converter, to register specific NumPy array scalars to their corresponding C++ types.
#include <boost/cstdint.hpp>
#include <boost/python.hpp>
#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include <numpy/arrayobject.h>// Mockup functions./// @brief Mockup function that will explicitly extract a uint8_t
/// from the Boost.Python object.
boost::uint8_t test_generic_uint8(boost::python::object object)
{return boost::python::extract<boost::uint8_t>(object)();
}/// @brief Mockup function that uses automatic conversions for uint8_t.
boost::uint8_t test_specific_uint8(boost::uint8_t value) { return value; }/// @brief Mokcup function that uses automatic conversions for int32_t.
boost::int32_t test_specific_int32(boost::int32_t value) { return value; }/// @brief Converter type that enables automatic conversions between NumPy
/// scalars and C++ types.
template <typename T, NPY_TYPES NumPyScalarType>
struct enable_numpy_scalar_converter
{enable_numpy_scalar_converter(){// Required NumPy call in order to use the NumPy C API within another// extension module.import_array();boost::python::converter::registry::push_back(&convertible,&construct,boost::python::type_id<T>());}static void* convertible(PyObject* object){// The object is convertible if all of the following are true:// - is a valid object.// - is a numpy array scalar.// - its descriptor type matches the type for this converter.return (object && // ValidPyArray_CheckScalar(object) && // ScalarPyArray_DescrFromScalar(object)->type_num == NumPyScalarType // Match)? object // The Python object can be converted.: NULL;}static void construct(PyObject* object,boost::python::converter::rvalue_from_python_stage1_data* data){// Obtain a handle to the memory block that the converter has allocated// for the C++ type.namespace python = boost::python;typedef python::converter::rvalue_from_python_storage<T> storage_type;void* storage = reinterpret_cast<storage_type*>(data)->storage.bytes;// Extract the array scalar type directly into the storage.PyArray_ScalarAsCtype(object, storage);// Set convertible to indicate success. data->convertible = storage;}
};BOOST_PYTHON_MODULE(example)
{namespace python = boost::python;// Enable numpy scalar conversions.enable_numpy_scalar_converter<boost::uint8_t, NPY_UBYTE>();enable_numpy_scalar_converter<boost::int32_t, NPY_INT>();// Expose test functions.python::def("test_generic_uint8", &test_generic_uint8);python::def("test_specific_uint8", &test_specific_uint8);python::def("test_specific_int32", &test_specific_int32);
}
Interactive usage:
>>> import numpy
>>> import example
>>> assert(42 == example.test_generic_uint8(42))
>>> assert(42 == example.test_generic_uint8(numpy.uint8(42)))
>>> assert(42 == example.test_specific_uint8(42))
>>> assert(42 == example.test_specific_uint8(numpy.uint8(42)))
>>> assert(42 == example.test_specific_int32(numpy.int32(42)))
>>> example.test_specific_int32(numpy.int8(42))
Traceback (most recent call last):File "<stdin>", line 1, in <module>
Boost.Python.ArgumentError: Python argument types inexample.test_specific_int32(numpy.int8)
did not match C++ signature:test_specific_int32(int)
>>> example.test_generic_uint8(numpy.int8(42))
Traceback (most recent call last):File "<stdin>", line 1, in <module>
TypeError: No registered converter was able to produce a C++ rvalue of typeunsigned char from this Python object of type numpy.int8
A few things to note from the interactive usage:
- Boost.Python was able to extract
boost::uint8_t from both numpy.uint8 and int Python objects.
- The
enable_numpy_scalar_converter does not support promotions. For instance, it should be safe for test_specific_int32() to accept a numpy.int8 object that is promoted to a larger scalar type, such as int. If one wishes to perform promotions:
convertible() will need to check for compatible NPY_TYPESconstruct() should use PyArray_CastScalarToCtype() to cast the extracted array scalar value to the desired C++ type.