Python binding for C++ operator overloading

2024/9/23 23:32:10

I have a class similar to the following:

class A {vector<double> v;double& x(int i) { return v[2*i]; }double& y(int i) { return v[2*i+1]; }double x(int i) const { return v[2*i]; }double y(int i) const { return v[2*i+1]; }
}

I want to have the following Python code work:

a = A()
a.x[0] = 4
print a.x[0]

I was thinking of __setattr__ and __getattr__, but not sure if it works. An alternative is to implement the following Python:

a = A()
a['x', 0] = 4
print a['x', 0]

not as good as the previous one, but might be easier to implement (with __slice__ ?).

PS. I am using sip to do the binding.

Thanks.

Answer

It is possible with __getattr__ and custom %MethodCode; however, there are a few points to take into consideration:

  • An intermediate type/object needs to be created, as a.x will return an object that provides __getitem__ and __setitem__. Both methods should raise an IndexError when out of bounds occurs, as this is part of the old protocol used to iterate via __getitem__; without it, a crash would occur when iterating over a.x.
  • In order to guarantee the lifetime of the vector, the a.x object needs to maintain a reference to the object that owns the vector (a). Consider the following code:

    a = A()
    x = a.x
    a = None # If 'x' has a reference to 'a.v' and not 'a', then it may have a# dangling reference, as 'a' is refcounted by python, and 'a.v' is# not refcounted.
    
  • Writing %MethodCode can be difficult, especially when having to manage the reference counting during error cases. It requires an understanding of the python C API and SIP.

For an alternative solution, consider:

  • Design the python bindings to provide functionality.
  • Design class(es) in python to provide the pythonic interface that uses the bindings.

While the approach has a few drawbacks, such as the code is separated into more files that may need to be distributed with the library, it does provide some major benefits:

  • It is much easier to implement a pythonic interface in python than in C or the interoperability library's interface.
  • Support for slicing, iterators, etc. can be more naturally implemented in python, instead of having to manage it through the C API.
  • Can leverage python's garbage collector to manage the lifetime of the underlying memory.
  • The pythonic interface is decoupled from whatever implementation is being used to provide interoperability between python and C++. With a flatter and simpler binding interface, changing between implementations, such as Boost.Python and SIP, is much easier.

Here is an walk-through demonstrating this approach. First, we start with the basic A class. In this example, I have provided a constructor that will set some initial data.

a.hpp:

#ifndef A_HPP
#define A_HPP#include <vector>class A
{std::vector< double > v;
public:A() { for ( int i = 0; i < 6; ++i ) v.push_back( i ); }double& x( int i )         { return v[2*i];       }double  x( int i ) const   { return v[2*i];       }double& y( int i )         { return v[2*i+1];     }double  y( int i ) const   { return v[2*i+1];     }std::size_t size() const   { return v.size() / 2; }
};#endif  // A_HPP

Before doing the bindings, lets examine the A interface. While it is an easy interface to use in C++, it has some difficulties in python:

  • Python does not support overloaded methods, and idioms to support overloading will fail when the argument type/counts are the same.
  • The concept of a reference to a double (float in Python) is different between the two languages. In Python, the float is an immutable type, so its value cannot be changed. For example, in Python the statement n = a.x[0] binds n to reference the float object returned from a.x[0]. The assignment n = 4 rebinds n to reference the int(4) object; it does not set a.x[0] to 4.
  • __len__ expects int, not std::size_t.

Lets create a basic intermediate class that will help simplify the bindings.

pya.hpp:

#ifndef PYA_HPP
#define PYA_HPP#include "a.hpp"struct PyA: A
{double get_x( int i )           { return x( i ); }void   set_x( int i, double v ) { x( i ) = v;    }double get_y( int i )           { return y( i ); }void   set_y( int i, double v ) { y( i ) = v;    }int    length()                 { return size(); }
};#endif // PYA_HPP

Great! PyA now provides member functions that do not return references, and length is returned as an int. It is not the best of interfaces, the bindings are being designed to provide the needed functionality, rather than the desired interface.

Now, lets write some simple bindings that will create class A in the cexample module.

Here is the bindings in SIP:

%Module cexampleclass PyA /PyName=A/
{
%TypeHeaderCode
#include "pya.hpp"
%End
public:double get_x( int );void set_x( int, double );double get_y( int );void set_y( int, double );int __len__();%MethodCodesipRes = sipCpp->length();%End
};

Or if you prefer Boost.Python:

#include "pya.hpp"
#include <boost/python.hpp>BOOST_PYTHON_MODULE(cexample)
{using namespace boost::python;class_< PyA >( "A" ).def( "get_x",   &PyA::get_x  ).def( "set_x",   &PyA::set_x  ).def( "get_y",   &PyA::get_y  ).def( "set_y",   &PyA::set_y  ).def( "__len__", &PyA::length );
}

Due to the PyA intermediate class, both of the bindings are fairly simple. Additionally, this approach requires less SIP and Python C API knowledge, as it requires less code within %MethodCode blocks.

Finally, create example.py that will provide the desired pythonic interface:

class A:class __Helper:def __init__( self, data, getter, setter ):self.__data   = dataself.__getter = getterself.__setter = setterdef __getitem__( self, index ):if len( self ) <= index:raise IndexError( "index out of range" )return self.__getter( index )def __setitem__( self, index, value ):if len( self ) <= index:raise IndexError( "index out of range" )self.__setter( index, value )def __len__( self ):return len( self.__data )def __init__( self ):import cexamplea = cexample.A()self.x = A.__Helper( a, a.get_x, a.set_x )self.y = A.__Helper( a, a.get_y, a.set_y )

In the end, the bindings provide the functionality we need, and python creates the interface we want. It is possible to have the bindings provide the interface; however, this can require a rich understanding of the differences between the two languages and the binding implementation.

>>> from example import A
>>> a = A()
>>> for x in a.x:
...   print x
... 
0.0
2.0
4.0
>>> a.x[0] = 4
>>> for x in a.x:
...   print x
... 
4.0
2.0
4.0
>>> x = a.x
>>> a = None
>>> print x[0]
4.0
https://en.xdnf.cn/q/71771.html

Related Q&A

python script to pickle entire environment

Im working inside the Python REPL, and I want to save my work periodically. Does anybody have a script to dump all the variables I have defined? Im looking for something like this:for o in dir():f=ope…

django-endless with class based views example

Im using Class Based Views for the first time. Im having trouble understating how using class based views I would implement django-endless-pagination twitter styling paging.Could I have an example of h…

Converting adjectives and adverbs to their noun forms

I am experimenting with word sense disambiguation using wordnet for my project. As a part of the project, I would like to convert a derived adjective or an adverb form to its root noun form.For exampl…

Sending a packet over physical loopback in scapy

Ive recently discovered Scapy & it looks wonderfulIm trying to look at simple traffic over a physical loopback module / stub on my NIC.But Scapy sniff doesnt give anythingWhat Im doing to send a pa…

Python Perfect Numbers

So I am supposed to write a Python program that will identify and print all the perfect numbers in some closed interval [ 2, n ], one per line. We only have to use nested while loops/ if-else statement…

Counting consecutive 1s in NumPy array

[1, 1, 1, 0, 0, 0, 1, 1, 0, 0]I have a NumPy array consisting of 0s and 1s like above. How can I add all consecutive 1s like below? Any time I encounter a 0, I reset.[1, 2, 3, 0, 0, 0, 1, 2, 0, 0]I ca…

python 3 replacement for dircache?

Before I go reinventing the wheel, can anyone tell me if theres a drop-in (or semi-drop-in) replacement for the single-line statement:allfiles = dircache.listdir(.)

AES_128_CTR encryption by openssl and PyCrypto

Wondering the right way to convert a AES_128_CTR encryption by openssl to PyCrypto.First, I did an encryption by openssl as following:openssl enc -aes-128-ctr -in input.mp4 -out output.openssl.mp4 -K 7…

How can i determine the exact size of a type used by python

>>> sys.getsizeof(int) 436 #? does this mean int occupies 436 bytes .>>> sys.getsizeof(1) 12 #12 bytes for int object, is this the memory requirement.I thought int in python is repre…

Python list.clear complexity [duplicate]

This question already has answers here:Python list.clear() time and space complexity?(4 answers)Closed 2 years ago.What is the complexity of the Python 3 method list.clear() ?It is not given here: ht…