I am trying to pass different functions which have pointers as arguments to a python function. One example of the input function as input parameter is the given normal
function:
Sample.pyx
from cpython cimport array
import cython
import ctypes
cimport numpy as np
cpdef void normal(np.ndarray[ndim=1, dtype=np.float64_t] u, np.ndarray[ndim=1, dtype=np.float64_t] yu, np.ndarray[ndim=1, dtype=np.float64_t] ypu):cdef int icdef int n=len(u)for i in prange(n, nogil=True):yu[i]=-u[i]*u[i]*0.5 ypu[i]=-u[i] return
cdef class _SampleFunc:cdef void (*func)(double *, double *, double *)cdef void sample(int* x, double* hx, double* hxx, void(*func)(double*, double*, double*), int n):def int ifor i from 0 <= i < n:func[0](&x[i], &hx[i], &hxx[i])return
cdef class myClass:sample_wrapper = _SampleFunc() sample_wrapper.func = Nulldef foo(np.ndarray[ndim=1, dtype=np.float64_t] x,np.ndarray[ndim=1, dtype=np.float64_t] hx,np.ndarray[ndim=1, dtype=np.float64_t] hxx,_SampleFunc sample_func, int k):cdef np.ndarray[ndim=1, dtype=np.float64_t] spcdef int num=len(x)func = sample_func.funcassert func is not NULL, "function value is NULL"cdef int jfor j from 0 <= j <k:sample(&x[0],&hx[0], &hxx[0], func, num)sp[j]=hx[0]return sp
test.py
import numpy as np
from sample import *
x = np.zeros(10, float)
hx = np.zeros(10, float)
hpx = np.zeros(10, float)x[0] = 0
x[1] = 1.0
x[2] = -1.0
def pynormal(x):return -x*x*0.5,-xhx[0], hpx[0] = pynormal(x[0])
hx[1], hpx[1] = pynormal(x[1])
hx[2], hpx[2] = pynormal(x[2])
num=20
ars=myClass()
s=ars.foo( x, hx, hpx, normal, num)
Running the test.py
code I am getting this error:
'ars._SampleFunc' object has no attribute 'func'
I am trying to write a wrapper for different C
functions which have three pointer arrays as their argument. My conclusion so far was that it can be done with a class, since the class can be accessible in python. I am wondering how I can pass the C
functions with pointer arrays as argument to myClass
class?
Update: Normal function
cdef void normal(int n,double* u, double* yu, double* ypu): cdef int i for i in prange(n, nogil=True):yu[i]=-u[i]*u[i]*0.5 ypu[i]=-u[i] return
The first thing to deal with is that a function of signature cdef void (*func)(double *, double *, double *)
does not pass the array length. You can't know how long these arrays are, and thus you can't safely access their elements. The sensible thing is to change the function signature to pass a length too:
cdef void (*func)(double *, double *, double *, int)
What is extra confusing is that you seem to be iterating over the same axis of a 1D array in both normal
and sample
. I suspect that isn't what you want to do, but I'm not going attempt to fix that.
Essentially your problem is that you want to pass an arbitrary Python callable as a C function pointer. The bad news is that Cython can't do it - a Python callable has a significant amount of information associated with it, while a C function pointer is simply the address of some executable memory. Therefore a C function pointer does not have the space available to hold the information in a Python callable. In order to make this work you need to generate code at runtime, which Python can't do.
I've recommended the ctypes standard library module as a solution to similar problems previously, since it can create a function pointer from a Python callable. There is a simpler but more limited solution if you only want to call cdef
Cython functions.
ctypes
Here's a minimal example which demonstrates how to implement the idea:
import numpy as np
import ctypesctypedef void (*func_t)(int, double *)cdef void sample(int n, double* x, func_t f):f(n,x)def call_sample(double[::1] x,f):def func_wrapper(n, arg1):# x is a slightly opaque ctypes type# first cast it to a ctypes array of known size# and then create a numpy array from thatarg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))return f(np.asarray(arg1_as_ctypes_array))FTYPE = ctypes.CFUNCTYPE(None, # return typectypes.c_int, # argumentsctypes.POINTER(ctypes.c_double))f_ctypes = FTYPE(func_wrapper) # convert Python callable to ctypes function pointer# a rather nasty line to convert to a C function pointercdef func_t f_ptr = (<func_t*><size_t>ctypes.addressof(f_ctypes))[0]sample(x.shape[0], &x[0], f_ptr)def example_function(x):# expects a numpy array like objectprint(x)def test():a = np.random.rand(20)print(a)call_sample(a,example_function)
I realise that there's some slightly messy conversion between ctypes and Cython - this is unavoidable.
A bit of explanation: I'm assuming you want to keep the Python interface simple, hence example_function
just takes a numpy array-like object. The function passed by ctypes needs to accept a number of elements and a pointer to match your C interface.
The ctypes pointer type (LP_c_double
) can do do indexing (i.e. arg1[5]
) so it works fine for simple uses, but it doesn't store its length internally. It's helpful (but not essential) to change it to a numpy array so you can use it more generally and thus we create a wrapper function to do this. We do:
arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))
to convert it to a known length ctypes array and then
np.asarray(arg1_as_ctypes_array)
to convert it to a numpy array. This shares the data rather than makes a copy, so if you change it then your original data will be changed. Because the conversion to a numpy array follows a standard pattern it's easy to generate a wrapper function in call_sample
.
(In the comments you ask how to do the conversion if you're just passing a double
, not a double*
. In this case you don't have to do anything since a ctypes double
behaves exactly like a Python type)
Only cdef
functions
If you're certain the functions you want to pass will always be cdef
functions then you can avoid ctypes and come up with something a bit simpler. You first need to make the function signature match the pointer exactly:
cdef void normal(int N, double *x): # other parameters as necessarycdef double[::1] x_as_mview = <double[:N:1]>x # cast to a memoryview# ... etc
You should then be able to use your definition of SampleFunc
almost as is to create module level objects:
# in Cython
normal_samplefunc = SampleFunc()
normal_samplefunc.func = &normal# in Python
s=ars.foo( x, hx, hpx, normal_samplefunc, num)
ars.foo
is the way you wrote it (no ctypes
code):
func = sample_func.func
# ...
sample(..., func,...)
This code will run quicker, but you want be able to call normal
from Python.
Python interface
You mention in the comments that you'd also like the be able to access normal
from Python. You're likely to need a different interface for the Python function and the one you pass to C, so I'd define a separate function for both uses, but share the implementation:
def normal(double[::1] u, # ... other arguments):# or cpdef, if you really wantimplementation goes here# then, depending on if you're using ctypes or not:
def normal_ctypes(int n, u # other arguments ...):u_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(x.contents))normal(u_as_ctypes_array, # other arguments)# or
cdef void normal_c(int n, double* u # ...):normal(<double[:N:1]>x # ...)