pass different C functions with pointer arrays as the function argument to a class

2024/7/5 11:05:42

I am trying to pass different functions which have pointers as arguments to a python function. One example of the input function as input parameter is the given normal function:

Sample.pyx

from cpython cimport array
import cython
import ctypes
cimport numpy as np
cpdef void normal(np.ndarray[ndim=1, dtype=np.float64_t] u, np.ndarray[ndim=1, dtype=np.float64_t] yu, np.ndarray[ndim=1, dtype=np.float64_t] ypu):cdef int icdef int n=len(u)for i in prange(n, nogil=True):yu[i]=-u[i]*u[i]*0.5                                                               ypu[i]=-u[i]                                                                    return                                                     
cdef class _SampleFunc:cdef void (*func)(double *, double *, double *)cdef void sample(int* x, double* hx, double* hxx, void(*func)(double*, double*, double*), int n):def int ifor i from 0 <= i < n:func[0](&x[i], &hx[i], &hxx[i])return 
cdef class myClass:sample_wrapper = _SampleFunc() sample_wrapper.func = Nulldef foo(np.ndarray[ndim=1, dtype=np.float64_t] x,np.ndarray[ndim=1, dtype=np.float64_t] hx,np.ndarray[ndim=1, dtype=np.float64_t] hxx,_SampleFunc sample_func, int k):cdef np.ndarray[ndim=1, dtype=np.float64_t] spcdef int num=len(x)func = sample_func.funcassert func is not NULL, "function value is NULL"cdef int jfor j from 0 <= j <k:sample(&x[0],&hx[0], &hxx[0], func, num)sp[j]=hx[0]return sp

test.py

import numpy as np
from sample import *
x = np.zeros(10, float)
hx = np.zeros(10, float)
hpx = np.zeros(10, float)x[0] = 0
x[1] = 1.0
x[2] = -1.0
def pynormal(x):return -x*x*0.5,-xhx[0], hpx[0] = pynormal(x[0])
hx[1], hpx[1] = pynormal(x[1])
hx[2], hpx[2] = pynormal(x[2])
num=20
ars=myClass()
s=ars.foo( x, hx, hpx, normal, num)

Running the test.py code I am getting this error:

'ars._SampleFunc' object has no attribute 'func'

I am trying to write a wrapper for different C functions which have three pointer arrays as their argument. My conclusion so far was that it can be done with a class, since the class can be accessible in python. I am wondering how I can pass the C functions with pointer arrays as argument to myClass class?

Update: Normal function

cdef void normal(int n,double* u, double* yu, double* ypu):          cdef int i          for i in prange(n, nogil=True):yu[i]=-u[i]*u[i]*0.5                                                               ypu[i]=-u[i]                                                                    return 
Answer

The first thing to deal with is that a function of signature cdef void (*func)(double *, double *, double *) does not pass the array length. You can't know how long these arrays are, and thus you can't safely access their elements. The sensible thing is to change the function signature to pass a length too:

cdef void (*func)(double *, double *, double *, int)

What is extra confusing is that you seem to be iterating over the same axis of a 1D array in both normal and sample. I suspect that isn't what you want to do, but I'm not going attempt to fix that.


Essentially your problem is that you want to pass an arbitrary Python callable as a C function pointer. The bad news is that Cython can't do it - a Python callable has a significant amount of information associated with it, while a C function pointer is simply the address of some executable memory. Therefore a C function pointer does not have the space available to hold the information in a Python callable. In order to make this work you need to generate code at runtime, which Python can't do.

I've recommended the ctypes standard library module as a solution to similar problems previously, since it can create a function pointer from a Python callable. There is a simpler but more limited solution if you only want to call cdef Cython functions.

ctypes

Here's a minimal example which demonstrates how to implement the idea:

import numpy as np
import ctypesctypedef void (*func_t)(int, double *)cdef void sample(int n, double* x, func_t f):f(n,x)def call_sample(double[::1] x,f):def func_wrapper(n, arg1):# x is a slightly opaque ctypes type# first cast it to a ctypes array of known size# and then create a numpy array from thatarg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))return f(np.asarray(arg1_as_ctypes_array))FTYPE = ctypes.CFUNCTYPE(None, # return typectypes.c_int, # argumentsctypes.POINTER(ctypes.c_double))f_ctypes = FTYPE(func_wrapper) # convert Python callable to ctypes function pointer# a rather nasty line to convert to a C function pointercdef func_t f_ptr = (<func_t*><size_t>ctypes.addressof(f_ctypes))[0]sample(x.shape[0], &x[0], f_ptr)def example_function(x):# expects a numpy array like objectprint(x)def test():a = np.random.rand(20)print(a)call_sample(a,example_function)

I realise that there's some slightly messy conversion between ctypes and Cython - this is unavoidable.

A bit of explanation: I'm assuming you want to keep the Python interface simple, hence example_function just takes a numpy array-like object. The function passed by ctypes needs to accept a number of elements and a pointer to match your C interface.

The ctypes pointer type (LP_c_double) can do do indexing (i.e. arg1[5]) so it works fine for simple uses, but it doesn't store its length internally. It's helpful (but not essential) to change it to a numpy array so you can use it more generally and thus we create a wrapper function to do this. We do:

arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))

to convert it to a known length ctypes array and then

np.asarray(arg1_as_ctypes_array)

to convert it to a numpy array. This shares the data rather than makes a copy, so if you change it then your original data will be changed. Because the conversion to a numpy array follows a standard pattern it's easy to generate a wrapper function in call_sample.

(In the comments you ask how to do the conversion if you're just passing a double, not a double*. In this case you don't have to do anything since a ctypes double behaves exactly like a Python type)

Only cdef functions

If you're certain the functions you want to pass will always be cdef functions then you can avoid ctypes and come up with something a bit simpler. You first need to make the function signature match the pointer exactly:

cdef void normal(int N, double *x): # other parameters as necessarycdef double[::1] x_as_mview = <double[:N:1]>x # cast to a memoryview# ... etc

You should then be able to use your definition of SampleFunc almost as is to create module level objects:

# in Cython
normal_samplefunc = SampleFunc()
normal_samplefunc.func = &normal# in Python
s=ars.foo( x, hx, hpx, normal_samplefunc, num)

ars.foo is the way you wrote it (no ctypes code):

func = sample_func.func
# ...
sample(..., func,...)

This code will run quicker, but you want be able to call normal from Python.


Python interface

You mention in the comments that you'd also like the be able to access normal from Python. You're likely to need a different interface for the Python function and the one you pass to C, so I'd define a separate function for both uses, but share the implementation:

def normal(double[::1] u, # ... other arguments):# or cpdef, if you really wantimplementation goes here# then, depending on if you're using ctypes or not:
def normal_ctypes(int n, u # other arguments ...):u_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(x.contents))normal(u_as_ctypes_array, # other arguments)# or
cdef void normal_c(int n, double* u # ...):normal(<double[:N:1]>x # ...)
https://en.xdnf.cn/q/120093.html

Related Q&A

dynamic filter choice field in django

I am using django-filter to filter results on a page. I am able to use static choice filters like this:filters.pyFILTER_CHOICES = ( (, All States), (AL, Alabama), (AK, Alaska), (AZ, Arizona), #and so f…

How to print a table from a text file and fill empty spaces?

I have the following table in a txt file:|Client | Container weight | Country of Origin | Product code ||S4378 | 450 Ton | China | 457841 || | 350 Ton | Japan…

Open a file in python from 2 directory back

I want to read a file from 2 folders back.. with open(../../test.txt, r) as file:lines = file.readlines()file.close()I want to read from ../../ two folders back. but not work.. How i can do that ?

Do string representations of dictionaries have order in Python 3.4?

I know dictionaries themselves in Python do not have order. However, Im rather curious if when you call str() on a dictionary if it is always in the same order. It appears to be sorted (by key), no mat…

BeautifulSoup Scraping Results not showing

I am playing around with BeautifulSoup to scrape data from websites. So I decided to scrape empireonlines website for 100 greatest movies of all time. Heres the link to the webpage: https://www.empireo…

How to verify username and password from CSV file in Python?

I am doing a Python project where I have to verify my username and password from a csv file where the first two rows and columns have the username and password as hi.Current Code: answer = input("…

adding validation to answer in quiz gives wrong answers

I am a complete novice with Python and working on a multiple choice quiz that reads questions from a file and keeps a score that then writes to a file. Everything was working perfectly until I added v…

Why do I get None as the output from a print statement? [duplicate]

This question already has answers here:Why is "None" printed after my functions output?(7 answers)Closed 2 years ago.def print_name(name):print(name)print(print_name(Annabel Lee))Why do I ge…

How to collect tweets about an event that are posted on specific date using python?

I wish to collect all tweets containing specific keywords(ex:manchesterattack,manchester) that are posted about the manchester attack from 22may. Can anyone provide me a code to collect tweets using py…

Pivoting a One-Hot-Encode Dataframe

I have a pandas dataframe that looks like this:genres.head()Drama Comedy Action Crime Romance Thriller Adventure Horror Mystery Fantasy ... History Music War Documentary Sport Musical W…