Passing 2 dimensional C array to python numpy

2024/10/10 6:15:51

I need some help regarding passing C array to python(numpy). I have 2d array of doubles NumRows x NumInputs, it seems that PyArray_SimpleNewFromData does not convert it right way - it is hard to see because debugger does not show much, only pointers.

What would be the right way to pass 2 dimensional array ?

int NumRows = X_test.size();
int NumInputs = X_test_row.size();double **X_test2 = new double*[NumRows];
for(int i = 0; i < NumRows; ++i) 
{X_test2[i] = new double[NumInputs];
}for(int r = 0; r < NumRows; ++r) 
{for(int c = 0; c < NumInputs; ++c) {X_test2[r][c] = X_test[r][c];}
}const char *ScriptFName = "100-ABN-PREDICT";
char *FunctionName=NULL;FunctionName="PredictGBC_DBG"; npy_intp Dims[2];
Dims[0]= NumRows;
Dims[1] = NumInputs;PyObject *ArgsArray;
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *pArgs;int row, col, rows, cols, size, type;const double* outArray;
double ArrayItem;//===================Py_Initialize();pName = PyBytes_FromString(ScriptFName);pModule = PyImport_ImportModule(ScriptFName);if (pModule != NULL)
{import_array(); // Required for the C-APIArgsArray = PyArray_SimpleNewFromData (2, Dims, NPY_DOUBLE, X_test2);//SOMETHING WRONG pDict = PyModule_GetDict(pModule);pArgs = PyTuple_New (1);PyTuple_SetItem (pArgs, 0, ArgsArray);pFunc = PyDict_GetItemString(pDict, FunctionName);if (pFunc && PyCallable_Check(pFunc)){pValue = PyObject_CallObject(pFunc, pArgs);//CRASHING HEREif (pValue != NULL) {rows = PyArray_DIM(pValue, 0);cols = PyArray_DIM(pValue, 1);size = PyArray_SIZE(pValue);type = PyArray_TYPE(pValue);// get direct access to the array data//PyObject* m_obj;outArray = static_cast<const double*>(PyArray_DATA(pValue));for (row=0; row < rows; row++) {ArrayItem = outArray[row];y_pred.push_back(ArrayItem);}}else {y_pred.push_back(EMPTY_VAL);}}else {PyErr_Print();}//pFunc && PyCallable_Check(pFunc)}//(pModule!=NULL
else
{PyErr_SetString(PyExc_TypeError, "Cannot call function ?!");PyErr_Print();
}Py_DECREF(pValue);
Py_DECREF(pFunc);Py_DECREF(ArgsArray);  
Py_DECREF(pModule);
Py_DECREF(pName);Py_Finalize (); 
Answer

You'll have to copy your data to a contiguous block of memory. To represent a 2d array, numpy does not use an array of pointers to 1d arrays. Numpy expects the array to be stored in a contiguous block of memory, in (by default) row major order.

If you create your array using PyArray_SimpleNew(...), numpy allocates the memory for you. You have to copy X_test2 to this array, using, say, std::memcpy or std::copy in a loop over the rows.

That is, change this:

ArgsArray = PyArray_SimpleNewFromData (2, Dims, NPY_DOUBLE, X_test2);//SOMETHING WRONG 

to something like this:

// PyArray_SimpleNew allocates the memory needed for the array.
ArgsArray = PyArray_SimpleNew(2, Dims, NPY_DOUBLE);// The pointer to the array data is accessed using PyArray_DATA()
double *p = (double *) PyArray_DATA(ArgsArray);// Copy the data from the "array of arrays" to the contiguous numpy array.
for (int k = 0; k < NumRows; ++k) {memcpy(p, X_test2[k], sizeof(double) * NumInputs);p += NumInputs;
}

(It looks like X_test2 is a copy of X_test, so you might want to modify the above code to copy directly from X_test to the numpy array.)

https://en.xdnf.cn/q/69923.html

Related Q&A

Best way to implement numpy.sin(x) / x where x might contain 0

What I am doing now is:import numpy as npeps = np.finfo(float).epsdef sindiv(x):x = np.abs(x)return np.maximum(eps, np.sin(x)) / np.maximum(eps, x)But there is quite a lot of additional array operation…

Scrapy process.crawl() to export data to json

This might be a subquestion of Passing arguments to process.crawl in Scrapy python but the author marked the answer (that doesnt answer the subquestion im asking myself) as a satisfying one.Heres my pr…

Embedding Python in C: Error in linking - undefined reference to PyString_AsString

I am trying to embed a python program inside a C program. My OS is Ubuntu 14.04I try to embed python 2.7 and python 3.4 interpreter in the same C code base (as separate applications). The compilation a…

How can I add properties to a class using a decorator that takes a list of names as argument?

I would like to add many dummy-properties to a class via a decorator, like this:def addAttrs(attr_names):def deco(cls):for attr_name in attr_names:def getAttr(self):return getattr(self, "_" +…

How to reshape only last dimensions in numpy?

Suppose I have A of shape (...,96) and want to reshape it into (...,32,3) keeping both lengths and number of preceding dimensions, if ever, intact.How to do this?If I writenp.reshape(A, (-1, 32, 2))it…

Relative import of submodule

In Python, how do I perform the equivalent of the followingimport http.clientbut using a relative import:from . import http.client import .http.clientFor a package http in the current package? I want …

Python regular expression to replace everything but specific words

I am trying to do the following with a regular expression:import re x = re.compile([^(going)|^(you)]) # words to replace s = I am going home now, thank you. # string to modify print re.sub(x, _, s)T…

How do I raise a window that is minimized or covered with PyGObject?

Id been using the answer provided in the PyGTK FAQ, but that doesnt seem to work with PyGObject. For your convenience, here is a test case that works with PyGTK, and then a translated version that does…

How to bind multiple widgets with one bind in Tkinter?

I am wondering how to bind multiple widgets with one "bind".For expample: I have three buttons and I want to change their color after hovering.from Tkinter import *def SetColor(event):event.w…

Iterate a large .xz file line by line in python

I have a large .xz file (few gigabytes). Its full of plain text. I want to process the text to create custom dataset. I want to read it line by line because it is too big. Anyone have an idea how to do…