Cython: Segmentation Fault Using API Embedding Cython to C

2024/10/2 20:33:04

I'm trying to embed Cython code into C following O'reilly Cython book chapter 8. I found this paragraph on Cython's documentation but still don't know what should I do:

If the C code wanting to use these functions is part of more than one shared library or executable, then import_modulename() function needs to be called in each of the shared libraries which use these functions. If you crash with a segmentation fault (SIGSEGV on linux) when calling into one of these api calls, this is likely an indication that the shared library which contains the api call which is generating the segmentation fault does not call the import_modulename() function before the api call which crashes.

I'm running Python 3.4, Cython 0.23 and GCC 5 on OS X. The source code are transcendentals.pyx and main.c:

main.c

#include "transcendentals_api.h"
#include <math.h>
#include <stdio.h>int main(int argc, char **argv)
{Py_SetPythonHome(L"/Users/spacegoing/anaconda");Py_Initialize();import_transcendentals();printf("pi**e: %f\n", pow(get_pi(), get_e()));Py_Finalize();return 0;
}

transcendentals.pyx

cdef api double get_pi():return 3.1415926cdef api double get_e():print("calling get_e()")return 2.718281828

I'm compiling those files using setup.py and Makefile:

setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonizesetup(ext_modules=cythonize([Extension("transcendentals", ["transcendentals.pyx"])])
)

Makefile

python-config=/Users/spacegoing/anaconda/bin/python3-config
ldflags:=$(shell $(python-config) --ldflags)
cflags:=$(shell $(python-config) --cflags)a.out: main.c transcendentals.sogcc-5 $(cflags) $(ldflags) transcendentals.c main.ctranscendentals.so: setup.py transcendentals.pyxpython setup.py build_ext --inplacecython transcendentals.pyxclean:rm -r a.out a.out.dSYM build transcendentals.[ch] transcendentals.so transcendentals_api.h

However, I came to error Segmentation fault: 11. Any idea can help with this? Thanks!

Answer

In that Makefile there is

transcendentals.so: setup.py transcendentals.pyxpython setup.py build_ext --inplace

Unless python refers to /Users/spacegoing/anaconda/bin/python3 it should be replaced since the module may be compiled for wrong python version, and cannot thus be loaded.

In main.c there is call import_transcendentals() that does not check the return value i.e. if the import fails or succeeds. In case of failure, get_pi() and get_e() point to invalid memory locations and trying to call them causes a segmentation fault.

Also, the module has to be located somewhere where it can be found. It seems that when embedding, the current directory is not searched for python modules. PYTHONPATH environment variable could be changed to include the directory where transcendentals.so is located.

The following is an altenative way of embedding the code to the C program and sidesteps the import issues since the module code is linked to the executable.

Essentially, a call to PyInit_transcendentals() is missing.

File transcendentals.h will be generated when the cython functions are defined public i.e.

cdef public api double get_pi():
...
cdef public api double get_e():

Your main.c should have the include directives

#include <Python.h>
#include "transcendentals.h"

and then in main

Py_Initialize();
PyInit_transcendentals();

There should be no #include "transcendentals_api.h" and no import_transcendentals()

The first reason is that according to the documentation

However, note that you should include either modulename.h ormodulename_api.h in a given C file, not both, otherwise you may getconflicting dual definitions.

The second reason is, that since transcendentals.c is linked to the program in

gcc $(cflags) $(ldflags) transcendentals.c main.c

there is no reason to import transcendentals module. The module has to be initialized though, PyInit_transcendentals() does that for Python 3

https://en.xdnf.cn/q/70807.html

Related Q&A

Computing AUC and ROC curve from multi-class data in scikit-learn (sklearn)?

I am trying to use the scikit-learn module to compute AUC and plot ROC curves for the output of three different classifiers to compare their performance. I am very new to this topic, and I am struggli…

Nested dictionary

I am working on some FASTA-like sequences (not FASTA, but something I have defined thats similar for some culled PDB from the PISCES server).I have a question. I have a small no of sequences called nCa…

How to create a Python script to automate software installation? [closed]

Closed. This question is seeking recommendations for books, tools, software libraries, and more. It does not meet Stack Overflow guidelines. It is not currently accepting answers.We don’t allow questi…

Using __str__() method in Django on Python 2

I am Learning Django using the Django project tutorial. Since I use python 2.7 I am unable to implement the following in python 2.7:from django.db import modelsclass Question(models.Model): # ...def _…

Losing merged cells border while editing Excel file with openpyxl

I have two sheets in an Excel file and the first one is a cover sheet which I dont need to edit. There are a few merged cells in the cover sheet, and when I edit the file using openpyxl, without even t…

Reset CollectorRegistry of Prometheus lib after each unit test

I have a class A that initializes a Counter in its initfrom prometheus_client import Counter class A:def __init__(self):self.my_counter = Counter(an_awesome_counter)def method_1(self):return 1def metho…

Why does Django ORM allow me to omit parameters for NOT NULL fields when creating an object?

Stupid question time. I seem to be able to create an object for a Django model even though I omit a column that was defined as NOT NULL and I dont understand why. Heres my model:class Movie(models.Mo…

Where should i do the django validations for objects and fields?

Im creating a django application which uses both the Django Rest Framework and the plain django-views as entrypoint for users.I want to do validation both independant fields of my models, and on object…

Why does bytes.fromhex() treat some hex values strangely?

Im trying to use the socket library in Python to send bytes of two hex digits to a piece of hardware programmed to accept them. To create the bytes from a user-entered string of hex digits, Im trying …

Extracting beats out of MP3 music with Python

What kind of solutions are there to analyze beats out of MP3 music in Python? The purpose of this would be to use rhythm information to time the keyframes of generated animation, export animation as v…