How to use np.empty inside numba compiled function; Error message All templates rejected

2024/9/8 9:57:59

I ran into this weird error when trying to use np.empty in a function definition compiled with numba, and turning on nopython=True to make sure optimized typing is in effect.

It's weird because numba claims to support np.empty with the first two arguments, and I am only using the first two arguments (correctly I think?), so I don't know why it's not typing correctly.

@jit(nopython=True)
def empty():return np.empty(5, np.float)

After defining the above function in an ipython notebook,

empty()

Gives the following error message:

---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
<ipython-input-88-927345c8757f> in <module>()
----> 1 empty()~/.../lib/python3.5/site-packages/numba/dispatcher.py in _compile_for_args(self, *args, **kws)342                 raise e343             else:
--> 344                 reraise(type(e), e, None)345         except errors.UnsupportedError as e:346             # Something unsupported is present in the user code, add help info~/.../lib/python3.5/site-packages/numba/six.py in reraise(tp, value, tb)656             value = tp()657         if value.__traceback__ is not tb:
--> 658             raise value.with_traceback(tb)659         raise value660 
TypingError: Failed at nopython (nopython frontend)
Invalid usage of Function(<built-in function empty>) with parameters (int64, Function(<class 'float'>))* parameterized
In definition 0:All templates rejected
[1] During: resolving callee type: Function(<built-in function empty>)
[2] During: typing of call at <ipython-input-87-8c7e8fa4c6eb> (3)File "<ipython-input-87-8c7e8fa4c6eb>", line 3:
def empty():return np.empty(5, np.float)^This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/dev/reference/numpysupported.htmlFor more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compileIf you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new
Answer

The problem is that np.float is not a valid datatype for a NumPy array in numba. You have to provide the explicit dtype to numba. This isn't just a problem with np.empty but also for other array-creation routines like np.ones, np.zeros, ... in numba.

To make your example work only a little change is needed:

from numba import jit
import numpy as np@jit(nopython=True)
def empty():return np.empty(5, np.float64)  # np.float64 instead of np.floatempty()

Or the shorthand np.float_. Or if you want 32 bit floats use np.float32 instead.


Note that np.float is just an alias for the normal Python float and as such not a real NumPy dtype:

>>> np.float is float
True
>>> issubclass(np.float, np.generic)
False
>>> issubclass(np.float64, np.generic)
True

Likewise there are some additional aliases that just are interpreted as if they were NumPy dtypes (source):

Built-in Python types

Several python types are equivalent to a corresponding array scalar when used to generate a dtype object:

int          int_
bool         bool_
float        float_
complex      cfloat
bytes        bytes_
str          bytes_ (Python2) or unicode_ (Python3)
unicode      unicode_
buffer       void
(all others) object_

However numba doesn't know about these aliases and even when not dealing with numba you are probably better off using the real dtypes directly:

Array types and conversions between types

NumPy supports a much greater variety of numerical types than Python does. This section shows which are available, and how to modify an array’s data-type.

Data type     Description
bool_         Boolean (True or False) stored as a byte
int_          Default integer type (same as C long; normally either int64 or int32)
intc          Identical to C int (normally int32 or int64)
intp          Integer used for indexing (same as C ssize_t; normally either int32 or int64)
int8          Byte (-128 to 127)
int16         Integer (-32768 to 32767)
int32         Integer (-2147483648 to 2147483647)
int64         Integer (-9223372036854775808 to 9223372036854775807)
uint8         Unsigned integer (0 to 255)
uint16        Unsigned integer (0 to 65535)
uint32        Unsigned integer (0 to 4294967295)
uint64        Unsigned integer (0 to 18446744073709551615)
float_        Shorthand for float64.
float16       Half precision float: sign bit, 5 bits exponent, 10 bits mantissa
float32       Single precision float: sign bit, 8 bits exponent, 23 bits mantissa
float64       Double precision float: sign bit, 11 bits exponent, 52 bits mantissa
complex_      Shorthand for complex128.
complex64     Complex number, represented by two 32-bit floats (real and imaginary components)
complex128    Complex number, represented by two 64-bit floats (real and imaginary components)

Note that some of these are not supported by numba!

https://en.xdnf.cn/q/73401.html

Related Q&A

Python ValueError: Invalid header name b:authority

I see the : is error, but I cant find a way to solve it. ValueError: Invalid header name b:authorityIts the error:File "tmall.py", line 23, in get_url response = sessions.get(url=url,headers…

Psycopg2 callproc and sql parameters

I got some SQL functionCREATE OR REPLACE FUNCTION tools.update_company(IN company_id integer, OUT value integer)RETURNS integer AS$BODY$BEGIN select * into value from function_making_int(company_id) E…

How to write Huffman coding to a file using Python?

I created a Python script to compress text by using the Huffman algorithm. Say I have the following string:string = The quick brown fox jumps over the lazy dogRunning my algorithm returns the following…

How to know if a Python multiprocessing.Lock is released or not?

>>> l = Lock() >>> l.acquire() True >>> l.release() >>> l.release() Traceback (most recent call last):File "<stdin>", line 1, in <module> Value…

How do you ensure a Celery chord callback gets called with failed subtasks?

I am using a Chord in Celery to have a callback that gets called when a Group of parallel tasks finish executing. Specifically, I have a group of functions that wrap calls to an external API. I want to…

Unpacking nested C structs in Python

I am trying to unpack a C struct that is handed to my Python program in binary form and includes another nested struct. The relevant part of the C header looks like this:typedef struct {uint8_t seq;uin…

Remove black borders on images with watermarks in Python

I have a bunch of image I would like to uniformise by removing black borders. Usually I use the Trim function of Imagemagick with the fuzz parameters but in the case the image have some watermark the r…

scipy cdist with sparse matrices

I need to calculate the distances between two sets of vectors, source_matrix and target_matrix.I have the following line, when both source_matrix and target_matrix are of type scipy.sparse.csr.csr_matr…

NumPy arrays with SQLite

The most common SQLite interface Ive seen in Python is sqlite3, but is there anything that works well with NumPy arrays or recarrays? By that I mean one that recognizes data types and does not requir…

Binary Phase Shift Keying in Python

Im currently working on some code to transmit messages/files/and other data over lasers using audio transformation. My current code uses the hexlify function from the binascii module in python to conve…