Should I preallocate a numpy array?

2024/10/5 7:17:58

I have a class and its method. The method repeats many times during execution. This method uses a numpy array as a temporary buffer. I don't need to store values inside the buffer between method calls. Should I create a member instance of the array to avoid time leaks on memory allocation during the method execution? I know that it is preferred to use local variables. But is Python smart enough to allocate memory for the array only once?

class MyClass:def __init__(self, n):self.temp = numpy.zeros(n)def method(self):# do some stuff using self.temp

Or

class MyClass:def __init__(self, n):self.n = ndef method(self):temp = numpy.zeros(self.n)# do some stuff using temp

Update: replaced np.empty with np.zeros

Answer

Numpy arrays are fast, once created. However, creating an array is pretty expensive. Much more than, say, creating a python list.

In a case such as yours, where you create a new array again and again (in a for loop?), I would ALWAYS pre-allocate the array structure and then reuse it.

I can't comment on whether Python is smart enough to optimize this, but I would guess it's not :)

How big is your array and how frequent are calls to this method?

https://en.xdnf.cn/q/70515.html

Related Q&A

dup, dup2, tmpfile and stdout in python

This is a follow up question from here.Where I want do go I would like to be able to temporarily redirect the stdout into a temp file, while python still is able to print to stdout. This would involve …

How do I write a Class-Based Django Validator?

Im using Django 1.8.The documentation on writing validators has an example of a function-based validator. It also says the following on using a class:You can also use a class with a __call__() method f…

python synthesize midi with fluidsynth

I cant import fluidsynth. [Maybe theres an better module?]Im trying to synthesize midi from python or pygame. I can send midi events from pygame. Im using mingus, and it seemed pyfluidsynth would be g…

Convenient way to handle deeply nested dictionary in Python

I have a deeply nested dictionary in python thats taking up a lot of room. Is there a way to abbreviate something like this master_dictionary[sub_categories][sub_cat_name][attributes][attribute_name][s…

Running tests against existing database using pytest-django

Does anybody know how to run Django Tests using pytest-django against an existing (e.g. production) database? I know that in general, this is not what unit tests are supposed to do, but in my case, I…

Python, Timeout of Try, Except Statement after X number of seconds?

Ive been searching on this but cant seem to find an exact answer (most get into more complicated things like multithreading, etc), I just want to do something like a Try, Except statement where if the …

MemoryError while pickling data in python

I am trying to dump a dictionary into pickle format, using dump command provided in python. The file size of the dictionary is around 150 mb, but an exception occurs when only 115 mb of the file is dum…

numpy dimensions

Im a newbie to Numpy and trying to understand the basic question of what is dimension,I tried the following commands and trying to understand why the ndim for last 2 arrays are same?>>> a= ar…

DataFrame Plot: how to sort X axis

I am plotting some counts from a field of dataframe (pandas) and I found that the X axis is sorted by the counts (descending order). Instead is it possible to sort by the alphabetical order of the fiel…

In-place custom object unpacking different behavior with __getitem__ python 3.5 vs python 3.6

a follow-up question on this question: i ran the code below on python 3.5 and python 3.6 - with very different results:class Container:KEYS = (a, b, c)def __init__(self, a=None, b=None, c=None):self.a …