Python list.clear complexity [duplicate]

2024/11/15 17:22:09

What is the complexity of the Python 3 method list.clear() ?

  • It is not given here: https://wiki.python.org/moin/TimeComplexity

  • In the documentation it is said to be equivalent with del a[:], but I do not know the complexity of this function itself. Is it O(n) or O(1) ?

  • I took a look in listobject.c. Found this.

    int
    PyList_ClearFreeList(void)
    {PyListObject *op;int ret = numfree;while (numfree) {op = free_list[--numfree];assert(PyList_CheckExact(op));PyObject_GC_Del(op);}return ret;
    }
    

    Here it seems like O(n), but I am not sure if this is the right code.

I am developing a program with performance needs, where a list is repeatedly filled and emptied, I am trying to find the best way to empty it (Since there is only one way to fill it).

If this function is O(n), I will just create a new list every time, which has it's own cost, but I don't know a better way.

Another issue crossed my mind is that Python has a garbage collector, so if I don't free these objects(create new lists every time, leaving the other unattended by reassigning the variable name), Python does the deletion in the background(I am not sure about this information), so I won't gain speed applying any of the methods above because result is the same.

Any knowledge is appreciated. Thanks.

Answer

The function that you found is not related to list.clear() in Python. What you need is _list_clear(PyListObject *a), and it can be found here.

So, if you look into an implementation of that method, it looks as follows:

...
static int
_list_clear(PyListObject *a)
{Py_ssize_t i;PyObject **item = a->ob_item;if (item != NULL) {/* Because XDECREF can recursively invoke operations onthis list, we make it empty first. */i = Py_SIZE(a);Py_SIZE(a) = 0;a->ob_item = NULL;a->allocated = 0;while (--i >= 0) {Py_XDECREF(item[i]);}PyMem_FREE(item);}/* Never fails; the return value can be ignored.Note that there is no guarantee that the list is actually emptyat this point, because XDECREF may have populated it again! */return 0;
}
...

However, the most important lines are one, in which you're retrieving a size of a list:

i = Py_SIZE(a);

And ones, in which you're removing an element:

...while (--i >= 0) {Py_XDECREF(item[i]);}
...

As performance of Py_XDECREF doesn't depend on the size of the list, we can consider it constant or O(1). Since Py_XDECREF is called size of list times, the overall time complexity is linear and so the time complexity of _list_clear is O(n).

As pointed out by @superb rain, Py_XDECREF may turn quite "heavy" for some elements (due to possible recursive calls), and although it's not directly related to the input size, we can account for this by introducing a parameter e - the total cost of decreasing the reference counts of the elements. In this interpretation, the total time complexity is O(n+e).

https://en.xdnf.cn/q/71761.html

Related Q&A

Unresolved import org.python / working with jython and java?

Im using Eclipse and I"m trying to create a java program that can run my python code. Im following the guidelines on this page: http://jythonpodcast.hostjava.net/jythonbook/en/1.0/JythonAndJavaInt…

elegant unpacking variable-length tuples

A real, if silly problem:https://github.com/joshmarshall/tornadorpc/blob/master/tornadorpc/base.pydef start_server(handlers, ...):...for (route, handler) in handlers:...Normally "handlers" is…

Extracting data from a 3D scatter plot in matplotlib

Im writing an interface for making 3D scatter plots in matplotlib, and Id like to access the data from a python script. For a 2D scatter plot, I know the process would be:import numpy as np from matpl…

How does Pythonic garbage collection with numpy array appends and deletes?

I am trying to adapt the underlying structure of plotting code (matplotlib) that is updated on a timer to go from using Python lists for the plot data to using numpy arrays. I want to be able to lower …

What is the default if I install virtualenv using pip and pip3 respectively?

I used sudo pip install virtualenv, then when I run virtualenv ENV in a directory, I get a Python 2 virtual enviroment.If I use pip3 install virtualenv to install virtualenv again, will it override the…

Flask Admin ModelView different fields between CREATE and EDIT

In a Flask app using Flask Admin I would like to be able to define different form fields in the Edit section of a ModelView than those in the Create section.The form_columns setting applies to both Cre…

Python compilation error: LONG_BIT definition appears wrong for platform

This error message is not unknown, I have already reinstalled many packages, but so far not yet found a solution.I get the following error from the command pip install cryptography/usr/include/python2.…

Randomly sampling lines from a file

I have a csv file which is ~40gb and 1800000 lines. I want to randomly sample 10,000 lines and print them to a new file.Right now, my approach is to use sed as:(sed -n $vars < input.txt) > output…

Is Pythons hashlib.sha256(x).hexdigest() equivalent to Rs digest(x,algo=sha256)

Im not a python programmer, but Im trying to translate some Python code to R. The piece of python code Im having trouble with is:hashlib.sha256(x).hexdigest()My interpretation of this code is that the…

How to do Histogram Equalization on specific area

I have a image and I want to do HE or CLAHE on specific area of the image. I already have a mask for the image. Is there any possible way to do so?