Python MemoryError on large array

2024/9/20 8:03:52

This is the python script that I'm trying to run:

n = 50000000000 ##50 billion 
b = [0]*n
for x in range(0,n):b[x] = random.randint(1,899999)

... But the output I'm getting is:

E:\python\> python sort.py
Traceback (most recent call last):File "E:\python\sort.py", line 8, in <module>b = [0]*n
MemoryError

So, what do I do now?

Answer

The size of the list you are generating (which is 50 billion not 5).

An int object instance takes 24 bytes (sys.getsizeof(int(899999)), the upper limit of your random numbers), so that list would take 50,000,000,000 * 24 bytes, which is about 1.09 TB.

In other words to create such a list you would need at least 1118 GB of RAM in your computer.

I don't know what your use case is, but you should consider a different approach to what you are trying to solve (maybe define a generator, or just don't store your numbers in memory and instead directly use the numbers in the for loop).

https://en.xdnf.cn/q/72525.html

Related Q&A

How to generate a PDF with non-ascii characters using from_string from python-pdfkit

Im struggling to generate just a simple PDF with non-ascii characters using Python 3.5.2, python-pdfkit and wkhtmltox-0.12.2.This is the easiest example I could write:import pdfkit html_content = u<…

Minimum window of days required to travel all cities

This is an interesting question that I came across in a coding challenge:There are k cities and n days. A travel agent is going to show you city k on day n. Youre supposed to find the minimum number of…

Force Nosetests to Use Python 2.7 instead of 3.4

Ive been learning Python using version 3.4. I recently started learning Web.py so have been using Python 2.7 for that, since web.py not supported in Python 3.4. I have nose 1.3.4 module installed for …

Python regex not to match http://

I am facing a problem to match and replace certain words, not contained in http:// Present Regex: http://.*?\s+This matches the pattern http://www.egg1.com http://www.egg2.com I need a regex to matc…

how can I maintain sequence of my list using set?

In [1]: l1 = [a,2,3,0,9.0,0,2,6,b,a]In [2]: l2 = list(set(l1))In [3]: l2 Out[3]: [a, 0, 2, 3, 6, 9.0, b]Here you can see the the list l2 is falling with different sequence then the original l1, I need …

How to reference a dict object?

I have a Python dict object d. d = {a: 1, b: 2, c: 3}. My problem is really simple. I want to reference a variable to the elements of d. For example, something like:In[1]: p = d[a] >>> p = 1 I…

Python 3.3: DeprecationWarning when using nose.tools.assert_equals

I am using nosetest tools for asserting a python unittest:... from nose.tools import assert_equals, assert_almost_equalclass TestPolycircles(unittest.TestCase):def setUp(self):self.latitude = 32.074322…

Panel/Hvplot interaction when variable is changing

Im trying to create a dashboard with two holoviews objects: a panel pn.widgets.Select object that contains a list of xarray variables, and a hvplot object that takes the selected variable on input, lik…

asyncio matplotlib show() still freezes program

I wish to run a simulation while at the same time output its progress in a plot. Ive been looking through a lot of examples of threading and multiprocessing, but they are all pretty complex. So I thoug…

celery .delay hangs (recent, not an auth problem)

I am running Celery 2.2.4/djCelery 2.2.4, using RabbitMQ 2.1.1 as a backend. I recently brought online two new celery servers -- I had been running 2 workers across two machines with a total of ~18 thr…