I made a binary search function and I'm curious what would happen if I used it on 4 billion numbers, but I get a MemoryError
every time I use it. Is there a way to store the list without this issue?
I made a binary search function and I'm curious what would happen if I used it on 4 billion numbers, but I get a MemoryError
every time I use it. Is there a way to store the list without this issue?
Yes, but it's gonna be expensive.
Assuming the numbers are 1-4,000,000,000, each number would take up either 28 bytes or 32 bytes. Going with 32 bytes, storing 4,000,000,000 numbers would take ~128GB. The list itself also takes up memory, 8 bytes per object (source), so it would require around 192GB of memory altogether.
If you're wanting to emulate a list, however, things become a lot more reasonable.
If your numbers follow some formula, you can make a class that "pretends" to be a list by following the answers here. This will create a custom class that works like a list, (e.g. you can do foo[3]
on it and it will return a number) but doesn't take up all that memory because it isn't actually storing 4,000,000,000 numbers. In fact, this is pretty similar to how range()
works, and is the reason why doing range(1,4_000_000_000)
doesn't take up 192GB of ram.