Python: efficiently check if integer is within *many* ranges

2024/11/19 13:30:19

I am working on a postage application which is required to check an integer postcode against a number of postcode ranges, and return a different code based on which range the postcode matches against.

Each code has more than one postcode range. For example, the M code should be returned if the postcode is within the ranges 1000-2429, 2545-2575, 2640-2686 or is equal to 2890.

I could write this as:

if 1000 <= postcode <= 2429 or 2545 <= postcode <= 2575 or 2640 <= postcode <= 2686 or postcode == 2890:return 'M'

but this seems like a lot of lines of code, given that there are 27 returnable codes and 77 total ranges to check against. Is there a more efficient (and preferably more concise) method of matching an integer to all these ranges using Python?


Edit: There's a lot of excellent solutions flying around, so I have implemented all the ones that I could, and benchmarked their performances.

The environment for this program is a web service (Django-powered actually) which needs to check postcode region codes one-by-one, on the fly. My preferred implementation, then, would be one that can be quickly used for each request, and does not need any process to be kept in memory, or needs to process many postcodes in bulk.

I tested the following solutions using timeit.Timer with default 1000000 repetitions using randomly generated postcodes each time.

IF solution (my original)

if 1000 <= postcode <= 2249 or 2555 <= postcode <= 2574 or ...:return 'M'
if 2250 <= postcode <= 2265 or ...:return 'N'
...

Time for 1m reps: 5.11 seconds.

Ranges in tuples (Jeff Mercado)

Somewhat more elegant to my mind and certainly easier to enter and read the ranges. Particularly good if they change over time, which is possible. But it did end up four times slower in my implementation.

if any(lower <= postcode <= upper for (lower, upper) in [(1000, 2249), (2555, 2574), ...]):return 'M'
if any(lower <= postcode <= upper for (lower, upper) in [(2250, 2265), ...]):return 'N'
...

Time for 1m reps: 19.61 seconds.

Set membership (gnibbler)

As stated by the author, "it's only better if you are building the set once to check against many postcodes in a loop". But I thought I would test it anyway to see.

if postcode in set(chain(*(xrange(start, end+1) for start, end in ((1000, 2249), (2555, 2574), ...)))):return 'M'
if postcode in set(chain(*(xrange(start, end+1) for start, end in ((2250, 2265), ...)))):return 'N'
...

Time for 1m reps: 339.35 seconds.

Bisect (robert king)

This one may have been a bit above my intellect level. I learnt a lot reading about the bisect module but just couldn't quite work out which parameters to give find_ge() to make a runnable test. I expect that it would be extremely fast with a loop of many postcodes, but not if it had to do the setup each time. So, with 1m repetitions of filling numbers, edgepairs, edgeanswers etc for just one postal region code (the M code with four ranges), but not actually running the fast_solver:

Time for 1m reps: 105.61 seconds.

Dict (sentinel)

Using one dict per postal region code pre-generated, cPickled in a source file (106 KB), and loaded for each run. I was expecting much better performance from this method, but on my system at least, the IO really destroyed it. The server is a not-quite-blindingly-fast-top-of-the-line Mac Mini.

Time for 1m reps: 5895.18 seconds (extrapolated from a 10,000 run).

The summary

Well, I was expecting someone to just give a simple 'duh' answer that I hadn't considered, but it turns out this is much more complicated (and even a little controversial).

If every nanosecond of efficiency counted in this case, I would probably keep a separate process running which implemented one of the binary search or dict solutions and kept the result in memory for an extremely fast lookup. However, since the IF tree takes only five seconds to run a million times, which is plenty fast enough for my small business, that's what I'll end up using in my application.

Thank you to everyone for contributing!

Answer

You can throw your ranges into tuples and put the tuples in a list. Then use any() to help you find if your value is within these ranges.

ranges = [(1000,2429), (2545,2575), (2640,2686), (2890, 2890)]
if any(lower <= postcode <= upper for (lower, upper) in ranges):print('M')
https://en.xdnf.cn/q/26431.html

Related Q&A

How to pass on argparse argument to function as kwargs?

I have a class defined as followsclass M(object):def __init__(self, **kwargs):...do_somethingand I have the result of argparse.parse_args(), for example:> args = parse_args() > print args Namespa…

How do you check whether a python method is bound or not?

Given a reference to a method, is there a way to check whether the method is bound to an object or not? Can you also access the instance that its bound to?

Most Pythonic way to declare an abstract class property

Assume youre writing an abstract class and one or more of its non-abstract class methods require the concrete class to have a specific class attribute; e.g., if instances of each concrete class can be …

PyQt on Android

Im working on PyQt now, and I have to create the application on Android, Ive seen the kivy library, but its too crude.Is there any way now to run an application on Android made on PyQt?

How to elementwise-multiply a scipy.sparse matrix by a broadcasted dense 1d array?

Suppose I have a 2d sparse array. In my real usecase both the number of rows and columns are much bigger (say 20000 and 50000) hence it cannot fit in memory when a dense representation is used:>>…

Do I have to do StringIO.close()?

Some code:import cStringIOdef f():buffer = cStringIO.StringIO()buffer.write(something)return buffer.getvalue()The documentation says:StringIO.close(): Free the memory buffer. Attempting to do furtherop…

Python: ulimit and nice for subprocess.call / subprocess.Popen?

I need to limit the amount of time and cpu taken by external command line apps I spawn from a python process using subprocess.call , mainly because sometimes the spawned process gets stuck and pins the…

array.shape() giving error tuple not callable

I have a 2D numpy array called results, which contains its own array of data, and I want to go into it and use each list: for r in results:print "r:"print ry_pred = np.array(r)print y_pred.sh…

Python unittest - Ran 0 tests in 0.000s

So I want to do this code Kata for practice. I want to implement the kata with tdd in separate files:The algorithm:# stringcalculator.py def Add(string):return 1and the tests:# stringcalculator.spec.…

How and where does py.test find fixtures

Where and how does py.test look for fixtures? I have the same code in 2 files in the same folder. When I delete conftest.py, cmdopt cannot be found running test_conf.py (also in same fo…