Optimizing loop. Faster ResultList.append( [ c, d, c[1]/d[1]] )? Array? Map?

2024/11/19 3:20:37

The following works well but I'd like to make it faster. The actual application could process Tuple1 and Tuple2 each with 30,000 elements and 17 nested sequences per element. I see numerous questions about faster loops and have tried an array and map without improvement.

for i in Tuple1:print i((1, 2.2, 3), (2, 3.3, 4))
((5, 6.6, 7), (6, 7.7, 8))for i in Tuple2:print i((10, 11, 12), (11, 12, 13), (12, 13, 14))
((20, 21, 22), (21, 22, 23), (22, 23, 24))ResultList = []for a in Tuple1:for b in Tuple2:for c in a:for d in b:ResultList.append( [c, d, c[1]/d[1]] )SomeFunction() # Processes ResultList and is not a concern.ResultList=[]

An example of a ResultList processed by SomeFunction.

[(1, 2.2, 3), (10, 11, 12), 0.2]
[(1, 2.2, 3), (11, 12, 13), 0.18333333333333335]
[(1, 2.2, 3), (12, 13, 14), 0.16923076923076924]
[(2, 3.3, 4), (10, 11, 12), 0.3]
[(2, 3.3, 4), (11, 12, 13), 0.27499999999999997]
[(2, 3.3, 4), (12, 13, 14), 0.25384615384615383]
Answer

Simply switching from an explicit for loop to a list comprehension and using a tuple instead of a list can significantly speed up your operation:

from itertools import productResultList = [(c, d, c[1] / d[1]) for a, b in product(t1, t2) for c, d in product(a, b)]
#             ^Use a tuple here instead of a list

As can be shown by the timeit trial below:

>>> from timeit import Timer
>>> original_test = Timer('original_version(Tuple1, Tuple2)', '''\
... Tuple1 = (
...     ((1, 2.2, 3), (2, 3.3, 4)), 
...     ((5, 6.6, 7), (6, 7.7, 8))
... )
... Tuple2 = (
...     ((10, 11, 12), (11, 12, 13), (12, 13, 14)), 
...     ((20, 21, 22), (21, 22, 23), (22, 23, 24))
... )
... def original_version(t1, t2):
...     ResultList = []
...     for a in t1:
...         for b in t2:
...             for c in a:
...                 for d in b:
...                     ResultList.append([c, d, c[1] / d[1]])''')
>>> improved_test = Timer('improved_version(Tuple1, Tuple2)', '''\
... from itertools import product
... Tuple1 = (
...     ((1, 2.2, 3), (2, 3.3, 4)), 
...     ((5, 6.6, 7), (6, 7.7, 8))
... )
... Tuple2 = (
...     ((10, 11, 12), (11, 12, 13), (12, 13, 14)), 
...     ((20, 21, 22), (21, 22, 23), (22, 23, 24))
... )
... def improved_version(t1, t2):
...     return [(c, d, c[1] / d[1]) for a, b in product(t1, t2) for c, d in product(a, b)]''')
>>> original_time = original_test.timeit()
>>> improved_time = improved_test.timeit()
>>> print 'Improved version is %{} faster'.format(
...     (original_time - improved_time) / original_time * 100
... )
Improved version is %30.0181954314 faster
>>> 
https://en.xdnf.cn/q/118594.html

Related Q&A

Why do I get an error name play is not defined when I think it is?

Full error: line 10, in <module>colour = play() NameError: name play is not definedI cant seem to find a reason for this issue anywhere on here. I am trying to assign the returned string to the v…

Error: unhashable type: dict

i have a problem with Django: I cant show the data from mysql database in the table. I see the error "Exception Value: unhashable type: dict" This is my code: views.py:List_of_date=El.objects…

terminal command line python3.3

Im following a book tutorial and its telling me to install python3.3 with the command linesudo apt-get install python3.3however Im getting errorsUnable to locate package python3.3 Couldnt find any pack…

SQLalchemy making errors after being updated to 1.4.0 [duplicate]

This question already has answers here:ImportError: cannot import name _ColumnEntity from sqlalchemy.orm.query(5 answers)ImportError: cannot import name _ColumnEntity Ubuntu20.10 [duplicate](1 answer)C…

Python string formatting with percentage (TypeError: not enough arguments for format string)

The following code fails to run.It goes through a CSV file and retrieves the values and formats them in a array of tuples (a insert query) to be used later. Problem is the csv last column is sometimes …

Circles touching edges

I am struggling with a program to tell whether a created disk touches the edge of a predefined box. The parser keeps saying things such asNameError: global name disksdescription is not defined Warning…

How to split data from a merged cell into other cells in its same row of a Python data frame?

I have a sample of a data frame which looks like this: +---+--------------------------------------------------------------------------------------+---------------+--------------------------------------…

Collect data in chunks from stdin: Python

I have the following Python code where I collect data from standard input into a list and run syntaxnet on it. The data is in the form of json objects from which I will extract the text field and feed …

Getting and calculating stuff through tkinter widets

I was wondering how to calculate stuff using tkinter buttons. Im making a simple program to calculate seconds to hours:minutes:seconds. The user inputs an integer using the entry widget on the seconds …

Why does this condition execute to false when it should execute to true?

I have this code in my spider basic.py file:if l.add_xpath(price, //*[@id="price"]/text(),MapCompose(lambda i: i.replace(,, ), float),re = [,.0-9]):l.add_value(available, 1) else:l.add_value(…