Is there any difference — performance or otherwise — between generator expressions and generator functions?
In [1]: def f():...: yield from range(4)...:In [2]: def g():...: return (i for i in range(4))...:In [3]: f()
Out[3]: <generator object f at 0x109902550>In [4]: list(f())
Out[4]: [0, 1, 2, 3]In [5]: list(g())
Out[5]: [0, 1, 2, 3]In [6]: g()
Out[6]: <generator object <genexpr> at 0x1099056e0>
I'm asking because I want to decide how I should decide between using the two. Sometimes generator functions are clearer and then the choice is clear. I am asking about those times when code clarity does not make one choice obvious.
The functions you provided have completely different semantics in the general case.
The first one, with yield from, passes the control to the iterable. This means that calls to send() and throw() during the iteration will be handled by the iterable and not by the function you are defining.
The second function only iterates over the elements of the iterable, and it will handle all the calls to send() and throw(). To see the difference check this code:
In [8]: def action():...: try:...: for el in range(4):...: yield el...: except ValueError:...: yield -1...: In [9]: def f():...: yield from action()...: In [10]: def g():...: return (el for el in action())...: In [11]: x = f()In [12]: next(x)
Out[12]: 0In [13]: x.throw(ValueError())
Out[13]: -1In [14]: next(x)
---------------------------------------------------------------------------
StopIteration Traceback (most recent call last)
<ipython-input-14-5e4e57af3a97> in <module>()
----> 1 next(x)StopIteration: In [15]: x = g()In [16]: next(x)
Out[16]: 0In [17]: x.throw(ValueError())
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-17-1006c792356f> in <module>()
----> 1 x.throw(ValueError())<ipython-input-10-f156e9011f2f> in <genexpr>(.0)1 def g():
----> 2 return (el for el in action())3 ValueError:
In fact, due to this reason, yield from probably has a higher overhead than the genexp, even though it is probably irrelevant.
Use yield from only when the above behaviour is what you want or if you are iterating over a simple iterable that is not a generator (so that yield from is equivalent to a loop + simple yields).
Stylistically speaking I'd prefer:
def h():for el in range(4):yield el
Instead of returning a genexp or using yield from when dealing with generators.
In fact the code used by the generator to perform the iteration is almost identical to the above function:
In [22]: dis.dis((i for i in range(4)).gi_code)1 0 LOAD_FAST 0 (.0)>> 3 FOR_ITER 11 (to 17)6 STORE_FAST 1 (i)9 LOAD_FAST 1 (i)12 YIELD_VALUE13 POP_TOP14 JUMP_ABSOLUTE 3>> 17 LOAD_CONST 0 (None)20 RETURN_VALUE
As you can see it does a FOR_ITER + YIELD_VALUE. note that the argument (.0), is iter(range(4)). The bytecode of the function also contains the calls to LOAD_GLOBAL and GET_ITER that are required to lookup range and obtain its iterable. However this actions must be performed by the genexp too, just not inside its code but before calling it.