Reusing generator expressions

2024/10/4 1:25:59

Generator expressions is an extremely useful tool, and has a huge advantage over list comprehensions, which is the fact that it does not allocate memory for a new array.

The problem I am facing with generator expressions, which eventually makes me end up writing list comprehensions, is that I can only use a such a generator once:

>>> names = ['John', 'George', 'Paul', 'Ringo']
>>> has_o = (name for name in names if 'o' in name)
>>> for name in has_o:
...   print(name.upper())
...
JOHN
GEORGE
RINGO
>>> for name in has_o:
...   print(name.lower())
...
>>>

The above code illustrates how the generator expression can only be used once. That's of course, because the generator expression returns an instance of the generator, rather than defining a generator function which could be instantiated again and again.

Is there a way to clone the generator each time it is used, in order to make it reusable, or to make the generator expressions syntax return a generator function rather than a single instance?

Answer

Make it a lambda:

has_o = lambda names: (name for name in names if 'o' in name)
for name in has_o(["hello","rrrrr"]):print(name.upper())
for name in has_o(["hello","rrrrr"]):print(name.upper())

lambda is a one-liner and returns a new generator each time. Here I chose to be able to pass the input list, but if it's fixed, you don't even need a parameter:

names = ["hello","rrrrr"]
has_o = lambda: (name for name in names if 'o' in name)
for name in has_o():print(name.upper())
for name in has_o():print(name.upper())

In that last case, be careful about the fact that if names changes or is reassigned, the lambda uses the new names object. You can fix the name reassigning by using the default value trick:

has_o = lambda lst=names: (name for name in lst if 'o' in name)

and you can fix the afterwards modification of names by using the default value-and-copy trick (not super-useful when you think your first goal was to avoid a list to be created :)):

has_o = lambda lst=names[:]: (name for name in lst if 'o' in name)

(now make your pick :))

https://en.xdnf.cn/q/70667.html

Related Q&A

ModuleNotFoundError: No module named librosa

Currently I am working on voice recognition where I wanted to use Librosa library. I install librosa with the command on ubuntu: conda install -c conda-forge librosaBut when I run the code I got the fo…

Python - Convert Very Large (6.4GB) XML files to JSON

Essentially, I have a 6.4GB XML file that Id like to convert to JSON then save it to disk. Im currently running OSX 10.8.4 with an i7 2700k and 16GBs of ram, and running Python 64bit (double checked). …

Python create tree from a JSON file

Lets say that we have the following JSON file. For the sake of the example its emulated by a string. The string is the input and a Tree object should be the output. Ill be using the graphical notation …

disable `functools.lru_cache` from inside function

I want to have a function that can use functools.lru_cache, but not by default. I am looking for a way to use a function parameter that can be used to disable the lru_cache. Currently, I have a two ver…

How to clear tf.flags?

If I run this code twice:tf.flags.DEFINE_integer("batch_size", "2", "batch size for training")I will get this error:DuplicateFlagError: The flag batch_size is defined twic…

Stochastic Optimization in Python

I am trying to combine cvxopt (an optimization solver) and PyMC (a sampler) to solve convex stochastic optimization problems. For reference, installing both packages with pip is straightforward: pip in…

Pandas convert yearly to monthly

Im working on pulling financial data, in which some is formatted in yearly and other is monthly. My model will need all of it monthly, therefore I need that same yearly value repeated for each month. …

Firebase database data to R

I have a database in Google Firebase that has streaming sensor data. I have a Shiny app that needs to read this data and map the sensors and their values.I am trying to pull the data from Firebase into…

Django 1.8 Migrations - NoneType object has no attribute _meta

Attempting to migrate a project from Django 1.7 to 1.8. After wrestling with code errors, Im able to get migrations to run. However, when I try to migrate, Im given the error "NoneType object has …

Manage dependencies of git submodules with poetry

We have a repository app-lib that is used as sub-module in 4 other repos and in each I have to add all dependencies for the sub-module. So if I add/remove a dependency in app-lib I have to adjust all o…