Filter values inside Python generator expressions

2024/4/14 4:44:03

I have a dictionary dct for which I want each of its values to be summed provided their corresponding keys exist in a specified list lst.

The code I am using so far is:

sum(dct[k] for k in lst)

In the above generator expression I would like to handle the KeyError in case a key from the list is not found inside the dictionary. I cannot seem to find how to implement (syntax-wise) either a try-except approach, nor an if-else approach inside this generator expression.

In case a key from the list is not found inside the dictionary, then it should carry on getting the other values. The end result of the sums should not be affected by any missing keys. In case none of the keys exist, then zero should be the sum's result.


Well, there are few options, preferred one is to use dict.get():

# 1
sum(dct.get(k, 0) for k in lst)
# 2
sum(dct[k] for k in lst if k in dct)

Also one of the option is to filter lst before iteraring over it:

sum(dct[k] for k in filter(lambda i: i in dct, lst))

And you can use reduce function on filtered list as an alternative to sum:

reduce(lambda a, k: a + dct[k], filter(lambda i: i in dct, lst))

Now let's find fastest approach with timeit:

from timeit import timeit
import randomlst = range(0, 10000)
dct = {x:x for x in lst if random.choice([True, False])}via_sum = lambda:(sum(dct.get(k, 0) for k in lst))
print("Via sum and get: %s" % timeit(via_sum, number=10000))
# Via sum and get: 16.725695848464966via_sum_and_cond = lambda:(sum(dct[k] for k in lst if k in dct))
print("Via sum and condition: %s" % timeit(via_sum_and_cond, number=10000))
# Via sum and condition: 9.4715681076via_reduce = lambda:(reduce(lambda a, k: a + dct[k], filter(lambda i: i in dct, lst)))
print("Via reduce: %s" % timeit(via_reduce, number=10000))
# Via reduce: 19.9522120953

So the fastest option is to sum items via if statement within generator expression

sum(dct[k] for k in lst if k in dct) # Via sum and condition: 9.4715681076

Good Luck !

Related Q&A

Python and tfidf algorithm, make it faster?

I am implementing the tf-idf algorithm in a web application using Python, however it runs extremely slow. What I basically do is:1) Create 2 dictionaries:First dictionary: key (document id), value (lis…

How to use Python to find all isbn in a text file?

I have a text file text_isbn with loads of ISBN in it. I want to write a script to parse it and write it to a new text file with each ISBN number in a new line.Thus far I could write the regular expres…

AWS Batch Job Execution Results in Step Function

Im newbie to AWS Step Functions and AWS Batch. Im trying to integrate AWS Batch Job with Step Function. AWS Batch Job executes simple python scripts which output string value (High level simplified req…

Simplify Django test set up with mock objects

Often when Im writing tests for my Django project, I have to write a lot more code to set up database records than I do to actually test the object under test. Currently, I try to use test fixtures to …

How to use with a nested shape?

I am building a dataset with two tensors of shape [batch,width,heigh,3] and [batch,class] for each element. For simplicity lets say class = 5.What shape do you feed to dataset.padded_batch(1000,shape)…

Python, thread and gobject

I am writing a program by a framework using pygtk. The main program doing the following things:Create a watchdog thread to monitor some resource Create a client to receive data from socket call gobjec…

How to type annotate overrided methods in a subclass?

Say I already have a method with type annotations:class Shape:def area(self) -> float:raise NotImplementedErrorWhich I will then subclass multiple times:class Circle:def area(self) -> float:retur…

Import Error: No module named pytz after using easy_install

Today is my first day at Python and have been going through problems. One that I was working on was, "Write a short program which extracts the current date and time from the operating system and p…

Python catch exception pandas.errors.ParserError: Error tokenizing data. C error

I am facing problem with my malfunction csv input file whole reading and which i can deal with by adding "error_bad_lines=False" in my read_csv func to remove those.But i need to report these…

Nested tags in BeautifulSoup - Python

Ive looked at many examples on websites and on stackoverflow but I couldnt find a universal solution to my question. Im dealing with a really messy website and Id like to scrape some data. The markup l…