Store the cache to a file functools.lru_cache in Python = 3.2

2024/11/19 5:29:03

I'm using @functools.lru_cache in Python 3.3. I would like to save the cache to a file, in order to restore it when the program will be restarted. How could I do?

Edit 1 Possible solution: We need to pickle any sort of callable

Problem pickling __closure__:

_pickle.PicklingError: Can't pickle <class 'cell'>: attribute lookup builtins.cell failed

If I try to restore the function without it, I get:

TypeError: arg 5 (closure) must be tuple
Answer

You can't do what you want using lru_cache, since it doesn't provide an API to access the cache, and it might be rewritten in C in future releases. If you really want to save the cache you have to use a different solution that gives you access to the cache.

It's simple enough to write a cache yourself. For example:

from functools import wrapsdef cached(func):func.cache = {}@wraps(func)def wrapper(*args):try:return func.cache[args]except KeyError:func.cache[args] = result = func(*args)return result   return wrapper

You can then apply it as a decorator:

>>> @cached
... def fibonacci(n):
...     if n < 2:
...             return n
...     return fibonacci(n-1) + fibonacci(n-2)
... 
>>> fibonacci(100)
354224848179261915075L

And retrieve the cache:

>>> fibonacci.cache
{(32,): 2178309, (23,): 28657, ... }

You can then pickle/unpickle the cache as you please and load it with:

fibonacci.cache = pickle.load(cache_file_object)

I found a feature request in python's issue tracker to add dumps/loads to lru_cache, but it wasn't accepted/implemented. Maybe in the future it will be possible to have built-in support for these operations via lru_cache.

https://en.xdnf.cn/q/26478.html

Related Q&A

Moon / Lunar Phase Algorithm

Does anyone know an algorithm to either calculate the moon phase or age on a given date or find the dates for new/full moons in a given year?Googling tells me the answer is in some Astronomy book, but…

Flask-RESTful API: multiple and complex endpoints

In my Flask-RESTful API, imagine I have two objects, users and cities. It is a 1-to-many relationship. Now when I create my API and add resources to it, all I can seem to do is map very easy and genera…

Setting initial Django form field value in the __init__ method

Django 1.6I have a working block of code in a Django form class as shown below. The data set from which Im building the form field list can include an initial value for any of the fields, and Im having…

Should I use a main() method in a simple Python script?

I have a lot of simple scripts that calculate some stuff or so. They consist of just a single module.Should I write main methods for them and call them with the if __name__ construct, or just dump it a…

Where can I find mad (mean absolute deviation) in scipy?

It seems scipy once provided a function mad to calculate the mean absolute deviation for a set of numbers:http://projects.scipy.org/scipy/browser/trunk/scipy/stats/models/utils.py?rev=3473However, I c…

Map of all points below a certain time of travel?

My question is very simple and can be understood in one line:Is there a way, tool, etc. using Google Maps to get an overlay of all surface which is below a certain time of travel?I hope the question i…

Pandas populate new dataframe column based on matching columns in another dataframe

I have a df which contains my main data which has one million rows. My main data also has 30 columns. Now I want to add another column to my df called category. The category is a column in df2 which co…

Remove an imported python module [duplicate]

This question already has answers here:Closed 11 years ago.Possible Duplicate:Unload a module in Python After importing Numpy, lets say I want to delete/remove numpy import referenceimport sys import…

Should I create each class in its own .py file?

Im quite new to Python in general.Im aware that I can create multiple classes in the same .py file, but Im wondering if I should create each class in its own .py file.In C# for instance, I would have a…

Should you put quotes around type annotations in python

Whats the difference between these two functions? Ive seen people put quotes around type annotations and other times leave them out but I couldnt find why people choose to use one or the other.def do_…