Given a pickle dump in python how to I determine the used protocol?

2024/10/6 20:31:47

Assume that I have a pickle dump - either as a file or just as a string - how can I determine the protocol that was used to create the pickle dump automatically?

And if so, do I need to read the entire dump to figure out the protocol or can this be achieved in O(1)? By O(1) I think about some header information at the beginning of the pickle string or file whose read out does not require processing the whole dump.

Thanks a lot!

EDIT: I have an update on this, apparently the answer given below does not always work under python 3.4. If I simply pickle the value True with protocol 1, sometimes I can only recover protocol 0 :-/

Answer

You could roll your own using picketools:

with open('your_pickle_file', 'rb') as fin:op, fst, snd = next(pickletools.genops(fin))proto = op.proto

It appears that a PROTO marker is only written as the first element where the protocol is 2 or greater. Otherwise, the first element is a marker or element that indicates if the protocol is 0 or 1.

Update into kludging even more land:

pops = pickletools.genops(pickle_source)
proto = 2 if next(pops)[0].proto == 2 else int(any(op.proto for op, fst, snd in pops))
https://en.xdnf.cn/q/70326.html

Related Q&A

Get First element by the recent date of each group

I have following model in djangoBusiness ID Business Name Business Revenue DateHere is the sample data:Business ID | Business Name | Business Revenue | Date 1 B1 1000 …

remote: ImportError: No module named gitlab

I wrote gitlab hook with python. And added to post-receive hooks in gitlab server. When i push to remote origin server from my laptop, i get following error. But it works when i run script manually in …

Using an Access database (.mdb) with Python on Ubuntu [duplicate]

This question already has answers here:Working with an Access database in Python on non-Windows platform (Linux or Mac)(5 answers)Closed 7 years ago.Im trying to use pyodbc to access a .mdb on Ubuntu. …

Pandas Grouper by weekday?

I have a pandas dataframe where the index is the date, from year 2007 to 2017.Id like to calculate the mean of each weekday for each year. I am able to group by year: groups = df.groupby(TimeGrouper(A)…

Can I move the pygame game window around the screen (pygame)

In the game Im making, Im trying to move the window around the screen for a mini game (dont ask) and Ive tried what I saw own threads and only found 1x = 100 y = 0 import os os.environ[SDL_VIDEO_WINDOW…

mocking a function within a class method

I want to mock a function which is called within a class method while testing the class method in a Django project. Consider the following structure: app/utils.py def func():...return resp # outcome i…

After resizing an image with cv2, how to get the new bounding box coordinate

I have an image of size 720 x 1280, and I can resize it to 256 x 256 like thisimport cv2 img = cv2.imread(sample_img.jpg) img_small = cv2.resize(img, (256, 256), interpolation=cv2.INTER_CUBIC)Say I hav…

convert a tsv file to xls/xlsx using python

I want to convert a file in tsv format to xls/xlsx..I tried usingos.rename("sample.tsv","sample.xlsx")But the file getting converted is corrupted. Is there any other method of doing…

How do you edit cells in a sparse matrix using scipy?

Im trying to manipulate some data in a sparse matrix. Once Ive created one, how do I add / alter / update values in it? This seems very basic, but I cant find it in the documentation for the sparse ma…

AttributeError: DataFrame object has no attribute _data

Azure Databricks execution error while parallelizing on pandas dataframe. The code is able to create RDD but breaks at the time of performing .collect() setup: import pandas as pd # initialize list of …