When ruamel.yaml loads @dataclass from string, __post_init__ is not called

2024/10/7 18:26:53

Assume I created a @dataclass class Foo, and added a __post_init__ to perform type checking and processing.

When I attempt to yaml.load a !Foo object, __post_init__ is not called.

from dataclasses import dataclass, fieldsfrom ruamel.yaml import yaml_object, YAMLyaml = YAML()@yaml_object(yaml)
@dataclass
class Foo:foo: intbar: intdef __post_init__(self):raise Exceptionfor field in fields(self):value = getattr(self, field.name)typ = field.typeif not isinstance(value, typ):raise Exceptions = '''\
!Foo
foo: "foo"
bar: "bar"
'''
yaml.load(s)

How do I perform parameter checking when loading dataclasses via ruamel.yaml?

This behavior occurs in Python 3.7 as well as 3.6 with pip install dataclasses.

Answer

This is now supported in 0.17.34, where you can do

from dataclasses import dataclass
from ruamel.yaml import YAMLyaml = ruamel.yaml.YAML()@yaml.register_class
@dataclass
class Foo:....

The reason why __post_init__ is not called, is because ruamel.yaml (and the PyYAML code in its Constructors), was created long before dataclasses was created.

Of course code for making a call to __post_init_() could be added to ruamel.yaml's Python object constructors, preferably after a test if something was created using @dataclass, as otherwise a non Data-Class class, that happens to have such a method named __post_init_, will all of a sudden have that method called during loading.

If you have no such classes, you can add your own, smarter, constructor to the YAML() instance before first loading/dumping (at which moment the constructor is instantiated) using yaml.Constructor = MyConstructor. But adding a constructor is not as trivial as subclassing the RoundTripConstructor, because all supported node types need to be registered on such a new constructor type.

Most of the time I find it easier to just patch the appropriate method on the RoundTripConstructor:

from dataclasses import dataclass, fields
from ruamel.yaml import yaml_object, YAML, RoundTripConstructordef my_construct_yaml_object(self, node, cls):for data in self.org_construct_yaml_object(node, cls):yield data# not doing a try-except, in case `__post_init__` does catch the AttributeErrorpost_init = getattr(data, '__post_init__', None)if post_init:post_init()RoundTripConstructor.org_construct_yaml_object = RoundTripConstructor.construct_yaml_object
RoundTripConstructor.construct_yaml_object = my_construct_yaml_objectyaml = YAML()
yaml.preserve_quotes = True@yaml_object(yaml)
@dataclass
class Foo:foo: intbar: intdef __post_init__(self):for field in fields(self):value = getattr(self, field.name)typ = field.typeif not isinstance(value, typ):raise Exceptions = '''\
!Foo
foo: "foo"
bar: "bar"
'''
d = yaml.load(s)

throws an exception:

Traceback (most recent call last):File "try.py", line 36, in <module>d = yaml.load(s)File "/home/venv/tmp-46489abf428c4cd4/lib/python3.7/site-packages/ruamel/yaml/main.py", line 266, in loadreturn constructor.get_single_data()File "/home/venv/tmp-46489abf428c4cd4/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 105, in get_single_datareturn self.construct_document(node)File "/home/venv/tmp-46489abf428c4cd4/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 115, in construct_documentfor dummy in generator:File "try.py", line 10, in my_construct_yaml_objectpost_init()File "try.py", line 29, in __post_init__raise Exception
Exception

Please note that the double quotes in your YAML are superfluous, so if you want to preserve these on round-trip you need to do yaml.preserve_quotes = True

https://en.xdnf.cn/q/70210.html

Related Q&A

How is the python module search path determined on Mac OS X?

When a non built-in module is imported, the interpreter searches in the locations given by sys.path. sys.path is initialized from these locations (http://docs.python.org/library/sys.html#sys.path):the …

Apply Mask Array 2d to 3d

I want to apply a mask of 2 dimensions (an NxM array) to a 3 dimensional array (a KxNxM array). How can I do this?2d = lat x lon 3d = time x lat x lonimport numpy as npa = np.array([[[ 0, 1, 2],[ 3,…

Server Side Google Markers Clustering - Python/Django

After experimenting with client side approach to clustering large numbers of Google markers I decided that it wont be possible for my project (social network with 28,000+ users).Are there any examples …

Tensorflow numpy image reshape [grayscale images]

I am trying to execute the Tensorflow "object_detection_tutorial.py" in jupyter notebook, with my trained neural network data but it throws a ValueError. The file mentioned above is part of S…

Merge numpy arrays returned from loop

I have a loop that generates numpy arrays:for x in range(0, 1000):myArray = myFunction(x)The returned array is always one dimensional. I want to combine all the arrays into one array (also one dimensio…

Play mp3 using Python, PyQt, and Phonon

I been trying all day to figure out the Qts Phonon library with Python. My long term goal is to see if I could get it to play a mms:// stream, but since I cant find an implementation of this done anywh…

Python dictionary keys(which are class objects) comparison with multiple comparer

I am using custom objects as keys in python dictionary. These objects has some default hash and eq methods defined which are being used in default comparison But in some function i need to use a diffe…

How can I make np.save work for an ndarray subclass?

I want to be able to save my array subclass to a npy file, and recover the result later.Something like:>>> class MyArray(np.ndarray): pass >>> data = MyArray(np.arange(10)) >>&g…

With ResNet50 the validation accuracy and loss is not changing

I am trying to do image recognition with ResNet50 in Python (keras). I tried to do the same task with VGG16, and I got some results like these (which seem okay to me): resultsVGG16 . The training and v…

string has incorrect type (expected str, got spacy.tokens.doc.Doc)

I have a dataframe:train_review = train[review] train_reviewIt looks like:0 With all this stuff going down at the moment w... 1 \The Classic War of the Worlds\" by Timothy Hi... 2 T…