ASDF swallows warnings raised by converters about asdf HOT 6 CLOSED

Thanks for opening the issue. Do you have an example of a warning that is swallowed? Testing with a simple converter I'm failing to reproduce this issue:

import asdf
import warnings


class Foo:
    pass

tag_uri = "asdf://example.com/tags/foo-1.0.0"


class FooConverter:
    types = [Foo]
    tags = [tag_uri]

    def to_yaml_tree(self, obj, tag, ctx):
        warnings.warn("to_yaml_tree", DeprecationWarning)
        return 'foo'

    def from_yaml_tree(self, obj, tag, ctx):
        warnings.warn("from_yaml_tree", DeprecationWarning)
        return Foo()


class FooExtension:
    converters=[FooConverter()]
    tags=[tag_uri]
    extension_uri = "asdf://example.com/extensions/foo-1.0.0"


with asdf.config_context() as cfg:
    cfg.add_extension(FooExtension())

    af = asdf.AsdfFile({"foo": Foo()})
    assert type(af["foo"]) == Foo
    af.write_to("test.asdf")

    with asdf.open("test.asdf") as af:
        assert type(af["foo"]) == Foo

I see 2 deprecation warnings:

DeprecationWarning: to_yaml_tree
  warnings.warn("to_yaml_tree", DeprecationWarning)
DeprecationWarning: from_yaml_tree
  warnings.warn("from_yaml_tree", DeprecationWarning)

from asdf.

Its a slightly complicated setup, but I don't see the warning I am issuing in my code.

To setup, write a ReadnoiseRefModel to an asdf file using the main branch of roman_datamodels: https://github.com/spacetelescope/roman_datamodels.

import roman_datamodels.datamodels as rdm
from roman_datamodels.maker_utils import mk_datamodel

model = mk_datamodel(rdm.ReadnoiseRefModel)
model.to_asdf("readnoise.asdf")

Now install my dev branch of roman_datamodels: https://github.com/WilliamJamieson/roman_datamodels/tree/feature/autogen. Then try to read the file:

import asdf
from roman_datamodels.datamodels import _generated
with asdf.open("readnoise.asdf") as af:
    model = af.tree["roman"]
    assert isinstance(model, _generated.ReadnoiseRefModel)

I only get the warning:

/Users/wjamieson/.pyenv/versions/roman_datamodels/lib/python3.11/site-packages/asdf/asdf.py:326: AsdfWarning: File 'file:///Users/wjamieson/workspaces/roman_datamodels/ReadnoiseRef.asdf' was created with extension URI 'asdf://stsci.edu/datamodels/roman/extensions/datamodels-1.0' (from package roman-datamodels==0.18.1.dev8+g70f05e8), which is not currently installed
  warnings.warn(msg, AsdfWarning)

Which is due to the slightly different versions of RAD and RDM used.

I should also get an additional warning:

DeprecationWarning: The tag: asdf://stsci.edu/datamodels/roman/tags/origin-1.0.0 has been removed as a stand alone tag, if this data is written the tag will not be included

from asdf.

That warning should be issued by: https://github.com/WilliamJamieson/roman_datamodels/blob/6643f5052f1994ac00a356563b8aae74e283ddf7/src/roman_datamodels/pydantic/converter.py#L90-L92.

I know this warning is run because if I turn it into an error then I get an exception during reading the file.

from asdf.

Could the warning be swallowed by ASDF issuing a warning due to the library version mismatches?

from asdf.

Thanks for sharing the details.

I was able to get the autogen branch installed but had to make a few edits to get it to run (more on that below). I am able to see the warning but as expected (since this is a DeprecationWarning) it only appears if run with python development mode (or with appropriately set warnings filters). Would you try running your test again with python -X dev <your script>?

To get the autogen branch to run I first ran into a ClassVar error:

TypeError: Plain typing.ClassVar is not valid as type argument

Removing the ClassVar annotations from the jinja templates (like this one) silenced that error.

The second error was TypeError: unhashable type: 'ModelPrivateAttr'. I modified the failing line: https://github.com/WilliamJamieson/roman_datamodels/blob/feature/autogen/src/roman_datamodels/pydantic/extension.py#L19
to instead use _tag_uri.default.

I doubt that either of the fixes impacted the warning filters and instead might point to version/dependency issues with the branch.

from asdf.

Those "fixes" unfortunately break other code. The issue you experienced was an oversight on my part. Python < 3.11 requires ClassVar to have a type argument. I was not using one for some of the autogenerated code. I have corrected this with ClassVar[str].

Note that the ClassVar annotation is necessary for Pydantic to correctly handle the field upon class creation in order to support some of my other functionality, so it cannot be removed.

from asdf.