librecat / catmandu-lido Goto Github PK

View Code? Open in Web Editor NEW

1.0 16.0 2.0 109 KB

Catmandu tools for working with LIDO data

License: Other

Perl 100.00%

catmandu-lido's Introduction

NAME

Catmandu::LIDO - Modules for handling LIDO data within the Catmandu framework

SYNOPSIS

Command line client catmandu:

catmandu convert LIDO to JSON --fix lido.fix < data/lido.xml > data/lido.json

catmandu convert JSON to LIDO --fix lido.fix < data/lido.json > data/lido.xml

See documentation of modules for more examples.

DESCRIPTION

Catmandu::LIDO contains modules to handle LIDO, an XML Schema for contributing content to cultural heritage repositories.

AVAILABLE MODULES

Catmandu::Exporter::LIDO

Serialize LIDO data
Catmandu::Importer::LIDO

Parse LIDO data

Custom Fixes

AUTHORS

Patrick Hochstenbach, patrick.hochstenbach at ugent.be
Matthias Vandermaesen, matthias.vandermaesen at vlaamsekunstcollectie.be
Pieter De Praetere, pieter at packed.be

CONTRIBUTORS

Patrick Hochstenbach, patrick.hochstenbach at ugent.be
Pieter De Praetere, pieter at packed.be
Matthias Vandermaesen, matthias.vandermaesen at vlaamsekunstcollectie.be

COPYRIGHT AND LICENSE

The Perl software is copyright (c) 2016 by PACKED vzw, VKC vzw and Patrick Hochstenbach. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

catmandu-lido's People

Contributors

Stargazers

Watchers

Forkers

manwar pkg-perl-tools

catmandu-lido's Issues

Encoding issue

There seems to be an enconding issue when converting from JSON to LIDO. All encodings are detected as UTF-8, but still some chars are invalid UTF-8 characters in the output of JSON to LIDO

When using following file: primavera.lido.org.xml.txt

$ catmandu convert LIDO to JSON < primavera.lido.org.xml > primavera.lido.json

does a nice job -> file encoding is correct and all characters are correct

$ file -bi primavera.lido.json 
text/plain; charset=utf-8

converting back to LIDO messes with encoding

$ catmandu convert JSON to LIDO < primavera.lido.json > primavera.lido.new.xml
$ file -bi primavera.lido.new.xml 
text/xml; charset=utf-8

but the resulting file contains invalid UTF-8

catmandu convert JSON to LIDO < primavera.lido.json | iconv -t latin1 -o primavera.lido.new.xml
$ file -bi primavera.lido.new.xml 
text/xml; charset=utf-8

does the trick; the file is still UTF-8 encoded and all characters are fine, which could mean that characters are encoded twice when converting to LIDO

lido_date throws an error "Use of uninitialized value in split"

When using the -latest_date and -latest_date_type, Catmandu will throw these warning errors:

Use of uninitialized value in split at /Users/netsensei/.plenv/versions/5.26.0/lib/perl5/site_perl/5.26.0/Catmandu/Fix.pm line 709.
Use of uninitialized value in split at /Users/netsensei/.plenv/versions/5.26.0/lib/perl5/site_perl/5.26.0/Catmandu/Fix.pm line 709.

Require newer Test-Exception

Some of my smokers show the following failures:

Undefined subroutine &Test2::Global::test2_stack called at /opt/perl-5.20.0/lib/site_perl/5.20.0/Test/Exception.pm line 351.
Compilation failed in require at t/Catmandu-Exporter.t line 4.
BEGIN failed--compilation aborted at t/Catmandu-Exporter.t line 4.
t/Catmandu-Exporter.t ... 
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run 
Undefined subroutine &Test2::Global::test2_stack called at /opt/perl-5.20.0/lib/site_perl/5.20.0/Test/Exception.pm line 351.
Compilation failed in require at t/Catmandu-Importer.t line 4.
BEGIN failed--compilation aborted at t/Catmandu-Importer.t line 4.
t/Catmandu-Importer.t ... 
Dubious, test returned 255 (wstat 65280, 0xff00)
No subtests run

Problem is that the newest Test-Simple (https://metacpan.org/release/EXODIST/Test-Simple-1.302014_001) breaks some other Test modules, amongst them older versions of Test::Exception. To avoid unnecessary failures it's probably better to require a newer version of Test::Exception --- 0.43 should be fine.

Using $append with lido_term causes conceptID and term to be split

When using $append as the last item in a path for lido_term, conceptID and term are separated in two instances of their containing element. They should remain together.

Using $last instead of $append is a work-around.

Example

lido_term(
  descriptiveMetadata.objectClassificationWrap.classificationWrap.classification.$append,
  lido.object_name.object_name,
  -conceptid: 'lido.object_name.object_name\.lref',
  -type: local,
  -source: 'Adlib'
)

Results in:

<lido:classification>
          <lido:term lido:pref="preferred" xml:lang="nl">TestTerm</lido:term>
</lido:classification>
<lido:classification>
          <lido:conceptID lido:pref="preferred" lido:type="global" lido:source="AAT">310035698</lido:conceptID>
</lido:classification>

While we expect:

<lido:classification>
          <lido:conceptID lido:pref="preferred" lido:type="global" lido:source="AAT">310035698</lido:conceptID>
          <lido:term lido:pref="preferred" xml:lang="nl">TestTerm</lido:term>
</lido:classification>