Comments (20)
Not sure if this is helpful here, but baseline technical metadata properties across file formats/types from the Hydra Technical Metadata Subgroup are available: https://docs.google.com/document/d/1SZCpSIdlGfXgoYrAnW2eRKlIt6O-1ADIDDhmLrvxeLc/edit#heading=h.a8hurtypz8qi
from hydra-pcdm.
In a block like so:
metadata do
type PCDM::File
property :foo, ...
end
from hydra-pcdm.
Reiterating @jlhardes comment about the technical metadata profile -- this is now available on the duraspace wiki at: https://wiki.duraspace.org/display/hydra/Technical+Metadata+Application+Profile
from hydra-pcdm.
@jcoyne Just to make sure I understand what I need to add here.
Is the goal of this ticket to have something like this at the end (assuming I get the correct Ruby classes for each of those predicates) ?
metadata do
configure type: RDFVocabularies::PCDMTerms.File
property :label, predicate: ::RDF::RDFS.label
# TODO: Get the proper Ruby classes for these predicates
property :file_name, predicate: ::ebucore.file
property :file_size, predicate: ::ebucore.fileSize
property :date_created, predicate: ::ebucore.dateCreated
property :file_hash, predicate: ::premis.hasMessageDigest
property :mime_type, predicate: ::ebucore.hasMimeType
property :date_modified, predicate: ::ebucore.dateModified
property :file_format, predicate: ::pronom.puid
property :byte_order, predicate: ::sweetjpl.byteOrder
end
from hydra-pcdm.
👍 Pronom's impossible, but besides that
from hydra-pcdm.
Should hydra-pcdm be concerned about what kind of tech metadata it is, or just that it has any kind of tech metadata?
from hydra-pcdm.
For this sprint, I think it makes sense to go with required properties (File Name and File Size) at a minimum. If recommended properties can also be includes (Label, Date Created, File Hash, File Format Type, and Has Mime Type) that would make it more complete. The optional fields can probably be safely ignored for the sprint - they aren't completely workable anyway (pronom:puid, for example).
from hydra-pcdm.
@jlhardes I agree. My question was really more about the schema. Are we enforcing a techdata schema at this level? I guess it doesn't matter, since it's RDF, if an implementer wants to use a different one, they just add it in. I think we can flesh out those details at the hydra-works level, with integration tests that serve as an example of someone who would want to build their own PCDM-approved object and add additional technical metadata.
from hydra-pcdm.
@jlhardes This is good to know since I've got File Size working (Fedora does that automatically using premis:hasSize as the predicate)
I need to talk with Esme about hasOriginalName since Fedora seems to do it out of the box but I cannot get it to work.
from hydra-pcdm.
@jlhardes Question: Is it OK if we use the equivalent predicates indicated in the document that you posted (e.g. use "premis:hasSize" instead of "ebucore:fileSize") or should we use the one indicated at the top of each one (e.g. "ebucore:fileSize") ?
from hydra-pcdm.
If we can stick with the properties at the top (Property name: ebucore:fileSize) that might make things easier for the sprint (not so much to implement). Those Property names are the main ones we'd like to see implemented anyway for technical metadata. The equivalent properties are listed to help explain the property and to provide options if the property we're listing can't be used for some reason.
from hydra-pcdm.
@hectorcorrea the logic behind using ebucore was that it is a comprehensive vocabulary for technical metadata. So rather than splitting the technical metadata properties across lots of different vocabularies (nfo, exif, dc, premis, etc, etc), it would be much more sane to start with a well supported, single vocabulary.
from hydra-pcdm.
@jlhardes @acoburn thanks for the background info. I'll look into implementing it with ebucore then.
from hydra-pcdm.
@jlhardes @acoburn Fedora automatically calculates and stores (as read-only) the following properties premis:fileSize, fedora:digest, and fedora:mimetype.
I could add three separate properties with educore predicates as the document recommends, but they would have to manually set and run the risk of having different values than what Fedora already stores. Do we really want to do that or should we stay with the Fedora provided properties?
//cc: @awead @jcoyne (thoughts?)
from hydra-pcdm.
@hectorcorrea part of the thinking here was that if an external tool (e.g. FITS) calculates these value, they can be put into the ebucore properties (since the existing properties are managed by the server and hence read-only). The advantage of using the additional properties include:
- There is a clearer line between server managed properties and externally managed properties (potentially useful for provenance)
- The fedora: namespace is much less widely used than ebucore: and so potentially less transferrable
- By keeping as much technical metadata within a single namespace, you make the data (potentially) more useable in a LOD context
- If an application chooses to use a different hashing algorithm than SHA1, that option is available
- If there is a mismatch between the server managed properties and an externally generated value, that might be useful for certain types of preservation activities.
The disadvantage of using the additional properties is:
- data duplication
- more code to write / manage
That said, I don't actually have a strong opinion one way or the other. @jlhardes thoughts?
from hydra-pcdm.
Yes, 👍 to that. But, I don't think hydra-pcdm should have any opinions about what tech data you're using or what your'e using to create it. It should just allow you to using whichever tool and schema you prefer.
from hydra-pcdm.
@awead I disagree with that somewhat. I think it should provide an opinionated default. You should be allowed to do something else though.
from hydra-pcdm.
We had some discussion about these properties in relation to properties that are already in Fedora and I wasn't quite sure which of these mapped, so you've helped clear that up, @hectorcorrea - thanks!
I don't actually understand how it works to NOT use what we are implementing on this sprint. It seems like we want to see a baseline of technical metadata across all Hydra implementations using PCDM to make things easier going across systems and sharing externally. I understand that we don't want to limit people's implementations by making these properties using these predicates a requirement but it seems like we do want to encourage their use.
I think for that reason and for the longer term it's better to go with a more externally-useful standard, so I'd stick with premis:hasMessageDigest, ebucore:hasMimeType, and ebucore:fileSize to express those properties, even though it is a bit of duplication.
Additionally, I don't think premis:fileSize actually exists (http://id.loc.gov/ontologies/premis.html) - at least not in RDF premis. I think the premis property might be hasSize so if Fedora is using premis:fileSize, I'm not sure what ontology is actually being used.
from hydra-pcdm.
@jcoyne agreed. If there's any additional tech metadata you want beyond what Fedora is giving you already, then it should be as easy as simply including a module with the additional properties. Any implementation would then override that module, or more realistically, just include their own. The side effect is that you may have extra triples with different predicates but duplicate object content.
So, if we use @jlhardes recommendations, you'd have two triples with the checksum, fedora:digest and premis:hasMessageDigest. And two for mime type: fedora:mimeType and ebucore:hasMimeType (assuming their object values can be the same). I think @hectorcorrea meant premis:hasSize. That's what comes back from Fedora if you do GET request on the binary's fcr:metadata node.
from hydra-pcdm.
So I went ahead and implemented the additional properties. The only caveats with the current implementation are:
- Fedora considers
PREMIS.hasMessageDigest
a server-managed property and therefore it does not let us change the value of this property (i.e. this is a read-only property.) - Property
pronum:puid
indicated in the documented linked by @jlhardes was not implemented since this ontology hasn't been published.
from hydra-pcdm.
Related Issues (20)
- Get Test Coverage above 90% HOT 3
- Identify copyright holder in LICENSE HOT 2
- Update contribution guidelines for Samvera changes
- Add contact/mailing list info
- Update Rubocop
- Update Travis build configurations HOT 1
- Clean up stale branches
- Rails 5.2 support
- Increase the test coverage to 100% HOT 1
- Test against Rails release 5.1.7 and Ruby releases 2.6.3, 2.5.5, and 2.4.6
- Add support for Ruby 2.7.z releases
- Add support for Rails 6.0.z releases HOT 1
- Update the link to CodeClimate HOT 1
- Add Circle CI step that fails if branch name is master
- Support Rails 6.1 releases
- Support Ruby 3.0.3 releases HOT 1
- Configure nightly CircleCI test suite executions
- Update the gemspec to require MFA for RubyGems releases HOT 1
- Issue a request for a new Product Owner
- Provide a roadmap for migration into the samvera/hyrax project
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hydra-pcdm.