Comments (7)
Thanks for this helpful context @gkamener. We'll take this into consideration.
Another API method that may help with your metadata validation use case, is Read Metadata. This returns, the full EML metadata record, in XML, and is the source of information that is indexed and returned through the Search Data Packages method. So, if you are looking for the information via Search Data Packages, you will also find it in the source metadata. Note, the indexed metadata is a considerably smaller subset of the source metadata record.
Now, you may be scratching your head asking "Why would I want to access the publication date through the Read Metadata method just to get the same value I get through Search Data Packages?", well, there is actually a transformation that occurs in the Search Data Packages pathway that you can bypass by reading the EML metadata and parsing the XML to get the <pubDate>
element value directly. For example:
> library(EDIutils)
> library(xml2)
>
> # Read the metadata of a data package and get the publication date
> eml <- read_metadata("knb-lter-fce.1076.4")
> pubdate <- xml_find_all(eml, xpath = ".//dataset/pubDate")
> xml_text(pubdate)
[1] "2019-03-05"
>
> # While we're at it, get the begin and end dates as well
> begindate <- xml_find_all(eml, xpath = ".//dataset/coverage//.//beginDate/calendarDate")
> xml_text(begindate)
[1] "1998-08-19"
>
> enddate <- xml_find_all(eml, xpath = ".//dataset/coverage//.//endDate/calendarDate")
> xml_text(enddate)
[1] "2006-12-03"
>
from ediutils.
Thanks for reporting this @gkamener. I'll have a look and get back to you here.
from ediutils.
Thanks for your patience @gkamener.
EDI is considering a change to the pubdate
field of the Search Data Packages API method, but before doing so, and possibly breaking anyones code currently using this result as it currently stands, we'd like to hear more about your particular use case.
Please note, one immediate fix to this issue is to call the Read Metadata Resource Metadata method with the full data package ID (e.g. knb-lter-fce.1076.4) to get the dateCreated
field, which contains the value of pubdate
but in the YYYY-MM-DD
format you are looking for.
from ediutils.
Thank you for reviewing this @clnsmth.
My use case is to utilize metadata from each FCE package in EDI's repository as a validation check against portions of metadata we have for those packages in the FCE database. We use the latter to track the current status and other details for each package, and the metadata returned from search_data_packages
has already helped me correct some erroneous enddate
values plus other metadata in our database.
Being able to retrieve the most recent pubdate
values in the YYYY-MM-DD
format for all FCE packages through search_data_packages
would be helpful, but I don't think making such changes just for my use case would be worth breaking anyone's code.
Thank you for suggesting read_metadata_resource_metadata
, I may look into that as a check to ensure that pubdates
from EDI align with what we have in the FCE database.
from ediutils.
Thanks for the suggestion @clnsmth! It's very helpful!
from ediutils.
Hi @gkamener. Is there anything else I can lend a hand with before closing this issue?
from ediutils.
Hi @clnsmth. I think I'm good. Thanks for the help!
from ediutils.
Related Issues (20)
- Vignettes covering common use cases HOT 1
- wrapper function to download all tables in a package HOT 3
- function validate_path() should tell me the invalid path HOT 1
- include EDIutils in EDIorg-repository-index HOT 3
- enhance input to api_get_provenance_metadata to accept urls and dois HOT 4
- LTER affiliation no longer valid HOT 1
- Make read_tables() more flexible HOT 2
- api_update_data_package() returning 401 after successful PUT HOT 3
- Function to push a "staged" data package to the production environment
- In vignettes/retrieve_downloads.Rmd documentation on using .Renviron is confusing. HOT 2
- Error : 'validate_file_names' is not an exported object from 'namespace:EDIutils' HOT 6
- Create vignette on working with EML and XML
- Add parameter to report functions for only listing warns and errors
- Demonstrate common search patterns using Solr queries
- xml2df() drops null elements HOT 2
- comma in entity names causes read_data_entity_names to return extra columns HOT 1
- Failing CRAN checks related to vcr HOT 2
- add vignette to get the newest version of a dataset HOT 6
- read_data_entity_names of multiple package_ids HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ediutils.