Per the MBON DMAC WG meeting on 11/8 - I'd like to start a page for a how to guide for MBON metadata. Starting with the EML OBIS-USA can use.

cc: @albenson-usgs

Any tweaks to https://iobis.github.io/bioeco-docs/#2-add-monitoring-programmes?

We have a section for guidance on titles for dataset metadata. Should we include a template for US MBON dataset titles?

Here's a thought on what a template title could look like:

[observations] collected by [observation method] and distributed through [MBON Project] from [start date] to [end date] in the [marine region].

For example,

Zooplankton occurrences collected by bottle samples and distributed through the Gulf of Maine Marine Biodiversity Observation Network from 2004 to 2007 in the Gulf of Maine.

See https://ioos.github.io/mbon-docs/metadata-eml.html#eml-metadata

The quick start guide should include:

• Register the dataset in the MBON dataset registration form. https://forms.gle/2meikdcMgYY6nMmG6
• Add a polygon and descriptive metadata about the observations that were collected to https://bioeco.goosocean.org/
• identification of what data goes where:
  • tabular data
    • raw data - RA ERDDAP
    • occurrence data - OBIS-USA
  • eDNA
    • raw sequences - NCBI
    • processed tabular data - RA ERDDAP
    • occurrence data - OBIS-USA
  • animal tracking
    • satellite telemetry - ATN
    • acoustic tracking - OTN/OTN node
  • passive acoustic
    • raw/processed audio files - NCEI
    • occurrence data - OBIS-USA
• Identification of visualization platform
  • MBON Data Portal
  • ATN Portal
  • NCEI Passive Acoustic Portal
  • Infographics
  • etc...

• Benson, Abigail; LaScala-Gruenewald, Diana; McGuinn, Robert; Satterthwaite, Erin; Beaulieu, Stace; Biddle, Mathew; et al. (2021): Biological Observation Data Standardization - A Primer for Data Managers. ESIP. Online resource. https://doi.org/10.6084/m9.figshare.16806712.v1

The new OSTP PARR memo puts an emphasis on digital persistent identifiers. While we don't have to do anything about it yet, we should be thinking about persistent identifiers anyways.

I propose that this documentation should include information/guidance on US MBON's approach to the following categories of digital persistent identifiers:

• Data (DOI through repositories (eg. NCEI, OBIS/GBIF/USGS??)
• Person (ORCiD)
• Institution (ROR, ORCiD, OceanExpert (although I don't think that's appropriate as a persistent identifier))

We should include information about:

1. Why we need them
2. How to get them
3. How they get used

Once data manager feels their data are ready for review, submit a "Marine Biological Observation Dataset Registration" in the bio_data_guide repository using the issue here https://github.com/ioos/bio_data_guide/issues/new/choose.

This notebook is a good start https://ioos.github.io/ioos_code_lab/content/code_gallery/data_access_notebooks/2022-11-23_pyobis_example.html

I think citations will come from GBIF, unless OBIS brings those into the available records.

There are now issue templates for "thinking about submitting data" and "ready for review" datasets when aligning to Darwin Core and submitting to OBIS-USA. The documentation here should be updated to reference those issue templates:

https://github.com/ioos/bio_data_guide/issues/new/choose

I want to take a look at the site again with the updated ToC before trying to break out into multiple pages. Here is the note I took on this today though:

page should broken into more targeted to a user-stories.
currently is big page with a lot of info for different purposes.

• data file standards
• MBON portal

Dataset: https://obis.org/dataset/62b97724-da17-4ca7-9b26-b2a22aeaab51
Protocols: https://mbari-bog.github.io/MBON-Protocols/

I think these would fit under samplingProtocol but should get confirmation from @albenson-usgs first.

If so, I can pull in the DwC-A from https://ipt-obis.gbif.us/archive.do?r=18s_monterey_bay_time_series_edna&v=1.2 and add samplingProtocol to the occurrence.txt file. Then, resubmit.

xref:

• ioos/documentation-theme-jekyll#2
• ioos/ioos-documentation-jekyll-skeleton#4

steps:
- name: Checkout repository
  uses: actions/checkout@v2
- name: Checkout submodules
  run: git submodule update --init --recursive

gh-action cron https://docs.getnacelle.com/deployment/scheduled-builds-github-actions.html#configure-the-schedule

Statement: There have been some conversations about how US MBON can build an integrated product with all the data being collected.

Requirements: IMO, product should be built on data that have been standardized and shared to OBIS.

Problem: What should an integrated product be? The minimum data shared with OBIS are occurrences of a species at a location and time. Can we do something with that? Are there additional standardized data available we can integrate to make something?

My thoughts:

• Out of all the datasets mobilized to OBIS from MBON projects (https://obis.org/institute/23070), I wonder how much and what type of additional data are available?
• Once we know that, can we make a useful product from the occurrences and those additional measures?
• Finally, from this exercise, develop guidance to other projects on what additional data to standardize in DwC and share?

I think this notebook will help as a start:

• https://ioos.github.io/ioos_code_lab/content/code_gallery/data_access_notebooks/2022-11-23_pyobis_example.html

• Best practices and recommendations for plankton imaging data management: Ensuring effective data flow towards European data infrastructures. http://dx.doi.org/10.25607/OBP-1742
• Standards and practices for reporting plankton and other particle observations from images. http://dx.doi.org/10.1575/1912/27377
  • connecting to DarwinCore
  • Standards and Best Practices For Reporting Flow Cytometry Observations http://dx.doi.org/10.25607/OBP-1864.2
    • connecting to SeaBASS and some interoperability with OBIS

paper: https://dx.doi.org/10.5670/OCEANOG.2021.220
Graphics:

and

As discussed during the 12/13 MBON DMAC WG meeting, we will be adding "US MBON" as the organizationName for an additional associatedParty.

The associatedParty should be the person who worked with Abby to align the data to DarwinCore. You can find information about the associatedParty in this eml documentation: https://eml.ecoinformatics.org/schema/eml-resource_xsd.html#ResourceGroup_associatedParty

The role can be Processor but that might not always be the case (?). We don't have a clear source for the roles, but the following ticket should help us identify where the source is:

• gbif/ipt#1915

Finally, I have started a conversation with OceanExpert on creating an institution with the following information:

Name: United States Marine Biodiversity Observation Network
Acronym: US MBON
Address: 1315 East-West Highway, Silver Spring MD 20910
Website: https://ioos.noaa.gov/project/mbon/

Just so everything is referenced in this repo.

• ioos/ioos-atn-data#77
• ioos/ioos-atn-data#27

Initial attempt at visualizing what is documented at https://www.ncei.noaa.gov/products/passive-acoustic-data#tab-3561

%%{
  init: {
    'theme': 'base',
    'themeVariables': {
      'primaryColor': '#007396',
      'primaryTextColor': '#fff',
      'primaryBorderColor': '#003087',
      'lineColor': '#003087',
      'secondaryColor': '#007396',
      'tertiaryColor': '#CCD1D1'
    },
   'flowchart': { 'curve': 'basis' }
  }
}%%

flowchart TD

A["hydrophone"] 

B{{"PassivePacker"}}

C{{"Darwin Core
Alignment"}}

D[("NCEI")]

E[("IPT
OBIS-USA")]

F[/"MBON 
Data Portal"\]

G([OBIS])

H([GBIF])

P[(IOOS RA ERDDAP)]

I(["IOOS Data Catalog
(data.ioos.us)"])

J([NOAA OneStop])

K([data.gov])

L(["Commerce
Data Hub"])

M[/"IOC-UNESCO Harmful Algae Information System"\]

N[/"Infographics"\]

%% O["GitHub /
%% protocols.io"]


A -- Species Detections --> C
A --raw audio files & 1-min hybrid millidecade bands --> B
B --> D
%% A -- Protocols --> O
C --> E
E --> D
%% O .-> E
E --> G
E --> H

%% NOAA STUFF
%% A -- Environmental Observations --> D
A -- non-acoustic data --> P
P --> I
P -- raw --> D
P -- occurrence --> C
I --> J
I --> K
I --> L
D --> J
D --> K
D --> L

G .-> Q
H .-> Q
F .-> Q
D .-> Q
J .-> Q
K .-> Q
L .-> Q

subgraph Q[Example Products]
M
N
F
end

click C "https://doi.org/10.35035/doc-vf1a-nr22" "GBIF eDNA Manual" _blank
click D "https://www.ncei.noaa.gov" "NCEI" _blank
click F "https://mbon.ioos.us" "MBON" _blank
click G "https://obis.org" "OBIS" _blank
click H "https://gbif.org" "GBIF" _blank
%%click I "https://data.ioos.us" "IOOS Catalog" _blank
%%click J "https://data.noaa.gov/onestop/" "NOAA OneStop" _blank
%%click K "https://data.gov" "data.gov" _blank

non-acoustic data is essentially anything measured in concert with the passive acoustic hydrophone. For example, abiotic measurements from sensors attached to the same platform, biotic observations nearby (camera trap), etc.

@bbest started marinebon.github.io/map-of-activities

I'd like to have this read from Bio-Eco portal and US MBON record at OBIS.

Bio-Eco Portal - https://geonode.goosocean.org/search/?title__icontains=MBON&abstract__icontains=MBON&purpose__icontains=MBON&f_method=or&limit=5&offset=0 (do we want all MBON or only US MBON??)

US MBON record at OBIS - https://obis.org/institute/23070

These two pages overlap with other pages on the site. We should consider removing:

• https://ioos.github.io/mbon-docs/metadata.html
• https://ioos.github.io/mbon-docs/data.html

We should provide a recommendation for which license to select.

Highly recommend CC0 1.0 see http://creativecommons.org/about/cc0

With this license the publisher has waived all rights to these data ,to the extent possible under law, and has dedicated them to the public domain. Users may copy, modify, distribute and use the work, including for commercial purposes, without restriction.

OBIS Manual reference https://manual.obis.org/policy.html#data-sharing-agreement

cc @albenson-usgs @sformel-usgs

update https://github.com/ioos/mbon-docs/blob/gh-pages/_docs/images/MBON_Data_Flow.png

using mermaid to show how records get to data.ioos.us, noaa one stop, data.gov.

This site is documenting how to contribute to the MBON data portal, so we should update the title to clearly state that.

Right now it's a little confusing having this help site for the MBON portal and this MBON Portal documentation site.

If we can merge the two together, that would be great!

I think the sections using the MBON portal, adding data to the map view, and how biodiversity products are created should be in the help site for the MBON portal as they are documenting specifics about the portal.

The site in this repository should be dedicated to providing documentation to the MBON projects on how to properly manage their biodiversity data. Not necessarily specific to the MBON portal.

In short, let's change the title so it's clear what each page is documenting. Then, lets look at merging the documentation.

Various data systems that we are using require different flavors of metadata. One consistency between them is they all allow for some "free text" strings of keywords.

It has been mentioned in multiple venues that it would be beneficial to have a set list of identified keywords for the MBON projects to include in metadata records.

We should compile a list of those terms and guidelines for what keywords should be included.

Thinking terms like:

• US MBON
• MBON
• affiliated Regional Association
• IOOS
• affiliated institution/university

The charter @albenson-usgs et al. helped build should be markdownified and shared on this site.

@Dylan-Pugh I've been documenting your process for the Gulf of Maine Wilkinson Basin Time Series Station Calanus Abundance Observations at https://ioos.github.io/mbon-docs/use-case.html#gulf-of-maine-wilkinson-basin-time-series-station-calanus-abundance-observations.

At some point we should tag up on how to get the source data (dataset on NERACOOS' ERDDAP) to NCEI. I've scoped out what the process could look like here, but am open to suggestions.

I have a few thoughts/opinions regarding the data file formats list. In no particular order:

1. change "containers" to "file archives"
2. can we add notes on compression to list of zip/gzip/tar?
   • especially for .tar, .tar.gz and .tar.bz2 are common and good practice
3. "databases" seems too generic.
4. what about .json and .sql for databases?
5. metadata file formats should be mentioned here?
6. what about notes for overall directory structure for "containers"("file archives")?
   • README file recommendation?
7. "text" section should not include .xml and should include .txt, .md, (maybe) .rtf?
8. what about "spreadsheets"? I would like to see an explicit recommendation to use an open format instead of proprietary .xlsx (eg .odf)
9. "still images" should include a vector format (.svg)
10. "still images" might include a multi-layer composited image format (.xcf?)

• submit sequences to NCBI
• submit occurrences to OBIS

Get insight from Diana's presentation.

Document where to put protocols, in what format? BeBOP's infrastructure? https://github.com/BeBOP-OBON

Document how to link those to the datasets.

This site should have a how-to page for getting data into the MBON Data Portal. The section exists [1] but we should look at how to make that more functional. Should they submit to the MBON registration form? Should they email someone? What's the communication pathway?

[1] https://ioos.github.io/mbon-docs/index.html#contributing-data-to-the-mbon-portal

I see documentation here

but no clue how to edit sidebar_ioos.yml.

• https://www.frontiersin.org/articles/10.3389/fmars.2021.703682/full
• https://www.frontiersin.org/articles/10.3389/fmars.2021.760840/full
• https://rwscollab.github.io/pam-data-mgmt/

MBON Colleagues,

I am very happy to announce that all datasets published through the OBIS-USA IPT are now archived at NCEI and will be archived quarterly going forward. This means when you publish data through OBIS-USA it will automatically be archived at NCEI....

Warmly,
Abby

In preparing for the MBON Data Portal demo on 1/10, we should create a page which documents the tutorial for reference after.