Map is broken at the bottom. Plus just needs a little ❤️, probably remove print statements, etc.

https://ioos.github.io/ioos_code_lab/content/ioos_installation_conda.html

After downloading Miniforge3, it's not clear how to do the next step. We need to add "Open Miniforge Prompt". On Windows it can be done from Start -> Miniforge3 -> Miniforge Prompt

@MathewBiddle if I recall correctly you wanted a gallery page similar to the EDS book, right? Sorry if there is an open issue for it but I could not find it. It turns out we had that in the previous incarnation of this book and I adapted it to the jupyterbook infrastructure. Here is a screenshot of what I have so far:

What do you think?

Looks like the interpolate ERDDAP notebook is failing during the run-tests, Notebook tests:

https://github.com/ioos/ioos_code_lab/actions/runs/3549906453/jobs/5962786926

data_analysis_and_visualization_notebooks/2021-10-25-ERDDAP-interpolate.ipynb

We need to shut down the site https://ioos.github.io/notebooks_demos/content/intro.html.

Probably not as simple as these instructions show. But, lets start there and see where we get.

I'm going through the installation process again with @kbailey-noaa and we identified a few points to improve and/or fix:

• We really need a redirect on all notebook_demos page! Just got bamboozled by that myself when googling for the installation;
• Use Miniforge instead of Miniconda: https://github.com/conda-forge/miniforge/releases/tag/4.10.3-10 b/c that will simplify the channel configuration part and it will be on the safe side of the Anaconda Terms of conditions;
• Make sure curl is installed on Windows/base env so we can simplify all the instructions to copy-n-paste code from the site;
• Add a "remove-all-and-reinstall" note when facing update problems.

• What is language(s) for used in the example?
  • probably Python
• Is it focused on a particular module/software or an IOOS data service?
  • not sure at the moment
• Can you provide a minimum example of the expected code and results in a notebook?
  • probably, once we figure out what to present

Please provide a detailed description of the suggested example below:
We don't have any AI/ML examples as of now. I think it would be nice to have a small example to show that we are doing some work there. I reached out to Hassan (IOOS POC on AI/ML) to see if he has any ideas for an example. I will update this ticket once we have a better idea.

Maybe we can bring something from the GSoC project over? https://github.com/ioos/seafloor-sampling-ml

Let's update this to use http://erddap.sensors.ioos.us/erddap/index.html and query for oxygen data.

cc Jim Reagan at NCEI (GH handle TBD).

At some point we will need to add relevant content into each of these chapter pages:

• data access
• data analysis and visualization
• data management

xref: ioos/notebooks_demos#424

Create a notebook that shows how to use ioos_qc for glider data.

xref.: ioos/ioos_qc#58 and ioos/notebooks_demos#410

Ping: @kerfoot

• What is language(s) for used in the example?
  • Python
• Is it focused on a particular module/software or an IOOS data service?
  • pyobis
• Can you provide a minimum example of the expected code and results in a notebook?
  • https://github.com/MathewBiddle/sandbox/blob/main/notebooks/IOOS_occurrences_in_OBIS.ipynb

Please provide a detailed description of the suggested example below:
The example notebook, provided above, uses two google spreadsheets (which contain OBIS dataset IDs affiliated with various IOOS efforts) to collect occurrence observations in OBIS. I would like to make a notebook that does the same thing, but uses the pyobis package to gather data.

Maybe we can think of a more logical way to manage the dataset IDs too? I don't think we can do searches in OBIS for related keywords, but could be possible?

It would like to see a heatmap of where affiliated IOOS networks have contributed occurrence observations.

Related to #20 we should change the master branch name to main. I think we would need to update GHA and some other configs? @ocefpaf I look for your guidance here.

• What is language(s) for used in the example?
  Python
• Is it focused on a particular module/software or an IOOS data service?
  Zarr
• Can you provide a minimum example of the expected code and results in a notebook?
  I think @ocefpaf has some

Please provide a detailed description of the suggested example below:
We should add an introduction to zarr notebook. Some things to include:

• Translating netCDF to zarr
• Creating zarr
• Working with zarr

• xref: ioos/notebooks_demos#352

@ocefpaf: ... and create a new one with the new functionalities.

@emiliom:

FYI, I recently benefited from the IOOS Data Demo siphon notebook. I could not find better demonstrations or documentations of catalog_ref navigation in the siphon repo, web site or related materials! I've been meaning to add an issue to the siphon repo to suggest to them to add the IOOS siphon notebook to their Siphon Examples page ...

• xref: ioos/notebooks_demos#366

Per a recent discussion concerning attribution, we can update the pocean timeSeries demo to include an attribution step.

From the discussion:

Organization W collected the oceanographic data data;
Organization X received the data from Organization W and published it;
Organization Y downloaded the data from Organization X and QC'd the datasets
Organization Z took the QC's data from Organization Y and generated netCDF for publication
Organization G reviewed and uploaded the data to its ERDDAP for wider distribution.

Question 1: Who is the data creator?

Question 2: Will acknowledging the chain of custody in the global attribute "Summary" sufficient?

https://github.com/ioos/ioos_code_lab/blob/main/jupyterbook/content/code_gallery/data_management_notebooks/2018-02-27-pocean-timeSeries-demo.ipynb

From @ocefpaf:
For future reference (copied from the ioos_tech thread):

creator_name: First Last
creator_email: [email protected]
creator_url: http://www.myorg.org/
creator_type: person
creator_institution: My Institution
institution: Usually the sama as the above
publisher_name: RA Data Manager
publisher_email: [email protected]
publisher_url: http://www.ra.org/
publisher_type: position
publisher_institution: The RA
program: List of Programs
project: List of projects
contributor_name: List of names
contributor_role: Principal Investigator, Field Program Manager, Data Manager, Quality Assessment
history: File generated my-awesome-script.py

See: https://data.nodc.noaa.gov/thredds/dodsC/ioos/nanoos/ohsucmop/woody/woody.240.A.CTD/200911-926.nc.html

We should modify it to be closer to https://ioos.github.io/ioos-metadata/ioos-metadata-profile-v1-2.html#attribution

Can we update this notebook to reflect current install guidance from here?

Might be worth showing the API interaction too? https://github.com/ioos/compliance-checker#api

• What is language(s) for used in the example?
  Python/web browser?
• Is it focused on a particular module/software or an IOOS data service?
  https://eds.ioos.us/map/#
• Can you provide a minimum example of the expected code and results in a notebook?
  Maybe?

Please provide a detailed description of the suggested example below:

I've been tinkering [1] with deploying Rstudio through mybinder [2] and it's pretty neat. Does pangeo offer the same functionality? Might be a link we could add to the readme.

I've successfully got it working here [3] but it takes a while to build and deploy (~20 mins).

[1] https://github.com/ioos/bio_mobilization_workshop
[2] https://github.com/binder-examples/r-conda
[3] https://mybinder.org/v2/gh/ioos/bio_mobilization_workshop/main?urlpath=rstudio

• What is language(s) for used in the example?
  • Python (I think)
• Is it focused on a particular module/software or an IOOS data service?
  • No (maybe it could be built into a package?)
• Can you provide a minimum example of the expected code and results in a notebook?
  • See this gist

Please provide a detailed description of the suggested example below:

• A requirement for OBIS is that all scientific names match to the World Register of Marine Species (WoRMS) and a scientificNameID is included. A scientificNameID looks like this urn:lsid:marinespecies.org:taxname:275730 with the last digits after the colon being the WoRMS aphia ID. In some cases, data providers need to go out to WoRMS to grab this information (or verify its correctness).In other cases a user might want to capture an lsid from another service, like ITIS. In the example linked above, it takes a data file and matches a column of taxanomic names to data sources using the Global Names Resolver. http://resolver.globalnames.org/api

Hi @MathewBiddle. I'll ask here, for lack of a better alternative other than email or Slack. The IOOS CodeLab has Jupyter notebooks in R. Has the thought or need for supporting Rmarkdown ever come up? Specifically, how do you best support contributors who do nothing but R, for whom using a Jupyter notebook to do R might be an annoying hurdle? I've been pondering that possibility, but I'm not aware of good options that can make both Jupyter+Python and R-ecosystem people happy. The NASA-Openscapes group used quarto to do this for their Cloud hackathon, but I haven't wrapped my head around its workflow yet. In my own ideal world, JupyterBook would support RMarkdown notebooks, but I haven't found solutions that implement this.

Not sure if GA is working correctly, I need to check the configuration we have for the current identifier. See ioos/notebooks_demos#413

• xref: ioos/notebooks_demos#341

from @rsignell-usgs:

It would be cool to have an intake catalog of IOOS model output that could be dropped into the new xrviz tool being developed during the Google Summer of Code project by @hdsingh

• xref: ioos/notebooks_demos#404

Update 2016-10-12-fetching_data
to use non SOS API.

From @LindsayRAbrams:
From discussions with IT, either our Data API or ERDDAP server would work, depending on what is more useful for your users.
More info on CO-OPS Data API: https://api.tidesandcurrents.noaa.gov/api/prod/
More info on CO-OPS ERDDAP: https://opendap.co-ops.nos.noaa.gov/erddap/index.html

• What is language(s) for used in the example?
  • Python
• Is it focused on a particular module/software or an IOOS data service?
  • FASTA file format, parquet format, and Dask
• Can you provide a minimum example of the expected code and results in a notebook?
  • See the notebooks in https://github.com/jonathancosme/COSME/tree/master/notebook_walkthroughs, especially A_convert_raw_to_parquet.ipynb.

Description of the notebook:
Folks are starting to do more work with metagenomics data in python. Jonathan Cosme (Deloitte) has written a few notebooks (linked above) that we should explore porting to the CodeLab. I like the first notebook as it converts raw FASTA files to parquet for manipulation in AI/ML techniques.

CC: @hmoustahfid-NOAA

use libmamba, so the build number is used in that operation instead of just versions. It usually increase stability b/c higher build numbers are preferred and, most of the time, more "correct."

We should add info about installing git on a local machine. Especially installing when user doesn't have admin privilege's. I'm working on that now and will document my findings for Windows.

Before we make the announcement we should push another release and update the DOI package. v2.0.0 or v1.1.0, I think the former as it's a significant change from v1.0.0.

@ocefpaf, you've cleaned a lot of this repo up and it would be nice to stamp this as a release.

See the current DOI package https://zenodo.org/record/5248490#.YUHxPZ1KiUk
Which uses this release https://github.com/ioos/ioos_code_lab/tree/v1.0.0

• What is language(s) for used in the example?
  - Python
• Is it focused on a particular module/software or an IOOS data service?
  - ERDDAP + ATN/Glider observations
• Can you provide a minimum example of the expected code and results in a notebook?

Please provide a detailed description of the suggested example below:

• The example would show how to pull in source data from a Glider or ATN trajectory (don't know where to source that data), collect the latitude, longitude and time.
• Using that information, query the ERDDAP interpolate function to return the interpolated values of interest.
• Make some plots.
• Maybe tie to #17?

xref:

• ioos/notebooks_demos#420

The new JupyterBook-based IOOS Code Lab looks great!

But one thing that I suggest should be added to each notebook (which was present in the previous IOOS Code Gallery, I think) is a date, making it clear to users when the notebook was developed. That's a useful piece of information to help decide if the code used in the notebook might be old and the packages used may have newer, better features.

Currently, a sharp user will notice that the date is in the notebook file name, in the url. But it'd be better to make that information less subtle. It could go right below the title or at the end next to the Author line.

Add a link to the contributing guide to the home page. Can we make some flashy button or something that links to https://github.com/ioos/ioos_code_lab/blob/main/CONTRIBUTING.md ?

<button name="button" onclick="https://github.com/ioos/ioos_code_lab/blob/main/CONTRIBUTING.md">Contribute here!</button>

• xref: ioos/notebooks_demos#405

@mwengren has some solid examples here: https://github.com/mwengren/notebooks-dev/tree/master/ioos_ckan_catalog

Lets make a plan for adding them to the code lab.

The mapping of RA acronym to name is incorrect for CeNCOOS:

"CeNCOOS": "Central and Northern California Coastal Ocean Observing System",

Should be:

"CeNCOOS": "Central and Northern California Ocean Observing System",

See https://www.cencoos.org/

we published two new notebooks:

• https://ioos.github.io/ioos_code_lab/content/code_gallery/data_management_notebooks/2023-03-20-Reading_and_writing_zarr.html
• https://ioos.github.io/ioos_code_lab/content/code_gallery/data_access_notebooks/2022-11-23_pyobis_example.html

Lets put together two/three sentences for EOTO/other sharing venues.

• xref: ioos/notebooks_demos#435

• https://ioos.noaa.gov/data/access-ioos-data/ contains references and links to the old Data Demo Center, not the rebranded Code Lab. See the fourth tab titled Resources.
• https://ioos.us/

Anywhere else?

• What is language(s) for used in the example?
  • Python
• Is it focused on a particular module/software or an IOOS data service?
  • ERDDAP and the St Johns River OFS
• Can you provide a minimum example of the expected code and results in a notebook?

Please provide a detailed description of the suggested example below:
Can we make a notebook that searches for currents, temperature, salinity, total water level, etc. for a specific region of interest without knowing what the endpoint should be?

Most of our example notebooks use the catalog to do this search, but I'm wondering if we can use erddapy.servers to do a "broad scale" search?

I'm thinking of putting this in the context of data assimilation/model calibration. Bring a bunch of data together to initialize a model or calibrate a model.

When I start up a jupyter notebook, I get a notification about a migration plan to Jupyter 7 and the Kernel doesn't connect. Below is a screenshot of the notebook followed by the command line STDOUT when running jupyter notebook. I think it's all the Forbidden notifications that keep me from connecting to the Kernel.

(IOOS) ~\GitProjects\ioos_code_lab>jupyter notebook

  _   _          _      _      
 | | | |_ __  __| |__ _| |_ ___
 | |_| | '_ \/ _` / _` |  _/ -_)
  \___/| .__/\__,_\__,_|\__\___|
       |_|

Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions.

https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html

Please note that updating to Notebook 7 might break some of your extensions.

[I 08:43:30.441 NotebookApp] Loading IPython parallel extension
[I 08:43:35.777 NotebookApp] Serving notebooks from local directory: ~\GitProjects\ioos_code_lab
[I 08:43:35.777 NotebookApp] Jupyter Notebook 6.5.3 is running at:
[W 08:48:44.795 NotebookApp] Forbidden
[W 08:48:44.796 NotebookApp] 403 POST /api/sessions (127.0.0.1) 1.000000ms referer=http://localhost:8888/notebooks/jupyterbook/content/code_gallery/data_access_notebooks/2022-11-23_pyobis_example.ipynb
[I 08:48:44.797 NotebookApp] 302 GET /kernelspecs/python3/logo-64x64.png (127.0.0.1) 0.990000ms
[W 08:48:44.798 NotebookApp] Forbidden
[W 08:48:44.800 NotebookApp] 403 GET /api/contents/jupyterbook/content/code_gallery/data_access_notebooks/2022-11-23_pyobis_example.ipynb/checkpoints?_=1679575722734 (127.0.0.1) 1.000000ms referer=http://localhost:8888/notebooks/jupyterbook/content/code_gallery/data
_access_notebooks/2022-11-23_pyobis_example.ipynb
[W 09:12:52.185 NotebookApp] Forbidden
[W 09:12:52.185 NotebookApp] 403 GET /api/sessions?_=1679575661738 (127.0.0.1) 0.000000ms referer=http://localhost:8888/tree/jupyterbook/content/code_gallery/data_access_notebooks 
[W 09:12:52.187 NotebookApp] Forbidden                                                                                                                                              
[W 09:12:52.188 NotebookApp] 403 GET /api/terminals?_=1679575661739 (127.0.0.1) 1.730000ms referer=http://localhost:8888/tree/jupyterbook/content/code_gallery/data_access_notebooks
[I 09:12:55.922 NotebookApp] Interrupted...
[I 09:12:55.923 NotebookApp] Shutting down 0 kernels  
[I 09:12:55.925 NotebookApp] Shutting down 0 terminals

The following notebooks should be moved to the Data Access chapter [1]:

• https://github.com/ioos/ioos_code_lab/blob/main/jupyterbook/content/code_gallery/data_management_notebooks/2018-02-20-obis.ipynb
• https://github.com/ioos/ioos_code_lab/blob/main/jupyterbook/content/code_gallery/data_management_notebooks/2020-10-10-GTS.ipynb

[1] - https://github.com/ioos/ioos_code_lab/tree/main/jupyterbook/content/code_gallery/data_access_notebooks

Your issue content here.

@ocefpaf could you add the date created to the markdown?

https://ioos.github.io/ioos_code_lab/content/intro.html

Programming Language

R

Module/Software/IOOS data service?

OBIS

Module/Software/IOOS data service?

https://hannahlowens.github.io/voluModel/articles/a_Introduction.html

Provide a description.

Using the example in the documentation for the R package voluModel as a template, this notebook will use a subset of IOOS Marine Life/MBON data to provide an example of species distribution modeling that accounts for the depth of the occurrence. Environmental variables are pulled from the World Ocean Atlas.

If you are opening an issue regarding an existing notebook/code or the webpage just ignore the instructions below.
If you want to suggest a notebook for the code gallery please answer these questions:

• What is language(s) for used in the example?
• Is it focused on a particular module/software or an IOOS data service?
• Can you provide a minimum example of the expected code and results in a notebook?

Please provide a detailed description of the suggested example below:
Bathymetry contours on a map. Puerto Rico PRCRMP has a Demo of MBON portal. I would like to do something where we bring the coral data in and include bathymetry contours, somewhat repeating the PRCRMP demo - benthic portion of the demo. The product can be found at https://mbon.ioos.us/#data/1 in MBON portal. I don’t want to repeat the effort, but I’d like to do something with these data.
Example making contour map from netCDF data at NCEI:

import xarray as xr
url = 'https://www.ngdc.noaa.gov/thredds/dodsC/regional/northern_california_36_msl_2005.nc'
ds = xr.open_dataset(url)  
ds.Band1.plot.contour(levels=50)

Initial steps:

1. Ingest PRCRMP data from OBIS (using robis in R)
2. Get bathymetry data (using Python)
3. Translate R response into Pythonese.
4. Make interactive graphics

xref: ioos/notebooks_demos#378

Bullet 4 in https://github.com/ioos/ioos_code_lab/blob/main/CONTRIBUTING.md#building-new-notebooks should include:

• High level guidance on creating a jupyter notebook. (Is there a reference we can drop in here?)
• details on what to do with large data
• if needed, details on HPC resources
• guidance on the content of markdown cells. Plain language description of what the code is doing.

• ioos/notebooks_demos#440
• ioos/notebooks_demos#406
• ioos/notebooks_demos#398
• ioos/notebooks_demos#393
• ioos/notebooks_demos#390
• ioos/notebooks_demos#370
• ioos/notebooks_demos#362
• ioos/notebooks_demos#356
• ioos/notebooks_demos#347

• xref: ioos/notebooks_demos#364

• What is language(s) for used in the example?
  R , maybe Python?
• Is it focused on a particular module/software or an IOOS data service?
  IOOS data service
• Can you provide a minimum example of the expected code and results in a notebook?

Please provide a detailed description of the suggested example below:
I'm not quite sure yet what we want to show, but MBON Pole to Pole has a bunch of data in OBIS. Maybe we can use the OBIS API to grab the data?

Using the existing R library would probably be a more effective use of our time, and more applicable to the biology folks.

I did find pyobis. It looks like that was last contributed to in 2017, so it might be out of date, but it could be worth exploring.

Access the data including the extended measurement or facts tables from OBIS API.

In the API occurrence endpoint the mof parameter (true/false) has been added which determines if MeasurementOrFact records are included in the results or not. In addition, filters have been added to find data with specific measurement types, values or units. For now this is a simple text search and does not yet take into account relationships between vocabulary terms.

For example, this request finds mollusc occurrences with biomass measurements: https://api.obis.org/occurrence?scientificname=Mollusca&measurementtype=biomass&mof=true

To get MeasurementOrFact support in the R package, please install the latest version from GitHub (version 2.2.0 or above).

See 279acdb

Some additional information about the obis api. Here is a list of all the measurements or facts currently in OBIS: https://reports.obis.org/moftypes/

If you are interested in seeing which dataset have a particular measurement you can get that from the API using the measurementTypeID- here is an example:
https://api.obis.org/dataset?measurementtypeid=http://vocab.nerc.ac.uk/collection/P01/current/SDBIOL05/

You can also search using the measurementType:
https://api.obis.org/dataset?measurementtype=weight

Maybe bring in seascapes data as well: https://github.com/eqmh/ERDDAP-extractions/blob/master/seascape_example.R

similarly: https://github.com/marinebon/p2p/blob/2557dc03abe8e5779f5392586eb470232f8aa604/_site_template.Rmd#L142-L178

See https://marinebon.org/p2p/z_usa-pumphouse-nahantma.html for example

@ocefpaf You put a lot of work into adding that gallery! Lets make a new release! v3.0.0 ? or v2.1.0 ?

xref (among others):

• #181
• #174
• #173

For windows, after installing minimamba, there should be instructions:

• click the window start button and browse to the "miniforge3" folder
• drag "miniforge prompt" onto your windows taskbar (the bar with the icons at the bottom of your screen)
• start the miniforge prompt by clicking on the miniforge prompt icon on your windows taskbar

Programming Language

Python

Module/Software/IOOS data service?

IOOS ERDDAP, arcGIS feature service

Module/Software/IOOS data service?

https://github.com/MathewBiddle/sandbox/blob/main/notebooks/union_RA_boundary_and_sanctuaries.ipynb

Provide a description.

I keep hearing about collaborations with IOOS and the National Marine Sanctuaries Office and I developed this notebook to find the intersection of IOOS RA boundaries and Sanctuaries boundaries. https://github.com/MathewBiddle/sandbox/blob/main/notebooks/union_RA_boundary_and_sanctuaries.ipynb Is there something we can do with this to add to the Code Lab? I'm drawing a blank on highlighting IOOS services with this effort, but I feel it should be in the code lab in some way