caporaso-lab / developing-with-qiime2 Goto Github PK

Now that we're set up with Read The Docs (RTD; https://develop.qiime2.org), we can stop hosting through GitHub pages. Ideally we'll have a index.html page or a permanent redirect at the old location (https://cap-lab.bio/developing-with-qiime2) that redirects to the new canonical URL (https://develop.qiime2.org).

• Delete the requirements.txt file as part of the PR for this issue, as that isn't needed by RTD (which uses the environment.yml file).
• Delete the GitHub Actions workflow.

Also, review all content to see what this adds over the plugin tutorial content on this topic.

I forgot that this existed:
https://docs.qiime2.org/2023.2/plugins/developing/#overview

Ultimately we'll transition all content to MyST Markdown. In the meantime we can embed rst in MyST Markdown files. This document discusses how to document APIs with MyST Markdown using sphinx-autodoc2.

Add guidance on current recommendations for distributing QIIME 2 plugins, including a sketch of the plans for where we are going with library.qiime2.org.

Forum x-ref

A good interface tutorial could be to develop a "documentation engine", that ultimately could be used to generate plugin documentation for all plugins in a distribution, like the current Available Plugins page in the user docs. This would use the sdk to load the PluginManager, and then explore the different functionality to find the available plugins and write documentation of those plugins. Following development of this, I could fork the example interface repository (caporaso-lab/q2docgen?), and make it available to plugin developers for auto-generating their plugin user documentation.

https://dev.qiime2.org/latest/quickstart/

This work was funded in part by NIH National Cancer Institute Informatics Technology for Cancer Research grant 1U24CA248454-01, and by grant DAF2019-207342 from the Chan Zuckerberg Initiative (CZI) DAF, an advised fund of Silicon Valley Community Foundation (CZI grant DOI: 10.37921/862772dbrrej; funder DOI 10.13039/100014989).

This book is built with MyST Markdown and Jupyter Book, which are supported in part with funding from the Alfred P. Sloan Foundation.

Initial support for the development of QIIME 2 was provided through a grant from the National Science Foundation.

https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en

This will build from this example: https://github.com/colinvwood/q2-r-example

Pending qiime2/sphinx-ext-qiime2#24

• Quickstart
• Architecture Overview
• The Plugin Object and Entrypoints
• Actions
• Parallelizing QIIME 2 Pipelines
• Pipeline Resumption in QIIME 2
• How Data is Stored
• Using Metadata
• Coordinating With Other Plugins
• Testing Plugins
• Publishing a Plugin on Library (still relevant?)
• Plugin tutorials (ported Usage Examples doc to a how-to - the others will be adapted into tutorials, but doing that later to figure out the right transition from the existing quickstart tutorial)
• Internal Details
• API Reference (see here)
• Glossary
• Getting Help
• remove note on front page referencing old dev docs
• host this book at dev.qiime2.org

Closely related to qiime2/q2-types#288.

That way when we address #38, they will appear in the docs, and in the meantime the source code links will at least be there.

Currently the Next link at the bottom of the final plugin tutorial section jumps right to How-tos, which will confuse readers:

Add a brief conclusion document.

What they are, how they differ from semantic types, and our movement away from using artifact class and semantic type as synonyms (which we previously did in some contexts, before we started using the name artifact class).

https://jupyterbook.org/en/stable/content/content-blocks.html#define-substitutions-for-your-whole-book

Many of these either don't have doc strings, or have some formatting in the doc strings that doesn't align with either the Numpy or Google conventions. As part of this effort, we should choose whether we prefer the Numpy or Google convention and use that throughout.

This work will primarily happen in the QIIME 2 framework codebase, though there should be content updates in DWQ2 as new API documentation is produced.

This, in _config.yml, isn't working:

parse:
  myst_links_external_new_tab: true

I also don't seem to be able to change other variables from their defaults, such as:

parse:
  myst_all_links_external: true

based on this logging output after running make clean && make html:

myst v0.18.1: MdParserConfig(commonmark_only=False,
  gfm_only=False,
  enable_extensions=['colon_fence', 'dollarmath', 'linkify', 'substitution', 'tasklist'],
  disable_syntax=[],
  all_links_external=False,
  url_schemes=['mailto', 'http', 'https'],
  ref_domains=None,
  highlight_code_blocks=True,
  number_code_blocks=[],
  title_to_header=False,
  heading_anchors=None,
  heading_slug_func=None,
  footnote_transition=True,
  words_per_minute=200,
  sub_delimiters=('{', '}'),
  linkify_fuzzy_links=True,
  dmath_allow_labels=True,
  dmath_allow_space=True,
  dmath_allow_digits=True,
  dmath_double_inline=False,
  update_mathjax=True,
  mathjax_classes='tex2jax_process|mathjax_process|math|output_area')

The same behavior was observed when building with myst-parser 2.0.0.

(I don't want to overwrite myst_all_links_external - just testing.)

This was moved from qiime2/dev-docs and temporarily stored here, but ultimately should move as it's a user doc, not a dev doc.

The cookiecutter plugin template is under BSD 3-clause, while the book is under CC BY-NC-ND 4.0 - make it clear that these are different.

Should be something like the following (adapted from the content in qiime2/dev-docs#50):

plugin.register_formats(EMPPairedEndDirFmt)
plugin.register_semantic_types(EMPPairedEndSequences)
plugin.register_artifact_class(EMPPairedEndSequences, artifact_format=EMPPairedEndDirFmt)

Porting issue from qiime2/dev-docs#15.

Porting issue from qiime2/dev-docs#15.

• use transform_format (q2-sapienns has a good example)
• to know the to and from values to provide as input
  • export the artifact that you would have used in your test using qiime tools export. the format that is reported on the command line should be used as to (alternatively you can find this using qiime tools peek- it's the Data format that is reported)
  • the view type that your function takes as input (or which you would have passed as view_type to Artifact.load("...").view(view_type)) should be provided as from
• generally this is a good way to go as:
  • it makes for easier test developing and debugging since you can see the test data more easily, and
  • generally speaking (though not in all cases - e.g., if the underlying format is binary) building your tests on data files (or objects) makes for more informative diffs if/when the test data changes across commits.

Here's a good example of this being done:

https://github.com/qiime2/q2-demux/blob/59b2378592acd7cd9d003fe776b5b469cf6bfa67/q2_demux/plugin_setup.py#L170
https://github.com/qiime2/q2-demux/blob/dev/q2_demux/_demux.py#L514