aws-batch: support Snakemake `--report` about cli HOT 7 CLOSED

What's the effect of the warnings? Is there useful information missing from the report? Or is just wanting to suppress the noise from Snakemake?

The generated report does not include any runtime info:

from cli.

1. Include .snakemake/metadata in the downloads from AWS Batch so that users can generate the Snakemake report locally.

We should do this, per above. I'll open a PR.

2. Automatically generate the Snakemake report within the AWS Batch job so that users can download the rendered report

[2] definitely seems like the nicer option and maybe should be applied across all runtimes for nextstrain build?

We could also do this as well, but it requires a little more consideration about how/where/when. Would you open it as a separate issue if you'd like to see it?

from cli.

2. Automatically generate the Snakemake report within the AWS Batch job so that users can download the rendered report
   [2] definitely seems like the nicer option and maybe should be applied across all runtimes for nextstrain build?

We could also do this as well, but it requires a little more consideration about how/where/when. Would you open it as a separate issue if you'd like to see it?

Hmm, maybe this doesn't need to be built into the Nextsrain CLI. It could just be a separate step in the pathogen-repo-build workflow so we have reports for our automated pathogens.

from cli.

Hmm. Downloading the Snakemake state locally may fix this problem, but it can/will cause other problems. I don't know if it'd be ok if we scope it down to not all of .snakemake/ but just .snakemake/metadata/ as you suggest in 1.

What's the effect of the warnings? Is there useful information missing from the report? Or is just wanting to suppress the noise from Snakemake?

from cli.

Ah, looking more closely at the contents of .snakemake/metadata/, I do think we want to start downloading it by default. It's mostly info used to determine if Snakemake needs to re-run rules based on inputs/outputs, and thus is akin to the file mtimes which we already preserve on download.

from cli.

This will also need a new docker-base image, as the same exclusions of .snakemake/ are recapitulated there:

https://github.com/nextstrain/docker-base/blob/ccac0787cbc6118d0e39518e2220f650e121b8bd/entrypoint-aws-batch#L43-L44

from cli.

It could just be a separate step in the pathogen-repo-build workflow so we have reports for our automated pathogens.

Totally.

from cli.