Apparently the latest builds of the QIIME 2 Vbox image aren't compatible with the latest VirtualBox. I just checked and I built 2017.9 using VBox 5.1.22 r115126. Moving forward, should we try and use the latest VBox? As well, should we update https://s3-us-west-2.amazonaws.com/qiime2-data/distro/core/virtualbox-images.txt to include the compatible version of VBox?

We should only need to build from ISO once per dev machine, ever. The provisioning scripts should handle updates.

#28 removed Workshop Infrastructure provisioning. Ideally, this will live in it's own repo, and needs to be fixed by the June LV Workshop. The scripts were removed as part of #28, and likely don't work as-is (they were last used to install QIIME 2.0.6), so setting up that new repo requires a bit of housekeeping to get things rolling. This issue is to serve as a reminder.

• Don't provision a new cert if one already exists
• Backup existing certs

Maybe this should be a dynamically allocated disk, say 500 GB?

Resizing a VMDK is a major pain.

The VirtualBox build now uses Ubuntu 16.04.3 instead of 16.04.2. It'd be good to also build the QIIME 2 AMI using the same version of Ubuntu.

Bug Description
Using feature classifier in Docker with multiple threads may cause this error:

[Errno 28] No space left on device

This is because the default JOBLIB_TEMP_FOLDER for sklearn is /dev/shm which is limited to 64MB by docker, which will cause the error above.

Proposed Behavior
Please considering add following configuration to your Docker file which will fix the problem:

ENV JOBLIB_TEMP_FOLDER=/tmp

Bug Description
While building the VBox images, if any errors are encountered, the packer job will continue working, instead of halting.

Steps to reproduce the behavior

1. Build the VBox image with an invalid release identifier --- when the script gets to the q2studio portion, it will fail, but continue to build.

Expected behavior
Any error should halt the build process.

Example Logs

    virtualbox-ovf: --2018-09-18 04:02:17--  https://codeload.github.com/qiime2/q2studio/zip/2018.8.0
    virtualbox-ovf: Resolving codeload.github.com (codeload.github.com)... 192.30.253.121, 192.30.253.120
    virtualbox-ovf: Connecting to codeload.github.com (codeload.github.com)|192.30.253.121|:443... connected.
    virtualbox-ovf: HTTP request sent, awaiting response... 404 Not Found
    virtualbox-ovf: 2018-09-18 04:02:17 ERROR 404: Not Found.
    virtualbox-ovf:
    virtualbox-ovf:   End-of-central-directory signature not found.  Either this file is not
    virtualbox-ovf:   a zipfile, or it constitutes one disk of a multi-part archive.  In the
    virtualbox-ovf: Archive:  q2studio-2018.8.0.zip
    virtualbox-ovf:   latter case the central directory and zipfile comment will be found on
    virtualbox-ovf:   the last disk(s) of this archive.
    virtualbox-ovf: unzip:  cannot find zipfile directory in one of q2studio-2018.8.0.zip or
    virtualbox-ovf:         q2studio-2018.8.0.zip.zip, and cannot find q2studio-2018.8.0.zip.ZIP, period.
    virtualbox-ovf: /tmp/script_9560.sh: line 32: cd: q2studio-2018.8.0: No such file or directory
    virtualbox-ovf: npm WARN enoent ENOENT: no such file or directory, open '/opt/package.json'
    virtualbox-ovf: npm WARN opt No description
    virtualbox-ovf: npm WARN opt No repository field.
    virtualbox-ovf: npm WARN opt No README data
    virtualbox-ovf: npm WARN opt No license field.
    virtualbox-ovf: npm ERR! Linux 4.4.0-87-generic
    virtualbox-ovf: npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "build"
    virtualbox-ovf: npm ERR! node v7.10.1
    virtualbox-ovf: npm ERR! npm  v4.2.0
    virtualbox-ovf: npm ERR! path /opt/package.json
    virtualbox-ovf: npm ERR! code ENOENT
    virtualbox-ovf: npm ERR! errno -2
    virtualbox-ovf: npm ERR! syscall open
    virtualbox-ovf:
    virtualbox-ovf: npm ERR! enoent ENOENT: no such file or directory, open '/opt/package.json'
    virtualbox-ovf: npm ERR! enoent ENOENT: no such file or directory, open '/opt/package.json'
    virtualbox-ovf: npm ERR! enoent This is most likely not a problem with npm itself
    virtualbox-ovf: npm ERR! enoent and is related to npm not being able to find a file.
    virtualbox-ovf: npm ERR! enoent
    virtualbox-ovf:
    virtualbox-ovf: npm ERR! Please include the following file with any support request:
    virtualbox-ovf: npm ERR!     /home/ubuntu/.npm/_logs/2018-09-18T04_02_18_479Z-debug.log
    virtualbox-ovf: bash: line 0: cd: /opt/q2studio-2018.8.0/: No such file or directory
    virtualbox-ovf: Directory '.' is not installable. File 'setup.py' not found.
    virtualbox-ovf: --2018-09-18 04:02:19--  https://raw.githubusercontent.com/qiime2/logos/master/raster/white/qiime2-square-100.png
    virtualbox-ovf: Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.48.133
    virtualbox-ovf: Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.48.133|:443... connected.
    virtualbox-ovf: HTTP request sent, awaiting response... 200 OK
    virtualbox-ovf: Length: 2378 (2.3K) [image/png]
    virtualbox-ovf: Saving to: ‘/usr/share/icons/hicolor/q2studio.png’

Bug Description

FYI, not a QIIME2 dev but a pipeline dev ;)
When I update from QIIME2 version 2021.8 to 2022.2, in a nextflow pipeline, the pipeline fails with any QIIME2 process.
RuntimeError: cannot cache function 'rdist': no locator available for file '/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/layouts.py'
as detailed in nf-core/ampliseq#468. This only occurs to me when using docker (in github action test) but not when locally executing it with singularity (there is runs smoothly).
Currently I have no idea what caused this.

I haven't tested version 2022.8 yet, but I intend to.

Computation Environment

• github action tests with docker, started in https://github.com/nf-core/ampliseq/blob/aed35e5b4bad907ed2bdf6587efb82ad2578e6b1/.github/workflows/ci.yml#L33

• Virtualbox
• Docker
• AWS AMI

• delete hide user ubuntu
• pin Terminal to sidebar
• init Q2 cache
• disable sudo banner
• install Chrome, make default browser, pin to sidebar
• alternative text editor (leafpad?)

Issue brought up here. The optical drive used to mount automatically in 2017.2, but appears to have regressed in 2017.4.

Currently docker images are being created with a -t qiime2/core:$(QIIME2_RELEASE) \, which doesn't allow for simply using docker pull qiime2/core to get the most recent version. Suggesting to possibly create a :VERSION and :latest, however that workflow goes.

jorden@ip-123-45-6-78:~$ docker pull qiime2/core
Using default tag: latest
Pulling repository docker.io/qiime2/core
Tag latest not found in repository docker.io/qiime2/core

edit:

Adding a second -t qiime2/core:latest here will resolve the issue: https://docs.docker.com/engine/reference/commandline/build/#tag-an-image--t

Improvement Description
Many EC2 types are EBS-backed, and having an AMI for that would be pretty great.

References
This recently came up on the forum.

This prevents the build OVF files from being loaded into the build context

This recently came up on the forum.

There a few other steps necessary to building VBox images, mostly surrounding setting up the install ISO.

This issue is to include the q2studio desktop icon into the virtualbox build. The instructions for packaging q2studio and creating the icon are attached.
qiime_desktop_icon_workflow.docx

• workshops
• ami
• vb
• docker
• shared