Commas in Sample IDs break the internal manifests for per-sample fastq formats about q2-types HOT 4 OPEN

👍 for updating the places where qiime2 is reading/writing manifests so they're compatible with commas. There'll still be the opportunity for users to run into the issue when they're writing their own manifest file (which they'll need to quote themselves), so some docs on the importing tutorial may be useful in addition to the q2-types updates. Does that sound good to you?

from q2-types.

Sounds good to me (updating q2-types and documenting the edge case)!

from q2-types.

I'm not sure we can do much about that since the format is CSV. Users will need to quote any fields containing commas to "escape" the delimiter. Do you think describing this edge-case in the "fastq manifest" part of the importing tutorial would resolve the issue?

from q2-types.

Couldn't we start emitting quoted fields and/or escaped fields in the internal manifest? It looks like we are already using pandas.read_csv, to read those manifests back into Q2, which should be able to handle quoted vs non-quoted editions of these files. I think we could do that in a backwards-compatible way, but maybe I am missing something! One gotcha is that there are a few places in the ecosystem where we grab the manifest and do some things with it, so depending on how that is currently implemented, there could be some unintended consequences 👿 .

I am up for documenting the edge case if we don't have a quick fix for this!

from q2-types.