pod5:Empty queue or timeout about pod5-file-format HOT 20 CLOSED

POD5_DEBUG=1 pod5 convert fast5 ./fast5/*.fast5 --output debug_pod5/

2023-06-27 20:21:44,357 DEBUG 66:'terminate_processes': ... SpawnProcess-11, stopped[SIGSEGV] daemon ...

--- Finishing previous file FAQ32498_pass_09083b73_65.fast5
2023-06-27 20:11:05,414 DEBUG 53:'convert_fast5_file':Done:37.193s
2023-06-27 20:11:05,425 DEBUG 53:'convert_fast5_file':Returned:4000
2023-06-27 20:11:05,427 INFO Enqueueing file end: FAQ32498_pass_09083b73_65.fast5 reads: 4000
2023-06-27 20:11:05,428 DEBUG c7:'enqueue_data'

--- Getting next file FAQ32498_pass_09083b73_71.fast5
2023-06-27 20:11:05,430 DEBUG 56:'get_input':(<pod5.tools.pod5_convert_from_fast5.QueueManager object at 0x7f8b6c3b5b10>,), {}
2023-06-27 20:11:05,430 DEBUG 56:'get_input':Done:0.000s
2023-06-27 20:11:05,430 DEBUG 56:'get_input':Returned:test/FAQ32498_pass_09083b73_71.fast5

--- Testing is_multi_read_fast5 on FAQ32498_pass_09083b73_71.fast5
2023-06-27 20:11:05,431 DEBUG 72:'is_multi_read_fast5':(PosixPath('test/FAQ32498_pass_09083b73_71.fast5'),), {}

--- Segfault

2023-06-27 20:10:26,479 DEBUG fd:'is_multi_read_fast5':(PosixPath('test/FAQ32498_pass_09083b73_65.fast5'),), {}
2023-06-27 20:10:28,220 DEBUG fd:'is_multi_read_fast5':Done:1.741s
2023-06-27 20:10:28,220 DEBUG fd:'is_multi_read_fast5':Returned:True

from pathlib import Path
path = Path("/home/user/ydliu/hitbic/HG002/test/FAQ32498_pass_09083b73_71.fast5")
assert path.exists()
print("666")

import h5py
h5 = h5py.File(path)
print("777")

assert len(h5) > 0
print("888")

from pod5.tools.pod5_convert_from_fast5 import is_multi_read_fast5
print(is_multi_read_fast5(path))

print("start")
with h5py.File(path) as _h5:
    print("open")           
    print(_h5)

    _h5.attrs
    print("can access_h5.attrs")
    print(_h5.attrs)

    # The "file_type" attribute might be present on supported multi-read fast5 files.
    if _h5.attrs.get("file_type") == "multi-read":
        print("True")
        # return True
    print( "is not multi-read file type")

    if len(_h5) == 0:
        print("True")
        # return True
    print( "is not len 0")

    # if there are "read_x" keys, this is a multi-read file
    if any(key for key in _h5 if key.startswith("read_")):
        print("found a read")
        # return True

    print("closing handle")
print("everything is fine?!")

start
open
<HDF5 file "FAQ32498_pass_09083b73_71.fast5" (mode r)>
can access_h5.attrs
<Attributes of HDF5 object at 139974581599904>
is not multi-read file type
is not len 0
Traceback (most recent call last):
  File "test.py", line 40, in <module>
    if any(key for key in _h5 if key.startswith("read_")):
  File "test.py", line 40, in <genexpr>
    if any(key for key in _h5 if key.startswith("read_")):
  File "/home/user/ydliu/miniconda3/envs/remora/lib/python3.8/site-packages/h5py/_hl/group.py", line 499, in __iter__
    for x in self.id.__iter__():
  File "h5py/h5g.pyx", line 128, in h5py.h5g.GroupIter.__next__
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5l.pyx", line 316, in h5py.h5l.LinkProxy.iterate
RuntimeError: Link iteration failed (incorrect metadata checksum after all read attempts)

usage: pod5 subset [-h] [-o OUTPUT] [-r] [-f] [-t THREADS] [--csv CSV]
                   [-s TABLE] [-R READ_ID_COLUMN] [-c COLUMNS [COLUMNS ...]]
                   [--template TEMPLATE] [-T] [-M] [-D]
                   inputs [inputs ...]

Given one or more pod5 input files, take subsets of reads into one or more pod5 output files by a user-supplied mapping.

positional arguments:
  inputs                Pod5 filepaths to use as inputs

optional arguments:
  -h, --help            show this help message and exit
  -o OUTPUT, --output OUTPUT
                        Destination directory to write outputs (default:
                        /home/user/ydliu/hitbic/HG002)
  -r, --recursive       Search for input files recursively matching `*.pod5`
                        (default: False)
  -f, --force-overwrite
                        Overwrite destination files (default: False)
  -t THREADS, --threads THREADS
                        Number of subsetting workers (default: 8)

direct mapping:
  --csv CSV             CSV file mapping output filename to read ids (default:
                        None)

table mapping:
  -s TABLE, --summary TABLE, --table TABLE
                        Table filepath (csv or tsv) (default: None)
  -R READ_ID_COLUMN, --read-id-column READ_ID_COLUMN
                        Name of the read_id column in the summary (default:
                        read_id)
  -c COLUMNS [COLUMNS ...], --columns COLUMNS [COLUMNS ...]
                        Names of --summary / --table columns to subset on
                        (default: None)
  --template TEMPLATE   template string to generate output filenames (e.g.
                        "mux-{mux}_barcode-{barcode}.pod5"). default is to
                        concatenate all columns to values as shown in the
                        example. (default: None)
  -T, --ignore-incomplete-template
                        Suppress the exception raised if the --template string
                        does not contain every --columns key (default: None)

content settings:
  -M, --missing-ok      Allow missing read_ids (default: False)
  -D, --duplicate-ok    Allow duplicate read_ids (default: False)

Example: pod5 subset inputs.pod5 --output subset_mux/ --summary summary.tsv --columns mux```