Hi,
Do you plan to integrate genome assmbly module into nanopype ?

Prerequisites

• [x ] Put an X between the brackets on this line if you have done all of the following:
  • Read the contributions guide at: https://nanopype.readthedocs.io/en/latest/contribute
  • Checked that your issue isn't already filed: https://github.com/giesselmann/nanopype/issues

Description

Steps to Reproduce

1. per instructions on https://nanopype.readthedocs.io/en/latest/installation/src/
   create nanotype conda environment and checked out Nanopype
   conda create -n nanopype python=3.6 anaconda
   source activate nanopype
   git clone https://github.com/giesselmann/nanopype
   cd nanopype
   git fetch --tags && git checkout v1.0.0
   python3 -m pip install --upgrade pip
   python3 -m pip install -r requirements.txt
   cd ..
2. Running snamemake commands there is en error when building vbz_compression.
   Command: snakemake --snakefile rules/install.smk --directory ~/nanopype vbz_compression
   Partial Error:

    [Thu Apr 15 18:24:28 2021]
    rule vbz_compression:
        output: lib/libvbz_hdf_plugin.so.1.0.1
        jobid: 0
    fatal: destination path 'vbz_compression' already exists and is not an empty directory.
    CMake Error: The source directory "/data/homezvol1/courtnkc/nanopype/src" does not appear to contain CMakeLists.txt.
    Specify --help for usage, or press the help button on the CMake GUI.
[Thu Apr 15 18:24:28 2021]
Error in rule vbz_compression:
    jobid: 0
    output: lib/libvbz_hdf_plugin.so.1.0.1
    shell:
        install_prefix=`pwd`
        mkdir -p lib
        mkdir -p src && cd src
        if [ ! -d singularity ]; then
            git clone https://github.com/nanoporetech/vbz_compression --recursive --branch v1.0.1 --depth=1 && cd vbz_compression
        else
            cd vbz_compression && git fetch --all --tags --prune && git checkout tags/v1.0.1
        fi
        rm -rf build
        mkdir build && cd build
        cmake -D CMAKE_BUILD_TYPE=Release -D ENABLE_CONAN=OFF -D ENABLE_PERF_TESTING=OFF -D ENABLE_PYTHON=OFF ..
        make
        cp bin/libvbz_hdf_plugin.so ${install_prefix}/lib/libvbz_hdf_plugin.so.1.0.1
        
        (exited with non-zero exit code)

This error results from a typo in nanopype/rules/install.smk file. For the "rule vbz_compression:" target
lIne

if [ ! -d singularity ]; then
should be changed to
if` [ ! -d vbz_compression ]; then

Then the closing of the repo works properly.

3. Once above typo is corrected, a new error indicates that the build process for vbz_compression
   cant find libzstd library during running cmake command. Partial error :

        -- HDF5: Using hdf5 compiler wrapper to determine C configuration
        -- Found HDF5: /data/homezvol1/.conda/envs/nanopype/lib/libhdf5.so;/usr/lib64/librt.so;/usr/lib64/libpthread.so;/
data/homezvol1/.conda/envs/nanopype/lib/libz.so;/usr/lib64/libdl.so;/usr/lib64/libm.so (found suitable version "1.10.4", 
minimum required is "1.8.16")  
        CMake Error at /opt/foundation/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
        Could NOT find zstd (missing: ZSTD_LIBRARY ZSTD_INCLUDE_DIR) (Required is
        at least version "1.3.1")
        Call Stack (most recent call first):
        /opt/foundation/share/cmake-3.12/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
        cmake/Findzstd.cmake:36 (find_package_handle_standard_args)
        CMakeLists.txt:139 (find_package)

The need library does exist in the user's conda nanopype environment and is at acceptable higher version 1.4.5
Please note, that the came conda nanopype environment provides HDF5 library and it is found just fine, yet the
libzstd.so in the same directory is not found.

To mitigate, I had to manually set

     export CMAKE_INCLUDE_PATH=$CONDA_PREFIX/include:$CMAKE_INCLUDE_PATH
     export CMAKE_LIBRARY_PATH=$CONDA_PREFIX/lib:$$CMAKE_LIBRARY_PATH

After that the compilation proceeded without an error.

Expected behavior:

Some prerequisites are provided via system RPMS, some from the conda environment. None were missing.
The install.smk file should take into account conda environment and set appropriate definitions
for cmake or any other commands. One should not need to patch environment when running tools installation.

Please also consider that many users who need to install nanotype are not software developers or installers
and cant really debug many of the steps of this install. A proper check for the prerequisites fulfilled and
better error handling d would be helpful to such users.

Versions

whatever is in the master branch currently for nanopype 1.1.0

Additional Information

Log Files

• Put an X between the brackets on this line if you have attached log files from the observed behavior.

Hi,
I'm testing Nanopype and it seems to be a very nice tool.
However, I'm facing an unexpected behaviour when running twice the methylation analysis: when I run it for the first time and ask a bedGraph, it uses the existing alignment and proceeds from from the nanopolish step, however, if I run it a second time and ask for a bw file, it reruns the alignment step and everything from there.
I newbie in Snakemake, but if I understood it right, snakemake only rerun things if the modification time of input files are more recent than the one of output files, or if an intermediate file is required and are no longer available. I tried to understand if any input file was changed during the methylation analysis, but I couldn't find anything changing it (from my inexperience point of view). I looked if there were any output file flagged as temporary and there were (.fofn), but those are produced after the nanopolish step, which should not have triggered the mapping step.
What do you thing that could be causing this behaviour?
Thanks in advance.
Miguel

P.s.1: I attached the dag files of both runs.
P.s.2: I've made a few changes and added a few things to the code (that I'm still testing locally) that improves the flexibility and cluster usage. Can I do a pull request? Thanks again.

Summary

It would be useful to have a test dataset of barcodes reads in order to test the demux functioning of nanopype.

Motivation

We would like to multiplex runs, and it would be helpful to run some test demux'ing with nanopype.

Describe alternatives you've considered

Running guppy_barcoder directly

Hello, I had a problem when I configured the software following the Installation Source guide command:
snakemake --snakefile path/to/install.smk --directory [INSTALL_PREFIX] all -j 8 --config threads_build=8
The ERROR reture as " (exited with non-zero exit code)".
How can I fit the problem?

Prerequisites

• Put an X between the brackets on this line if you have done all of the following:
  • Read the contributions guide at: https://nanopype.readthedocs.io/en/latest/contribution
  • Checked that your issue isn't already filed: https://github.com/giesselmann/nanopype/issues

Description

Hi,
I'm trying to obtain the report.pdf from my run, but it stops with an error in the report_alignments_stats checkpoint (see bellow). Before I run it with ngmlr and it worked, but now with minimap2 it's breaking.
I hope you can help me. Thanks in advance.
Miguel

P.s.: I submitted the issue before I finish writing it (I needed to edit)

Steps to Reproduce

1. Regular Nanopype run with ngmlr (WORKED):

snakemake --snakefile ngmlr/nanopype/Snakefile --directory ngmlr/nanopype/ --profile ngmlr/nanopype/profiles/slurm/ --use-singularity --singularity-args '-B /data/storage_data_raw/lr_HEL_experiment -B /data/reference' --printshellcmds --reason --local-cores 1 methylation/nanopolish/ngmlr/guppy/HEL.5x.GRCh38_p12_Release_96.bedGraph

2. Nanopype report for ngmlr (WORKED):

snakemake --snakefile ngmlr/nanopype/Snakefile --directory ngmlr/nanopype/ --profile ngmlr/nanopype/profiles/slurm/ --use-singularity --printshellcmds --reason --local-cores 1 report.pdf

3. Regular Nanopype run with minimap2 (WORKED):

snakemake --snakefile minimap2/nanopype/Snakefile --directory minimap2/nanopype/ --profile minimap2/nanopype/profiles/slurm/ --use-singularity --singularity-args '-B /data/storage_data_raw/lr_HEL_experiment -B /data/reference' --printshellcmds --reason --local-cores 1 methylation/nanopolish/minimap2/guppy/HEL.5x.GRCh38_p12_Release_96.bedGraph

4. Nanopype report for minimap2 (NOT WORKED):

snakemake --snakefile minimap2/nanopype/Snakefile --directory minimap2/nanopype/ --profile minimap2/nanopype/profiles/slurm/ --use-singularity --printshellcmds --reason --local-cores 1 report.pdf

Expected behavior:

Nanopype to produce the report.pdf file.

Actual behavior:

Got the following error:

[Sun Aug 16 10:11:17 2020]
localcheckpoint report_alignments_stats:
    input: alignments/minimap2/guppy/batches/HEL/20191120_FAL66483_FLO-MIN106D_SQK-RAD004_HEL_original_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191127_FAL66209_FLO-MIN106D_SQK-RAD004_HEL_original_2ndrun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191203_FAL66972_FLO-MIN106D_SQK-RAD004_HEL_purified_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20200122_FAL66375_FLO-MIN106D_SQK-RAD004_HEL_overload.GRCh38_p12_Release_96.hdf5
    output: report/plots/alignments/
    jobid: 3
    reason: Missing output files: report/plots/alignments/; Input files updated by another job: alignments/minimap2/guppy/batches/HEL/20191127_FAL66209_FLO-MIN106D_SQK-RAD004_HEL_original_2ndrun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191120_FAL66483_FLO-MIN106D_SQK-RAD004_HEL_original_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20200122_FAL66375_FLO-MIN106D_SQK-RAD004_HEL_overload.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191203_FAL66972_FLO-MIN106D_SQK-RAD004_HEL_purified_1strun.GRCh38_p12_Release_96.hdf5
Downstream jobs will be updated after completion.

/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/base.py:2456: RuntimeWarning: '<' not supported between instances of 'tuple' and 'int', sort order is undefined for incomparable objects
  return Index.union(this, other, sort=sort).astype(object, copy=False)
[Sun Aug 16 10:12:26 2020]
Error in rule report_alignments_stats:
    jobid: 3
    output: report/plots/alignments/

RuleException:
TypeError in line 241 of /path/minimap2/nanopype/rules/report.smk:
object of type 'int' has no len()
  File "/path/minimap2/nanopype/rules/report.smk", line 241, in __rule_report_alignments_stats
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 287, in concat
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 498, in get_result
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 2325, in reindex
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 501, in from_tuples
  File "pandas/_libs/lib.pyx", line 2455, in pandas._libs.lib.tuples_to_object_array
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/concurrent/futures/thread.py", line 57, in run
Removing output files of failed job report_alignments_stats since they might be corrupted:
report/plots/alignments/

Versions

1.0.0

Additional Information

Let me know if you need anything else.

Log Files

• Put an X between the brackets on this line if you have attached log files from the observed behavior.
  20200816_10_13_34_933237.nanopype.log
  2020-08-16T093542.850487.snakemake.log

Hey Pay,
I am facing some problems during the installation of the dependencies with snakemake.
I received the following error message:

(nanopype) epicwl@epicwl-D3348-B2:~/pipelines/nanopype$ snakemake --snakefile rules/install.smk --directory ../.. all
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 Flappie
1 OpenBLAS
1 UCSCtools
1 all
1 bedtools
1 deepbinner
1 gitlfs
1 graphmap
1 hdf5
1 htslib
1 minimap2
1 nanopolish
1 ngmlr
1 samtools
1 sniffles
15

[Tue Feb 26 11:06:36 2019]
rule gitlfs:
input: /usr/bin
output: bin/git-lfs
jobid: 11

Fetching origin
remote: Enumerating objects: 36758, done.
remote: Counting objects: 100% (36758/36758), done.
remote: Compressing objects: 100% (10216/10216), done.
remote: Total 36272 (delta 25821), reused 35745 (delta 25309), pack-reused 0
Receiving objects: 100% (36272/36272), 9.48 MiB | 13.07 MiB/s, done.
Resolving deltas: 100% (25821/25821), completed with 394 local objects.
From https://github.com/git-lfs/git-lfs

• [new tag] 2.0.2 -> 2.0.2
• [new tag] v0.1.0 -> v0.1.0
• [new tag] v0.2.0 -> v0.2.0
• [new tag] v0.2.1 -> v0.2.1
• [new tag] v0.2.1-p1 -> v0.2.1-p1
• [new tag] v0.2.2 -> v0.2.2
• [new tag] v0.2.3 -> v0.2.3
• [new tag] v0.3.0 -> v0.3.0
• [new tag] v0.3.1 -> v0.3.1
• [new tag] v0.3.2 -> v0.3.2
• [new tag] v0.3.3 -> v0.3.3
• [new tag] v0.3.4 -> v0.3.4
• [new tag] v0.3.5 -> v0.3.5
• [new tag] v0.3.6 -> v0.3.6
• [new tag] v0.4.0 -> v0.4.0
• [new tag] v0.4.1 -> v0.4.1
• [new tag] v0.4.2 -> v0.4.2
• [new tag] v0.4.2.1 -> v0.4.2.1
• [new tag] v0.5.0 -> v0.5.0
• [new tag] v0.5.0.pre1 -> v0.5.0.pre1
• [new tag] v0.5.0.pre2 -> v0.5.0.pre2
• [new tag] v0.5.1 -> v0.5.1
• [new tag] v0.5.1-tracing -> v0.5.1-tracing
• [new tag] v0.5.2 -> v0.5.2
• [new tag] v0.5.3 -> v0.5.3
• [new tag] v0.5.4 -> v0.5.4
• [new tag] v0.5.4-homebrew -> v0.5.4-homebrew
• [new tag] v0.6.0 -> v0.6.0
• [new tag] v1.0.0 -> v1.0.0
• [new tag] v1.0.1 -> v1.0.1
• [new tag] v1.0.2 -> v1.0.2
• [new tag] v1.1.0 -> v1.1.0
• [new tag] v1.1.1 -> v1.1.1
• [new tag] v1.1.1-pre-push-tracing -> v1.1.1-pre-push-tracing
• [new tag] v1.1.2 -> v1.1.2
• [new tag] v1.2.0 -> v1.2.0
• [new tag] v1.2.1 -> v1.2.1
• [new tag] v1.3.0 -> v1.3.0
• [new tag] v1.3.1 -> v1.3.1
• [new tag] v1.4.0 -> v1.4.0
• [new tag] v1.4.1 -> v1.4.1
• [new tag] v1.4.2 -> v1.4.2
• [new tag] v1.4.3 -> v1.4.3
• [new tag] v1.4.4 -> v1.4.4
• [new tag] v1.5.0 -> v1.5.0
• [new tag] v1.5.1 -> v1.5.1
• [new tag] v1.5.2 -> v1.5.2
• [new tag] v1.5.2-go1.8beta1 -> v1.5.2-go1.8beta1
• [new tag] v1.5.3 -> v1.5.3
• [new tag] v1.5.4 -> v1.5.4
• [new tag] v1.5.5 -> v1.5.5
• [new tag] v1.5.6 -> v1.5.6
• [new tag] v2.0.0 -> v2.0.0
• [new tag] v2.0.1 -> v2.0.1
• [new tag] v2.0.2 -> v2.0.2
• [new tag] v2.0.2-rc1 -> v2.0.2-rc1
• [new tag] v2.1.0 -> v2.1.0
• [new tag] v2.1.1 -> v2.1.1
• [new tag] v2.2.0 -> v2.2.0
• [new tag] v2.2.1 -> v2.2.1
• [new tag] v2.2.2-example -> v2.2.2-example
• [new tag] v2.3.0 -> v2.3.0
• [new tag] v2.3.1 -> v2.3.1
• [new tag] v2.3.2 -> v2.3.2
• [new tag] v2.3.3 -> v2.3.3
• [new tag] v2.3.4 -> v2.3.4
• [new tag] v2.4.0 -> v2.4.0
• [new tag] v2.4.1 -> v2.4.1
• [new tag] v2.4.2 -> v2.4.2
• [new tag] v2.5.0 -> v2.5.0
• [new tag] v2.5.1 -> v2.5.1
• [new tag] v2.5.2 -> v2.5.2
• [new tag] v2.6.1 -> v2.6.1
• [new tag] v2.7.0 -> v2.7.0
  HEAD is now at 93702d75 Merge pull request #3355 from bk2204/bump-version-2.6.0
  can't load package: package github.com/git-lfs/git-lfs/commands: cannot find package "github.com/git-lfs/git-lfs/commands" in any of:
  /usr/lib/go-1.10/src/github.com/git-lfs/git-lfs/commands (from $GOROOT)
  /home/epicwl/go/src/github.com/git-lfs/git-lfs/commands (from $GOPATH)
  template: main:1:31: executing "main" at <.Module>: can't evaluate field Module in type *load.PackagePublic
  git-lfs: skipping fmt, no goimports found at goimports ...
  go generate github.com/git-lfs/git-lfs/commands
  can't load package: package github.com/git-lfs/git-lfs/commands: cannot find package "github.com/git-lfs/git-lfs/commands" in any of:
  /usr/lib/go-1.10/src/github.com/git-lfs/git-lfs/commands (from $GOROOT)
  /home/epicwl/go/src/github.com/git-lfs/git-lfs/commands (from $GOPATH)
  Makefile:160: recipe for target 'commands/mancontent_gen.go' failed
  make: *** [commands/mancontent_gen.go] Error 1
  [Tue Feb 26 11:06:40 2019]
  Error in rule gitlfs:
  jobid: 11
  output: bin/git-lfs

RuleException:
CalledProcessError in line 247 of /home/epicwl/pipelines/nanopype/rules/install.smk:
Command 'set -euo pipefail; export PATH=$(pwd)//usr/bin:$PATH
mkdir -p src && cd src
if [ ! -d git-lfs ]; then
git clone https://github.com/git-lfs/git-lfs.git --branch v2.6.0 --depth=1 && cd git-lfs
else
cd git-lfs && git fetch --all --tags --prune && git checkout tags/v2.6.0
fi
make
cp bin/git-lfs ../../bin' returned non-zero exit status 2.
File "/home/epicwl/pipelines/nanopype/rules/install.smk", line 247, in __rule_gitlfs
File "/home/epicwl/miniconda3/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/epicwl/.snakemake/log/2019-02-26T110636.098902.snakemake.log

I think it could be due to some of the github links....

Thanks for your support.
Best,

Joschka

Hello,
your workflow fits perfectly my needs and I really hope I can manage to get it working in our set-up, which is on a HPC, with slurm.
I could install all tools but I want to use the singularity image that would be more easy to share the tool with people here.
I don't understand how I am supposed to use this image, I created it after pulling from docker hub.
the import of data when fine:

after booking interactive resources, and activating the virtual env of nanopype

python3  ~/install/virtualenvs/nanopype/nanopype/scripts/nanopype_import.py \
  data/raw/ EM_S1/20190806_0812_MN22103_FAK07438_17f539d2/fast5

gives

20.08.2019 09:49:22 [INFO] Logger created
20.08.2019 09:49:22 [INFO] Writing output to /mnt/irisgpfs/projects/lsru/minion/mouse_embryo/data/raw/reads
20.08.2019 09:49:22 [INFO] Inspect existing files and archives
20.08.2019 09:49:22 [INFO] 0 raw files already archived
20.08.2019 09:49:22 [INFO] 41 raw files to be archived
20.08.2019 09:50:45 [INFO] Archived 41 reads in /mnt/irisgpfs/projects/lsru/minion/mouse_embryo/data/raw/reads/0.tar
20.08.2019 09:50:45 [INFO] Mission accomplished

afterwards I naively tried this after creating a slurm profile

snakemake -j 999 --cluster-config cluster.json --cluster "sbatch -c 2 -t 1" --profile slurm \
  --use-singularity --singularity-prefix ~/scratch/nanopype-v0.8.0.simg \
  --snakefile ~/install/virtualenvs/nanopype/nanopype/Snakefile data/raw/reads.fofn 

gives

/mnt/irisgpfs/users/aginolhac/install/virtualenvs/nanopype/lib/python3.7/site-packages/snakemake/workflow.py:85: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Load
er is unsafe. Please read https://msg.pyyaml.org/load for full details.
  self.globals = globals()
[WARNING] Genome for GRCm38.p3 not found in ~/lsru/minion/references/GRCm38.p3.fasta, skipping entry.
[WARNING] Genome for test not found in references/chr6.fa, skipping entry.
[WARNING] flappie not found as flappie and is only available in singularity rules.
[WARNING] bedtools not found as bedtools and is only available in singularity rules.
[WARNING] graphmap not found as graphmap and is only available in singularity rules.
[WARNING] ngmlr not found as ngmlr and is only available in singularity rules.
[WARNING] sniffles not found as sniffles and is only available in singularity rules.
[WARNING] deepbinner not found as deepbinner-runner.py and is only available in singularity rules.
[WARNING] racon not found as racon and is only available in singularity rules.
[WARNING] cdna_classifier not found as cdna_classifier.py and is only available in singularity rules.
[WARNING] spliced_bam2gff not found as spliced_bam2gff and is only available in singularity rules.
[WARNING] cluster_gff not found as cluster_gff and is only available in singularity rules.
[WARNING] collapse_partials not found as collapse_partials and is only available in singularity rules.
[WARNING] polish_clusters not found as polish_clusters and is only available in singularity rules.
[WARNING] strique not found as STRique.py and is only available in singularity rules.
RuleException in line 76 of /home/users/aginolhac/install/virtualenvs/nanopype/nanopype/rules/demux.smk:
Singularity directive is only allowed with shell, script or wrapper directives (not with run).
  File "/home/users/aginolhac/install/virtualenvs/nanopype/nanopype/Snakefile", line 231, in <module>
  File "/home/users/aginolhac/install/virtualenvs/nanopype/nanopype/rules/demux.smk", line 76, in <module>

I tried to bind the mouse genome folder in the singularity image but I don't get it right. Also, what about all the warnings concerning the binary missing? Are they not found in the singularity image?

Then it is complaining about the demux.smk that is

 singularity:
        "docker://nanopype/demux:{tag}".format(tag=config['version']['tag'])
    run:
        import os, itertools, collection

Forgot to say that I successfully run the tests when I was inside the singularity image but not when running python3 test/test_rules.py test_unit_singularity which gave the same error as trying to index the fast5 files.

Thanks in advance for your time.

Aurelien

Hi,

Thanks for having implemented a such pipeline. We would like to test it with our data but our computer are under Mac OS 10.14. But Albacore can't be installed on Mas OS 10.14 and it's archived by ONT. If I want to use it, I must create a virtual machine. I don't see if we can use Albacore software with our flowcell, recent or in the future.
So do you think adapt Nanopype with only guppy ? Thanks a lot for your answer.
Regards,
Emmanuelle

Hi,

I am trying to install flappie in ubuntu 16.04, I followed the instructions, but there are some errors:
(base) (katuali) leo@leo:~/NGS/katuali/flappie/flappie$ make
cd build &&
cmake .. -DCMAKE_BUILD_TYPE=Release -DHDF5_ROOT='' &&
make flappie
-- Configuring done
-- Generating done
-- Build files have been written to: /home/leo/NGS/katuali/flappie/flappie/build
make[1]: Entering directory '/home/leo/NGS/katuali/flappie/flappie/build'
make[2]: Entering directory '/home/leo/NGS/katuali/flappie/flappie/build'
make[3]: Entering directory '/home/leo/NGS/katuali/flappie/flappie/build'
make[4]: Entering directory '/home/leo/NGS/katuali/flappie/flappie/build'
make[4]: Leaving directory '/home/leo/NGS/katuali/flappie/flappie/build'
make[4]: Entering directory '/home/leo/NGS/katuali/flappie/flappie/build'
[ 7%] Building C object CMakeFiles/flappie_objects.dir/src/networks.c.o
In file included from /home/leo/NGS/katuali/flappie/flappie/src/networks.c:10:0:
/home/leo/NGS/katuali/flappie/flappie/src/models/flipflop_r941native.h:1:1: error: unknown type name ‘version’
version https://git-lfs.github.com/spec/v1
^
/home/leo/NGS/katuali/flappie/flappie/src/models/flipflop_r941native.h:1:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘:’ token
version https://git-lfs.github.com/spec/v1
^
/home/leo/NGS/katuali/flappie/flappie/src/models/flipflop_r941native.h:2:12: error: invalid suffix "dd6adb73a55d95014a6701658d399069b144451dff9179cbefbfb39bb230e" on integer constant
oid sha256:418dd6adb73a55d95014a6701658d399069b144451dff9179cbefbfb39bb230e
^
In file included from /home/leo/NGS/katuali/flappie/flappie/src/networks.c:11:0:
/home/leo/NGS/katuali/flappie/flappie/src/models/flipflop_r941native5mC.h:2:12: error: invalid suffix "bffdb61b0389174ce2be46ff2e41ae9efaf1332fef6638b8b8e41cf67838" on integer constant
oid sha256:2015bffdb61b0389174ce2be46ff2e41ae9efaf1332fef6638b8b8e41cf67838
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:160:16: error: ‘_conv_rnnrf_flipflop_r941native_W’ undeclared here (not in a function)
.conv_W = &_conv_rnnrf_flipflop_r941native_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:161:16: error: ‘_conv_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.conv_b = &_conv_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:162:20: error: ‘conv_rnnrf_flipflop_r941native_stride’ undeclared here (not in a function)
.conv_stride = conv_rnnrf_flipflop_r941native_stride,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:165:18: error: ‘_gruB1_rnnrf_flipflop_r941native_iW’ undeclared here (not in a function)
.gruB1_iW = &_gruB1_rnnrf_flipflop_r941native_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:166:18: error: ‘_gruB1_rnnrf_flipflop_r941native_sW’ undeclared here (not in a function)
.gruB1_sW = &_gruB1_rnnrf_flipflop_r941native_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:167:17: error: ‘_gruB1_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.gruB1_b = &_gruB1_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:169:18: error: ‘_gruF2_rnnrf_flipflop_r941native_iW’ undeclared here (not in a function)
.gruF2_iW = &_gruF2_rnnrf_flipflop_r941native_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:170:18: error: ‘_gruF2_rnnrf_flipflop_r941native_sW’ undeclared here (not in a function)
.gruF2_sW = &_gruF2_rnnrf_flipflop_r941native_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:171:17: error: ‘_gruF2_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.gruF2_b = &_gruF2_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:173:18: error: ‘_gruB3_rnnrf_flipflop_r941native_iW’ undeclared here (not in a function)
.gruB3_iW = &_gruB3_rnnrf_flipflop_r941native_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:174:18: error: ‘_gruB3_rnnrf_flipflop_r941native_sW’ undeclared here (not in a function)
.gruB3_sW = &_gruB3_rnnrf_flipflop_r941native_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:175:17: error: ‘_gruB3_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.gruB3_b = &_gruB3_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:177:18: error: ‘_gruF4_rnnrf_flipflop_r941native_iW’ undeclared here (not in a function)
.gruF4_iW = &_gruF4_rnnrf_flipflop_r941native_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:178:18: error: ‘_gruF4_rnnrf_flipflop_r941native_sW’ undeclared here (not in a function)
.gruF4_sW = &_gruF4_rnnrf_flipflop_r941native_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:179:17: error: ‘_gruF4_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.gruF4_b = &_gruF4_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:181:18: error: ‘_gruB5_rnnrf_flipflop_r941native_iW’ undeclared here (not in a function)
.gruB5_iW = &_gruB5_rnnrf_flipflop_r941native_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:182:18: error: ‘_gruB5_rnnrf_flipflop_r941native_sW’ undeclared here (not in a function)
.gruB5_sW = &_gruB5_rnnrf_flipflop_r941native_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:183:17: error: ‘_gruB5_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.gruB5_b = &_gruB5_rnnrf_flipflop_r941native_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:185:14: error: ‘_FF_rnnrf_flipflop_r941native_W’ undeclared here (not in a function)
.FF_W = &_FF_rnnrf_flipflop_r941native_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:186:14: error: ‘_FF_rnnrf_flipflop_r941native_b’ undeclared here (not in a function)
.FF_b = &_FF_rnnrf_flipflop_r941native_b
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:192:16: error: ‘_conv_rnnrf_flipflop_r941native5mC_W’ undeclared here (not in a function)
.conv_W = &_conv_rnnrf_flipflop_r941native5mC_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:193:16: error: ‘_conv_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.conv_b = &_conv_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:194:20: error: ‘conv_rnnrf_flipflop_r941native5mC_stride’ undeclared here (not in a function)
.conv_stride = conv_rnnrf_flipflop_r941native5mC_stride,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:197:18: error: ‘_gruB1_rnnrf_flipflop_r941native5mC_iW’ undeclared here (not in a function)
.gruB1_iW = &_gruB1_rnnrf_flipflop_r941native5mC_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:198:18: error: ‘_gruB1_rnnrf_flipflop_r941native5mC_sW’ undeclared here (not in a function)
.gruB1_sW = &_gruB1_rnnrf_flipflop_r941native5mC_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:199:17: error: ‘_gruB1_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.gruB1_b = &_gruB1_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:201:18: error: ‘_gruF2_rnnrf_flipflop_r941native5mC_iW’ undeclared here (not in a function)
.gruF2_iW = &_gruF2_rnnrf_flipflop_r941native5mC_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:202:18: error: ‘_gruF2_rnnrf_flipflop_r941native5mC_sW’ undeclared here (not in a function)
.gruF2_sW = &_gruF2_rnnrf_flipflop_r941native5mC_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:203:17: error: ‘_gruF2_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.gruF2_b = &_gruF2_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:205:18: error: ‘_gruB3_rnnrf_flipflop_r941native5mC_iW’ undeclared here (not in a function)
.gruB3_iW = &_gruB3_rnnrf_flipflop_r941native5mC_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:206:18: error: ‘_gruB3_rnnrf_flipflop_r941native5mC_sW’ undeclared here (not in a function)
.gruB3_sW = &_gruB3_rnnrf_flipflop_r941native5mC_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:207:17: error: ‘_gruB3_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.gruB3_b = &_gruB3_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:209:18: error: ‘_gruF4_rnnrf_flipflop_r941native5mC_iW’ undeclared here (not in a function)
.gruF4_iW = &_gruF4_rnnrf_flipflop_r941native5mC_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:210:18: error: ‘_gruF4_rnnrf_flipflop_r941native5mC_sW’ undeclared here (not in a function)
.gruF4_sW = &_gruF4_rnnrf_flipflop_r941native5mC_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:211:17: error: ‘_gruF4_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.gruF4_b = &_gruF4_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:213:18: error: ‘_gruB5_rnnrf_flipflop_r941native5mC_iW’ undeclared here (not in a function)
.gruB5_iW = &_gruB5_rnnrf_flipflop_r941native5mC_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:214:18: error: ‘_gruB5_rnnrf_flipflop_r941native5mC_sW’ undeclared here (not in a function)
.gruB5_sW = &_gruB5_rnnrf_flipflop_r941native5mC_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:215:17: error: ‘_gruB5_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.gruB5_b = &_gruB5_rnnrf_flipflop_r941native5mC_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:217:14: error: ‘_FF_rnnrf_flipflop_r941native5mC_W’ undeclared here (not in a function)
.FF_W = &_FF_rnnrf_flipflop_r941native5mC_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:218:14: error: ‘_FF_rnnrf_flipflop_r941native5mC_b’ undeclared here (not in a function)
.FF_b = &_FF_rnnrf_flipflop_r941native5mC_b
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:224:16: error: ‘_conv_rnnrf_flipflop_r10Cpcr_W’ undeclared here (not in a function)
.conv_W = &_conv_rnnrf_flipflop_r10Cpcr_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:225:16: error: ‘_conv_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.conv_b = &_conv_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:226:20: error: ‘conv_rnnrf_flipflop_r10Cpcr_stride’ undeclared here (not in a function)
.conv_stride = conv_rnnrf_flipflop_r10Cpcr_stride,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:229:18: error: ‘_gruB1_rnnrf_flipflop_r10Cpcr_iW’ undeclared here (not in a function)
.gruB1_iW = &_gruB1_rnnrf_flipflop_r10Cpcr_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:230:18: error: ‘_gruB1_rnnrf_flipflop_r10Cpcr_sW’ undeclared here (not in a function)
.gruB1_sW = &_gruB1_rnnrf_flipflop_r10Cpcr_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:231:17: error: ‘_gruB1_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.gruB1_b = &_gruB1_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:233:18: error: ‘_gruF2_rnnrf_flipflop_r10Cpcr_iW’ undeclared here (not in a function)
.gruF2_iW = &_gruF2_rnnrf_flipflop_r10Cpcr_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:234:18: error: ‘_gruF2_rnnrf_flipflop_r10Cpcr_sW’ undeclared here (not in a function)
.gruF2_sW = &_gruF2_rnnrf_flipflop_r10Cpcr_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:235:17: error: ‘_gruF2_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.gruF2_b = &_gruF2_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:237:18: error: ‘_gruB3_rnnrf_flipflop_r10Cpcr_iW’ undeclared here (not in a function)
.gruB3_iW = &_gruB3_rnnrf_flipflop_r10Cpcr_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:238:18: error: ‘_gruB3_rnnrf_flipflop_r10Cpcr_sW’ undeclared here (not in a function)
.gruB3_sW = &_gruB3_rnnrf_flipflop_r10Cpcr_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:239:17: error: ‘_gruB3_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.gruB3_b = &_gruB3_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:241:18: error: ‘_gruF4_rnnrf_flipflop_r10Cpcr_iW’ undeclared here (not in a function)
.gruF4_iW = &_gruF4_rnnrf_flipflop_r10Cpcr_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:242:18: error: ‘_gruF4_rnnrf_flipflop_r10Cpcr_sW’ undeclared here (not in a function)
.gruF4_sW = &_gruF4_rnnrf_flipflop_r10Cpcr_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:243:17: error: ‘_gruF4_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.gruF4_b = &_gruF4_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:245:18: error: ‘_gruB5_rnnrf_flipflop_r10Cpcr_iW’ undeclared here (not in a function)
.gruB5_iW = &_gruB5_rnnrf_flipflop_r10Cpcr_iW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:246:18: error: ‘_gruB5_rnnrf_flipflop_r10Cpcr_sW’ undeclared here (not in a function)
.gruB5_sW = &_gruB5_rnnrf_flipflop_r10Cpcr_sW,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:247:17: error: ‘_gruB5_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.gruB5_b = &_gruB5_rnnrf_flipflop_r10Cpcr_b,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:249:14: error: ‘_FF_rnnrf_flipflop_r10Cpcr_W’ undeclared here (not in a function)
.FF_W = &_FF_rnnrf_flipflop_r10Cpcr_W,
^
/home/leo/NGS/katuali/flappie/flappie/src/networks.c:250:14: error: ‘_FF_rnnrf_flipflop_r10Cpcr_b’ undeclared here (not in a function)
.FF_b = &_FF_rnnrf_flipflop_r10Cpcr_b
^
CMakeFiles/flappie_objects.dir/build.make:88: recipe for target 'CMakeFiles/flappie_objects.dir/src/networks.c.o' failed
make[4]: *** [CMakeFiles/flappie_objects.dir/src/networks.c.o] Error 1
make[4]: Leaving directory '/home/leo/NGS/katuali/flappie/flappie/build'
CMakeFiles/Makefile2:109: recipe for target 'CMakeFiles/flappie_objects.dir/all' failed
make[3]: *** [CMakeFiles/flappie_objects.dir/all] Error 2
make[3]: Leaving directory '/home/leo/NGS/katuali/flappie/flappie/build'
CMakeFiles/Makefile2:158: recipe for target 'CMakeFiles/flappie.dir/rule' failed
make[2]: *** [CMakeFiles/flappie.dir/rule] Error 2
make[2]: Leaving directory '/home/leo/NGS/katuali/flappie/flappie/build'
Makefile:223: recipe for target 'flappie' failed
make[1]: *** [flappie] Error 2
make[1]: Leaving directory '/home/leo/NGS/katuali/flappie/flappie/build'
Makefile:30: recipe for target 'build/flappie' failed
make: *** [build/flappie] Error 2

When you have time, could you give me the solutions, thank you in advance,

Best regards,
Leo

Hello,

I am using the docker container version of your tool for the analysis of minION data. After fixing a few small issues related to STRique and albacore not being found in $PATH, I was able to run the tests you provided in the test folder without any issues.

I used the nanopype_import script to pack my fast5 files into a tar archive. The corresponding folder structure looks like this:

• /data/raw/
  • 20191204_FAK....._FLO-MIN106_SQK-LSK109/
    • reads/
      • 0.tar
    • fast5/
      • ...0.fast5
      • ...1.fast5
      • ...
    • reads.fofn

Inside /data/processing i ran:
snakemake --snakefile ~/path/to/nanopype/Snakefile -j 7 sequences/guppy/20191204_FAK....._FLO-MIN106_SQK-LSK109.fastq.gz

Which gives the following result:

Now the pipeline actually produces the file in sequences/guppy/20191204_FAK....._FLO-MIN106_SQK-LSK109.fastq.gz however, it stays empty. Also judging from htop, nothing is happening and the job is rather being kept active.

I did not change anything in the nanopype.yaml and env.yaml files with the exception of replacing the reference chromosome in env.yaml like this

Currently, I am running docker version 19.03.1 on CentOS 7.

Please let me know, if there is anything wrong with my folder structure, the way I invoke the pipeline or if you need any additional information.

Thanks in advance!

I need a newer guppy install. I also want/need GPU. It SEEMS that I can alter the install.smk script to update these things. What I dont know how to do is to create a new singularity image using this updated install.smk.

Is this possible? Should I be worried about this approch?