ccdmb / predector Goto Github PK

View Code? Open in Web Editor NEW

10.0 10.0 7.0 41.43 MB

Effector prediction pipeline based on protein properties.

License: Apache License 2.0

Makefile 0.82% Dockerfile 0.70% Shell 64.36% Nextflow 25.33% TeX 8.74% Batchfile 0.04%

predector's People

Contributors

Stargazers

Watchers

Forkers

jwdebler truwl fmobegi idobar yirenheihei stef-cr mmastert

predector's Issues

New docs section: Extended intallation information

Despite my efforts to make the conda, docker based setup as easy as possible, we still had some issues.

I suggest that on the main repo README we have a quick-install version, and in the wiki we have an extended INSTALL document.

We should add common mistakes and FAQs as encounter them.
We've already encountered:

using sudo when installing miniconda
using WSL1 (possibly WSL2 won't work either, we'll see)
Distributions not supporting i386 instructions (afaik this is only an issue for signalp3, possibly we can drop this as a dependency if it's not still better for some organisms). Suggest installing libc-i386.
not running the -register scripts (better documentation should help avoid this).

BUG: deepsig error

Describe the bug
deepsig crashes with below error message. May have to do with keras-team/keras#14265

WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Using TensorFlow backend.
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /opt/conda/envs/predector/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4074: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

Using TensorFlow backend.
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /opt/conda/envs/predector/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4074: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

[Tue, 01 Jun 2021 07:26:46] Setting up job temporary enviroment [/data/me14_predector/work/3f/aad17b9ddc33340c3635fccfb78c04/job.tmpd.768wmj_n]
[Tue, 01 Jun 2021 07:26:46] Reading input data
[Tue, 01 Jun 2021 07:26:47] Read 100 protein sequences
[Tue, 01 Jun 2021 07:26:47] Detecting signal peptides
[Tue, 01 Jun 2021 07:26:47] Errors occured during execution
[Tue, 01 Jun 2021 07:26:47] Leaving outdir unchanged
Traceback (most recent call last):
  File "/opt/conda/envs/predector/bin/deepsig.py", line 68, in <module>
    main()
  File "/opt/conda/envs/predector/bin/deepsig.py", line 44, in main
    Y, Ytm, Ync, cls, Ytm_norm, Ync_norm = detectsp(X, ns.organism)
  File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 84, in detectsp
    dnn.load_weights("%s.hdf5" % modpfx)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
    return load_function(*args, **kwargs)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/network.py", line 1230, in load_weights
    f, self.layers, reshape=reshape)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 1183, in load_weights_from_hdf5_group
    original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
Using TensorFlow backend.
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /opt/conda/envs/predector/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4074: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

[Tue, 01 Jun 2021 07:26:46] Setting up job temporary enviroment [/data/me14_predector/work/3f/aad17b9ddc33340c3635fccfb78c04/job.tmpd.0otzvqcz]
[Tue, 01 Jun 2021 07:26:46] Reading input data
[Tue, 01 Jun 2021 07:26:47] Read 100 protein sequences
[Tue, 01 Jun 2021 07:26:47] Detecting signal peptides
[Tue, 01 Jun 2021 07:26:47] Errors occured during execution
[Tue, 01 Jun 2021 07:26:47] Leaving outdir unchanged
Traceback (most recent call last):
  File "/opt/conda/envs/predector/bin/deepsig.py", line 68, in <module>
    main()
  File "/opt/conda/envs/predector/bin/deepsig.py", line 44, in main
    Y, Ytm, Ync, cls, Ytm_norm, Ync_norm = detectsp(X, ns.organism)
  File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 84, in detectsp
    dnn.load_weights("%s.hdf5" % modpfx)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
    return load_function(*args, **kwargs)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/network.py", line 1230, in load_weights
    f, self.layers, reshape=reshape)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 1183, in load_weights_from_hdf5_group
    original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
Using TensorFlow backend.
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/opt/conda/envs/predector/lib/python3.7/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:tensorflow:From /opt/conda/envs/predector/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4074: The name tf.nn.avg_pool is deprecated. Please use tf.nn.avg_pool2d instead.

[Tue, 01 Jun 2021 07:26:46] Setting up job temporary enviroment [/data/me14_predector/work/3f/aad17b9ddc33340c3635fccfb78c04/job.tmpd.5m6e8u5g]
[Tue, 01 Jun 2021 07:26:46] Reading input data
[Tue, 01 Jun 2021 07:26:47] Read 100 protein sequences
[Tue, 01 Jun 2021 07:26:47] Detecting signal peptides
[Tue, 01 Jun 2021 07:26:47] Errors occured during execution
[Tue, 01 Jun 2021 07:26:47] Leaving outdir unchanged
Traceback (most recent call last):
  File "/opt/conda/envs/predector/bin/deepsig.py", line 68, in <module>
    main()
  File "/opt/conda/envs/predector/bin/deepsig.py", line 44, in main
    Y, Ytm, Ync, cls, Ytm_norm, Ync_norm = detectsp(X, ns.organism)
  File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 84, in detectsp
    dnn.load_weights("%s.hdf5" % modpfx)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
    return load_function(*args, **kwargs)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/network.py", line 1230, in load_weights
    f, self.layers, reshape=reshape)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 1183, in load_weights_from_hdf5_group
    original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
[Tue, 01 Jun 2021 07:26:46] Setting up job temporary enviroment [/data/me14_predector/work/3f/aad17b9ddc33340c3635fccfb78c04/job.tmpd.dimrb8ln]
[Tue, 01 Jun 2021 07:26:46] Reading input data
[Tue, 01 Jun 2021 07:26:47] Read 100 protein sequences
[Tue, 01 Jun 2021 07:26:47] Detecting signal peptides
[Tue, 01 Jun 2021 07:26:47] Errors occured during execution
[Tue, 01 Jun 2021 07:26:47] Leaving outdir unchanged
Traceback (most recent call last):
  File "/opt/conda/envs/predector/bin/deepsig.py", line 68, in <module>
    main()
  File "/opt/conda/envs/predector/bin/deepsig.py", line 44, in main
    Y, Ytm, Ync, cls, Ytm_norm, Ync_norm = detectsp(X, ns.organism)
  File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 84, in detectsp
    dnn.load_weights("%s.hdf5" % modpfx)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 492, in load_wrapper
    return load_function(*args, **kwargs)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/network.py", line 1230, in load_weights
    f, self.layers, reshape=reshape)
  File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/engine/saving.py", line 1183, in load_weights_from_hdf5_group
    original_keras_version = f.attrs['keras_version'].decode('utf8')
AttributeError: 'str' object has no attribute 'decode'
parallel: This job failed:
run $PARALLEL_TMP

Fredrick pointed out that the individual analysis tables don't include the names of the protein sequences.
I've fixed this in predector-utils but still need to update the pipeline to use the update.

Should be done in a few days.

BUG: Process `signalp_v6 (1)` terminated with an error exit status (1)

Describe the bug
I received the error "Process signalp_v6 (1) terminated with an error exit status (1)" during the second iteration of signap_v6. Although I included a much larger chunk of the output below, the most pertinent part of the output is probably as follows:

Error executing process > 'signalp_v6 (1)'

Caused by:
Process signalp_v6 (1) terminated with an error exit status (1)

Command executed:

export OMP_NUM_THREADS="4"

run () {
set -e
TMPDIR="tmpdir$$"
mkdir "${TMPDIR}"

  signalp6           --fastafile "${1}"           --output_dir "${TMPDIR}"           --format txt           --organism eukarya           --mode fast           --bsize 64           1>&2

  cat "${TMPDIR}"/prediction_results.txt
  rm -rf -- "${TMPDIR}"

}

export -f run

parallel --halt now,fail=1 --joblog log.txt -j 1 -N 1000 --line-buffer --recstart '>' --cat run < in.fasta | cat > out.txt

predutils r2js --pipeline-version "1.2.3" --software-version "6.0" signalp6 "out.txt" in.fasta > "out.ldjson"

Command exit status:
1

Command output:
(empty)

To Reproduce
I submitted the following bash script on an HPC:

module load conda
module load nextflow

nextflow run
-resume
-r 1.2.3
-with-conda /home/user/.conda/envs/predector
ccdmb/predector
--proteome my_proteomes/prot.fasta

Expected behavior
Second iteration of Singalp6.0 failed to complete. I ran the bash script several times with the same result.

**Screenshots or copy-pasted terminal outut **
I have pasted the last few iterations of the executor beginning from before the error occurred until the end of the terminal output:

executor > local (36)
[57/22cb94] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[7d/a1c720] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[e7/d499af] process > validate_input:download_dbcan [100%] 1 of 1, cached: 1 ✔
[1c/27b0c2] process > validate_input:download_phi... [100%] 1 of 1, cached: 1 ✔
[da/7fbb86] process > validate_input:download_eff... [100%] 1 of 1, cached: 1 ✔
[9e/93585d] process > check_env:get_signalp3_version [100%] 1 of 1, cached: 1 ✔
[93/12a850] process > check_env:get_signalp4_version [100%] 1 of 1, cached: 1 ✔
[9d/da6e27] process > check_env:get_signalp5_version [100%] 1 of 1, cached: 1 ✔
[60/af11b4] process > check_env:get_signalp6_version [100%] 1 of 1, cached: 1 ✔
[e9/f46f4f] process > check_env:get_targetp2_version [100%] 1 of 1, cached: 1 ✔
[bc/68e4ad] process > check_env:get_tmhmm2_version [100%] 1 of 1, cached: 1 ✔
[88/f94c72] process > check_env:get_deeploc1_version [100%] 1 of 1, cached: 1 ✔
[0a/d078c2] process > check_env:get_phobius_version [100%] 1 of 1, cached: 1 ✔
[df/47be41] process > check_env:get_effectorp1_ve... [100%] 1 of 1, cached: 1 ✔
[4a/f3b045] process > check_env:get_effectorp2_ve... [100%] 1 of 1, cached: 1 ✔
[2f/333448] process > check_env:get_effectorp3_ve... [100%] 1 of 1, cached: 1 ✔
[79/fa95fd] process > check_env:get_localizer_ver... [100%] 1 of 1, cached: 1 ✔
[ac/db9062] process > check_env:get_apoplastp_ver... [100%] 1 of 1, cached: 1 ✔
[f0/250afa] process > check_env:get_deepsig_version [100%] 1 of 1, cached: 1 ✔
[6d/385ded] process > check_env:get_emboss_version [100%] 1 of 1, cached: 1 ✔
[1a/2cc2a0] process > check_env:get_mmseqs2_version [100%] 1 of 1, cached: 1 ✔
[4b/1a9a23] process > check_env:get_hmmer_version [100%] 1 of 1, cached: 1 ✔
[45/7befe7] process > check_env:get_deepredeff_ve... [100%] 1 of 1, cached: 1 ✔
[08/f9a06c] process > check_env:get_predutils_ver... [100%] 1 of 1, cached: 1 ✔
[37/65dfbb] process > gen_target_table [100%] 1 of 1, cached: 1 ✔
[5d/4847c9] process > sanitise_phibase [100%] 1 of 1, cached: 1 ✔
[36/57e10d] process > encode_seqs [100%] 1 of 1, cached: 1 ✔
[36/fb7c74] process > split_fasta (deduplicated) [100%] 1 of 1, cached: 1 ✔
[8d/788f67] process > signalp_v3_hmm (2) [100%] 2 of 2, cached: 1 ✔
[1c/8cacf0] process > signalp_v3_nn (2) [100%] 2 of 2 ✔
[67/54b594] process > signalp_v4 (1) [100%] 2 of 2, cached: 2 ✔
[6e/7233fd] process > signalp_v5 (1) [100%] 2 of 2 ✔
[df/7b61fd] process > signalp_v6 (1) [ 50%] 1 of 2
[7a/2eeb82] process > deepsig (1) [100%] 2 of 2 ✔
[75/ddfd79] process > phobius (1) [100%] 2 of 2 ✔
[b7/7ad598] process > tmhmm (1) [100%] 2 of 2, cached: 1 ✔
[fa/6f0e36] process > targetp (2) [100%] 2 of 2 ✔
[7a/de1608] process > deeploc (2) [100%] 2 of 2, cached: 2 ✔
[21/c9dfbf] process > apoplastp (2) [100%] 2 of 2 ✔
[d4/45e1e5] process > localizer (2) [100%] 2 of 2, cached: 1 ✔
[08/e997b3] process > effectorp_v1 (2) [100%] 2 of 2 ✔
[33/a718c2] process > effectorp_v2 (2) [100%] 2 of 2 ✔
[9d/ead121] process > effectorp_v3 (2) [100%] 2 of 2, cached: 2 ✔
[27/8ce3a3] process > deepredeff_fungi_v1 (2) [100%] 2 of 2 ✔
[29/8b4414] process > deepredeff_oomycete_v1 (1) [100%] 2 of 2 ✔
[13/717d0d] process > kex2_regex (1) [100%] 2 of 2, cached: 2 ✔
[19/cfd84d] process > rxlrlike_regex (2) [100%] 2 of 2, cached: 2 ✔
[7f/9cad74] process > pepstats (1) [100%] 2 of 2, cached: 2 ✔
[d0/809e9e] process > press_pfam_hmmer [100%] 1 of 1, cached: 1 ✔
[3a/8ef994] process > pfamscan (2) [100%] 2 of 2 ✔
[1e/847cc8] process > press_dbcan_hmmer [100%] 1 of 1, cached: 1 ✔
[02/177300] process > hmmscan_dbcan (2) [100%] 2 of 2 ✔
[25/e905ac] process > press_effectordb_hmmer [100%] 1 of 1, cached: 1 ✔
[0d/751b85] process > hmmscan_effectordb (2) [100%] 2 of 2 ✔
[82/f29312] process > mmseqs_index_proteomes (1) [100%] 2 of 2, cached: 2 ✔
[d5/85a21b] process > mmseqs_index_phibase [100%] 1 of 1, cached: 1 ✔
[58/32f6d9] process > mmseqs_search_phibase (1) [100%] 1 of 1 ✔
[7e/047bbc] process > decode_seqs (4) [100%] 4 of 4
[- ] process > collect_file -
[- ] process > gff_results -
[- ] process > tabular_results -
[- ] process > rank_results -
[51/9d17e6] process > publish_it (downloads/phi-b... [100%] 8 of 8, cached: 8
Error executing process > 'signalp_v6 (1)'

Caused by:
Process signalp_v6 (1) terminated with an error exit status (1)

Command executed:

export OMP_NUM_THREADS="4"

run () {
set -e
TMPDIR="tmpdir$$"
mkdir "${TMPDIR}"

  signalp6           --fastafile "${1}"           --output_dir "${TMPDIR}"           --format txt           --organism eukarya           --mode fast           --bsize 64           1>&2

  cat "${TMPDIR}"/prediction_results.txt
  rm -rf -- "${TMPDIR}"

}

export -f run

parallel --halt now,fail=1 --joblog log.txt -j 1 -N 1000 --line-buffer --recstart '>' --cat run < in.fasta | cat > out.txt

predutils r2js --pipeline-version "1.2.3" --software-version "6.0" signalp6 "out.txt" in.fasta > "out.ldjson"

Command exit status:
1

Command output:
(empty)

Command error:
Predicting: 77%|███████▋ | 768/1000 [03:31<01:03, 3.68sequences/s]
Predicting: 83%|████████▎ | 832/1000 [03:49<00:45, 3.67sequences/s]
Predicting: 90%|████████▉ | 896/1000 [04:06<00:28, 3.67sequences/s]
Predicting: 96%|█████████▌| 960/1000 [04:23<00:10, 3.67sequences/s]
Predicting: 100%|██████████| 1000/1000 [04:34<00:00, 3.67sequences/s]
Predicting: 100%|██████████| 1000/1000 [04:34<00:00, 3.64sequences/s]

Writing files: 0%| | 0/1000 [00:00<?, ?it/s]
Writing files: 15%|█▌ | 152/1000 [00:00<00:00, 1496.18it/s]
Writing files: 31%|███ | 309/1000 [00:00<00:00, 1538.64it/s]
Writing files: 46%|████▋ | 463/1000 [00:00<00:00, 1474.01it/s]
Writing files: 61%|██████ | 611/1000 [00:00<00:00, 1465.62it/s]
Writing files: 78%|███████▊ | 781/1000 [00:00<00:00, 1535.92it/s]
Writing files: 94%|█████████▎| 937/1000 [00:00<00:00, 1533.97it/s]
Writing files: 100%|██████████| 1000/1000 [00:00<00:00, 1511.15it/s]

Predicting: 0%| | 0/1000 [00:00<?, ?sequences/s]/home/user/.conda/envs/predector/lib/python3.6/site-packages/torch/nn/modules/module.py:1051: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at /tmp/pip-req-build-vflcj7wg/aten/src/ATen/native/TensorCompare.cpp:255.)
return forward_call(*input, **kwargs)

Predicting: 6%|▋ | 64/1000 [00:18<04:28, 3.48sequences/s]
Predicting: 13%|█▎ | 128/1000 [00:36<04:10, 3.48sequences/s]
Predicting: 19%|█▉ | 192/1000 [00:54<03:48, 3.54sequences/s]
Predicting: 26%|██▌ | 256/1000 [01:11<03:25, 3.61sequences/s]
Predicting: 32%|███▏ | 320/1000 [01:29<03:07, 3.62sequences/s]
Predicting: 38%|███▊ | 384/1000 [01:46<02:49, 3.64sequences/s]
Predicting: 45%|████▍ | 448/1000 [02:04<02:31, 3.66sequences/s]
Predicting: 51%|█████ | 512/1000 [02:21<02:13, 3.66sequences/s]
Predicting: 58%|█████▊ | 576/1000 [02:38<01:55, 3.67sequences/s]
Predicting: 64%|██████▍ | 640/1000 [02:56<01:38, 3.67sequences/s]
Predicting: 70%|███████ | 704/1000 [03:13<01:20, 3.66sequences/s]
Predicting: 77%|███████▋ | 768/1000 [03:31<01:03, 3.67sequences/s]
Predicting: 83%|████████▎ | 832/1000 [03:48<00:45, 3.68sequences/s]
Predicting: 90%|████████▉ | 896/1000 [04:06<00:28, 3.65sequences/s]
Predicting: 96%|█████████▌| 960/1000 [04:23<00:10, 3.67sequences/s]
Predicting: 100%|██████████| 1000/1000 [04:34<00:00, 3.68sequences/s]
Predicting: 100%|██████████| 1000/1000 [04:34<00:00, 3.65sequences/s]
Traceback (most recent call last):
File "/home/user/.conda/envs/predector/bin/signalp6", line 8, in
sys.exit(predict())
File "/home/user/.conda/envs/predector/lib/python3.6/site-packages/signalp/init.py", line 6, in predict
main()
File "/home/user/.conda/envs/predector/lib/python3.6/site-packages/signalp/predict.py", line 235, in main
resolve_viterbi_marginal_conflicts(global_probs, marginal_probs, cleavage_sites, viterbi_paths)
File "/home/user/.conda/envs/predector/lib/python3.6/site-packages/signalp/utils.py", line 262, in resolve_viterbi_marginal_conflicts
cleavage_sites[i] = sp_idx.max() +1
File "/home/user/.conda/envs/predector/lib/python3.6/site-packages/numpy/core/_methods.py", line 39, in _amax
return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity
parallel: This job failed:
run $PARALLEL_TMP

Work dir:
user/predector/work/df/7b61fd7002d69e0f095a3582e6d192

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

executor > local (36)
[57/22cb94] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[7d/a1c720] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[e7/d499af] process > validate_input:download_dbcan [100%] 1 of 1, cached: 1 ✔
[1c/27b0c2] process > validate_input:download_phi... [100%] 1 of 1, cached: 1 ✔
[da/7fbb86] process > validate_input:download_eff... [100%] 1 of 1, cached: 1 ✔
[9e/93585d] process > check_env:get_signalp3_version [100%] 1 of 1, cached: 1 ✔
[93/12a850] process > check_env:get_signalp4_version [100%] 1 of 1, cached: 1 ✔
[9d/da6e27] process > check_env:get_signalp5_version [100%] 1 of 1, cached: 1 ✔
[60/af11b4] process > check_env:get_signalp6_version [100%] 1 of 1, cached: 1 ✔
[e9/f46f4f] process > check_env:get_targetp2_version [100%] 1 of 1, cached: 1 ✔
[bc/68e4ad] process > check_env:get_tmhmm2_version [100%] 1 of 1, cached: 1 ✔
[88/f94c72] process > check_env:get_deeploc1_version [100%] 1 of 1, cached: 1 ✔
[0a/d078c2] process > check_env:get_phobius_version [100%] 1 of 1, cached: 1 ✔
[df/47be41] process > check_env:get_effectorp1_ve... [100%] 1 of 1, cached: 1 ✔
[4a/f3b045] process > check_env:get_effectorp2_ve... [100%] 1 of 1, cached: 1 ✔
[2f/333448] process > check_env:get_effectorp3_ve... [100%] 1 of 1, cached: 1 ✔
[79/fa95fd] process > check_env:get_localizer_ver... [100%] 1 of 1, cached: 1 ✔
[ac/db9062] process > check_env:get_apoplastp_ver... [100%] 1 of 1, cached: 1 ✔
[f0/250afa] process > check_env:get_deepsig_version [100%] 1 of 1, cached: 1 ✔
[6d/385ded] process > check_env:get_emboss_version [100%] 1 of 1, cached: 1 ✔
[1a/2cc2a0] process > check_env:get_mmseqs2_version [100%] 1 of 1, cached: 1 ✔
[4b/1a9a23] process > check_env:get_hmmer_version [100%] 1 of 1, cached: 1 ✔
[45/7befe7] process > check_env:get_deepredeff_ve... [100%] 1 of 1, cached: 1 ✔
[08/f9a06c] process > check_env:get_predutils_ver... [100%] 1 of 1, cached: 1 ✔
[37/65dfbb] process > gen_target_table [100%] 1 of 1, cached: 1 ✔
[5d/4847c9] process > sanitise_phibase [100%] 1 of 1, cached: 1 ✔
[36/57e10d] process > encode_seqs [100%] 1 of 1, cached: 1 ✔
[36/fb7c74] process > split_fasta (deduplicated) [100%] 1 of 1, cached: 1 ✔
[8d/788f67] process > signalp_v3_hmm (2) [100%] 2 of 2, cached: 1 ✔
[1c/8cacf0] process > signalp_v3_nn (2) [100%] 2 of 2 ✔
[67/54b594] process > signalp_v4 (1) [100%] 2 of 2, cached: 2 ✔
[6e/7233fd] process > signalp_v5 (1) [100%] 2 of 2 ✔
[df/7b61fd] process > signalp_v6 (1) [100%] 2 of 2, failed: 1 ✘
[7a/2eeb82] process > deepsig (1) [100%] 2 of 2 ✔
[75/ddfd79] process > phobius (1) [100%] 2 of 2 ✔
[b7/7ad598] process > tmhmm (1) [100%] 2 of 2, cached: 1 ✔
[fa/6f0e36] process > targetp (2) [100%] 2 of 2 ✔
[7a/de1608] process > deeploc (2) [100%] 2 of 2, cached: 2 ✔
[21/c9dfbf] process > apoplastp (2) [100%] 2 of 2 ✔
[d4/45e1e5] process > localizer (2) [100%] 2 of 2, cached: 1 ✔
[08/e997b3] process > effectorp_v1 (2) [100%] 2 of 2 ✔
[33/a718c2] process > effectorp_v2 (2) [100%] 2 of 2 ✔
[9d/ead121] process > effectorp_v3 (2) [100%] 2 of 2, cached: 2 ✔
[27/8ce3a3] process > deepredeff_fungi_v1 (2) [100%] 2 of 2 ✔
[29/8b4414] process > deepredeff_oomycete_v1 (1) [100%] 2 of 2 ✔
[13/717d0d] process > kex2_regex (1) [100%] 2 of 2, cached: 2 ✔
[19/cfd84d] process > rxlrlike_regex (2) [100%] 2 of 2, cached: 2 ✔
[7f/9cad74] process > pepstats (1) [100%] 2 of 2, cached: 2 ✔
[d0/809e9e] process > press_pfam_hmmer [100%] 1 of 1, cached: 1 ✔
[3a/8ef994] process > pfamscan (2) [100%] 2 of 2 ✔
[1e/847cc8] process > press_dbcan_hmmer [100%] 1 of 1, cached: 1 ✔
[02/177300] process > hmmscan_dbcan (2) [100%] 2 of 2 ✔
[25/e905ac] process > press_effectordb_hmmer [100%] 1 of 1, cached: 1 ✔
[0d/751b85] process > hmmscan_effectordb (2) [100%] 2 of 2 ✔
[82/f29312] process > mmseqs_index_proteomes (1) [100%] 2 of 2, cached: 2 ✔
[d5/85a21b] process > mmseqs_index_phibase [100%] 1 of 1, cached: 1 ✔
[58/32f6d9] process > mmseqs_search_phibase (1) [100%] 1 of 1 ✔
[7e/047bbc] process > decode_seqs (4) [ 80%] 4 of 5
[- ] process > collect_file -
[- ] process > gff_results -
[- ] process > tabular_results -
[- ] process > rank_results -
[51/9d17e6] process > publish_it (downloads/phi-b... [100%] 8 of 8, cached: 8
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'signalp_v6 (1)'

Caused by:
Process signalp_v6 (1) terminated with an error exit status (1)

Command executed:

export OMP_NUM_THREADS="4"

run () {
set -e
TMPDIR="tmpdir$$"
mkdir "${TMPDIR}"

  signalp6           --fastafile "${1}"           --output_dir "${TMPDIR}"           --format txt           --organism eukarya           --mode fast           --bsize 64           1>&2

  cat "${TMPDIR}"/prediction_results.txt
  rm -rf -- "${TMPDIR}"

}

export -f run

parallel --halt now,fail=1 --joblog log.txt -j 1 -N 1000 --line-buffer --recstart '>' --cat run < in.fasta | cat > out.txt

predutils r2js --pipeline-version "1.2.3" --software-version "6.0" signalp6 "out.txt" in.fasta > "out.ldjson"

Command exit status:
1

Command output:
(empty)

Work dir:
user/predector/work/df/7b61fd7002d69e0f095a3582e6d192

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

java.nio.channels.ClosedByInterruptException

executor > local (36)
[57/22cb94] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[7d/a1c720] process > validate_input:download_pfa... [100%] 1 of 1, cached: 1 ✔
[e7/d499af] process > validate_input:download_dbcan [100%] 1 of 1, cached: 1 ✔
[1c/27b0c2] process > validate_input:download_phi... [100%] 1 of 1, cached: 1 ✔
[da/7fbb86] process > validate_input:download_eff... [100%] 1 of 1, cached: 1 ✔
[9e/93585d] process > check_env:get_signalp3_version [100%] 1 of 1, cached: 1 ✔
[93/12a850] process > check_env:get_signalp4_version [100%] 1 of 1, cached: 1 ✔
[9d/da6e27] process > check_env:get_signalp5_version [100%] 1 of 1, cached: 1 ✔
[60/af11b4] process > check_env:get_signalp6_version [100%] 1 of 1, cached: 1 ✔
[e9/f46f4f] process > check_env:get_targetp2_version [100%] 1 of 1, cached: 1 ✔
[bc/68e4ad] process > check_env:get_tmhmm2_version [100%] 1 of 1, cached: 1 ✔
[88/f94c72] process > check_env:get_deeploc1_version [100%] 1 of 1, cached: 1 ✔
[0a/d078c2] process > check_env:get_phobius_version [100%] 1 of 1, cached: 1 ✔
[df/47be41] process > check_env:get_effectorp1_ve... [100%] 1 of 1, cached: 1 ✔
[4a/f3b045] process > check_env:get_effectorp2_ve... [100%] 1 of 1, cached: 1 ✔
[2f/333448] process > check_env:get_effectorp3_ve... [100%] 1 of 1, cached: 1 ✔
[79/fa95fd] process > check_env:get_localizer_ver... [100%] 1 of 1, cached: 1 ✔
[ac/db9062] process > check_env:get_apoplastp_ver... [100%] 1 of 1, cached: 1 ✔
[f0/250afa] process > check_env:get_deepsig_version [100%] 1 of 1, cached: 1 ✔
[6d/385ded] process > check_env:get_emboss_version [100%] 1 of 1, cached: 1 ✔
[1a/2cc2a0] process > check_env:get_mmseqs2_version [100%] 1 of 1, cached: 1 ✔
[4b/1a9a23] process > check_env:get_hmmer_version [100%] 1 of 1, cached: 1 ✔
[45/7befe7] process > check_env:get_deepredeff_ve... [100%] 1 of 1, cached: 1 ✔
[08/f9a06c] process > check_env:get_predutils_ver... [100%] 1 of 1, cached: 1 ✔
[37/65dfbb] process > gen_target_table [100%] 1 of 1, cached: 1 ✔
[5d/4847c9] process > sanitise_phibase [100%] 1 of 1, cached: 1 ✔
[36/57e10d] process > encode_seqs [100%] 1 of 1, cached: 1 ✔
[36/fb7c74] process > split_fasta (deduplicated) [100%] 1 of 1, cached: 1 ✔
[8d/788f67] process > signalp_v3_hmm (2) [100%] 2 of 2, cached: 1 ✔
[1c/8cacf0] process > signalp_v3_nn (2) [100%] 2 of 2 ✔
[67/54b594] process > signalp_v4 (1) [100%] 2 of 2, cached: 2 ✔
[6e/7233fd] process > signalp_v5 (1) [100%] 2 of 2 ✔
[df/7b61fd] process > signalp_v6 (1) [100%] 2 of 2, failed: 1 ✘
[7a/2eeb82] process > deepsig (1) [100%] 2 of 2 ✔
[75/ddfd79] process > phobius (1) [100%] 2 of 2 ✔
[b7/7ad598] process > tmhmm (1) [100%] 2 of 2, cached: 1 ✔
[fa/6f0e36] process > targetp (2) [100%] 2 of 2 ✔
[7a/de1608] process > deeploc (2) [100%] 2 of 2, cached: 2 ✔
[21/c9dfbf] process > apoplastp (2) [100%] 2 of 2 ✔
[d4/45e1e5] process > localizer (2) [100%] 2 of 2, cached: 1 ✔
[08/e997b3] process > effectorp_v1 (2) [100%] 2 of 2 ✔
[33/a718c2] process > effectorp_v2 (2) [100%] 2 of 2 ✔
[9d/ead121] process > effectorp_v3 (2) [100%] 2 of 2, cached: 2 ✔
[27/8ce3a3] process > deepredeff_fungi_v1 (2) [100%] 2 of 2 ✔
[29/8b4414] process > deepredeff_oomycete_v1 (1) [100%] 2 of 2 ✔
[13/717d0d] process > kex2_regex (1) [100%] 2 of 2, cached: 2 ✔
[19/cfd84d] process > rxlrlike_regex (2) [100%] 2 of 2, cached: 2 ✔
[7f/9cad74] process > pepstats (1) [100%] 2 of 2, cached: 2 ✔
[d0/809e9e] process > press_pfam_hmmer [100%] 1 of 1, cached: 1 ✔
[3a/8ef994] process > pfamscan (2) [100%] 2 of 2 ✔
[1e/847cc8] process > press_dbcan_hmmer [100%] 1 of 1, cached: 1 ✔
[02/177300] process > hmmscan_dbcan (2) [100%] 2 of 2 ✔
[25/e905ac] process > press_effectordb_hmmer [100%] 1 of 1, cached: 1 ✔
[0d/751b85] process > hmmscan_effectordb (2) [100%] 2 of 2 ✔
[82/f29312] process > mmseqs_index_proteomes (1) [100%] 2 of 2, cached: 2 ✔
[d5/85a21b] process > mmseqs_index_phibase [100%] 1 of 1, cached: 1 ✔
[58/32f6d9] process > mmseqs_search_phibase (1) [100%] 1 of 1 ✔
[7e/047bbc] process > decode_seqs (4) [ 80%] 4 of 5
[- ] process > collect_file -
[- ] process > gff_results -
[- ] process > tabular_results -
[- ] process > rank_results -
[51/9d17e6] process > publish_it (downloads/phi-b... [100%] 8 of 8, cached: 8
Execution cancelled -- Finishing pending tasks before exit
WARN: Got an interrupted exception while taking agent result | java.lang.InterruptedException
Error executing process > 'signalp_v6 (1)'

Caused by:
Process signalp_v6 (1) terminated with an error exit status (1)

Command executed:

export OMP_NUM_THREADS="4"

run () {
set -e
TMPDIR="tmpdir$$"
mkdir "${TMPDIR}"

  signalp6           --fastafile "${1}"           --output_dir "${TMPDIR}"           --format txt           --organism eukarya           --mode fast           --bsize 64           1>&2

  cat "${TMPDIR}"/prediction_results.txt
  rm -rf -- "${TMPDIR}"

}

export -f run

parallel --halt now,fail=1 --joblog log.txt -j 1 -N 1000 --line-buffer --recstart '>' --cat run < in.fasta | cat > out.txt

predutils r2js --pipeline-version "1.2.3" --software-version "6.0" signalp6 "out.txt" in.fasta > "out.ldjson"

Command exit status:
1

Command output:
(empty)

Work dir:
user/predector/work/df/7b61fd7002d69e0f095a3582e6d192

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

java.nio.channels.ClosedByInterruptException

A DataflowVariable can only be assigned once. Use bind() to allow for equal values to be passed into already-bound variables.

Operating system (please enter the following information as appropriate):
Linux on HPC

Additional context
Add any other context about the problem here.

FEATURE REQUEST: Look into integrating post-processing information (e.g. Kex2)

What kind of feature are you proposing?

Quite a few effectors undergo post-translational processing to remove an additional propeptide sequence.
The best known of these appear to be processed by Kex2, but i'm sure others exist given the number of secreted proteases.

I'd like two things.

A tool that can identify cut-site motifs in proteins. Particularly focussing on those that happen in the host or maybe in extracellular vesicles.
An effector prediction tool that works on protein fragments (i.e. the propeptide and mature peptides).

These could both be separate tools from Predector, but they should be integrated into the pipeline.

It would also be nice users could provide their own custom cut-site regular expressions, PSSMs, or HMMs.

Additional context
https://nph.onlinelibrary.wiley.com/doi/10.1111/nph.17516
https://bmcmicrobiol.biomedcentral.com/articles/10.1186/1471-2180-8-116

New documentation page: Pipeline output.

We need a section about the output files.

Descriptions of the columns, rows, etc.
How you can use them.
Etc.

This should happen before we actually implement the final output format, because that way we prioritise user end experience over dev needs.

build docker container failed

Dear authors,

I am trying to install predector docker according to your introduction. However it turned out to be an error despite long struggle. Sorry that I hardly understand what was the reason so I paste my error message here for some help... Please see below. I am using Mac OS Monterey 12.0.1

Thank you.
Tian

Error message
[+] Building 28.6s (16/16) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.27kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/predector/predector-base:1.2.1 1.4s
=> [auth] predector/predector-base:pull token for registry-1.docker.io 0.0s
=> [ 1/11] FROM docker.io/predector/predector-base:1.2.1@sha256:ed9ea0a4 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 3.71kB 0.0s
=> CACHED [ 2/11] RUN mkdir -p /tmp/onbuild 0.0s
=> CACHED [ 3/11] COPY signalp-3.0.Linux.tar.Z /tmp/onbuild/ 0.0s
=> CACHED [ 4/11] COPY signalp-4.1g.Linux.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [ 5/11] COPY signalp-5.0b.Linux.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [ 6/11] COPY signalp-6.0b.fast.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [ 7/11] COPY targetp-2.0.Linux.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [ 8/11] COPY deeploc-1.0.All.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [ 9/11] COPY phobius101_linux.tar.gz /tmp/onbuild/ 0.0s
=> CACHED [10/11] COPY tmhmm-2.0c.Linux.tar.gz /tmp/onbuild/ 0.0s
=> ERROR [11/11] RUN echo && signalp3-register "/tmp/onbuild/$(basenam 27.1s

[11/11] RUN echo && signalp3-register "/tmp/onbuild/$(basename "signalp-3.0.Linux.tar.Z")" && echo && signalp4-register "/tmp/onbuild/$(basename "signalp-4.1g.Linux.tar.gz")" && echo && signalp5-register "/tmp/onbuild/$(basename "signalp-5.0b.Linux.tar.gz")" && echo && signalp6-register "/tmp/onbuild/$(basename "signalp-6.0b.fast.tar.gz")" && echo && targetp2-register "/tmp/onbuild/$(basename "targetp-2.0.Linux.tar.gz")" && echo && deeploc-register "/tmp/onbuild/$(basename "deeploc-1.0.All.tar.gz")" && echo && phobius-register "/tmp/onbuild/$(basename "phobius101_linux.tar.gz")" && echo && tmhmm2-register "/tmp/onbuild/$(basename "tmhmm-2.0c.Linux.tar.gz")" && echo && rm -rf -- /tmp/onbuild:
#16 0.174
#16 0.178 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 0.181 Registering source file /tmp/onbuild/signalp-3.0.Linux.tar.Z for signalp3 into conda environment at:
#16 0.181 /opt/conda/envs/predector/share/signalp3-3.0b-1
#16 0.181
#16 0.181 Unregistering old source files if they exist.
#16 0.182 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 0.197
#16 0.244 patching file signalp
#16 0.254 Finished registering signalp3.
#16 0.254 Testing installation...
#16 0.404 Test succeeded.
#16 0.404 signalp3 is now fully installed!
#16 0.408
#16 0.410 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 0.413 Registering source file /tmp/onbuild/signalp-4.1g.Linux.tar.gz for signalp4 into conda environment at:
#16 0.413 /opt/conda/envs/predector/share/signalp4-4.1g-2
#16 0.413
#16 0.413 Unregistering old source files if they exist.
#16 0.415 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 0.425
#16 0.660 patching file signalp
#16 0.669 Finished registering signalp4.
#16 0.669 Testing installation...
#16 1.038 Test succeeded.
#16 1.038 signalp4 is now fully installed!
#16 1.049
#16 1.052 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 1.054 Registering source file /tmp/onbuild/signalp-5.0b.Linux.tar.gz for signalp5 into conda environment at:
#16 1.054 /opt/conda/envs/predector/share/signalp5-5.0b-1
#16 1.054
#16 1.054 Unregistering old source files if they exist.
#16 1.056 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 1.065 Done
#16 1.968 Finished registering signalp5.
#16 1.968 Testing installation...
#16 7.235 Test succeeded.
#16 7.235 signalp5 is now fully installed!
#16 7.235
#16 7.237
#16 7.240 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 7.242 WARNING: /tmp/onbuild/signalp-6.0b.fast.tar.gz filename doesn't appear to match the expected filename
#16 7.242 signalp-6.0.fast.tar.gz.
#16 7.242
#16 7.242 WARNING: We'll attempt to continue the installation but you may want to
#16 7.242 double check that you really mean to use this file for package
#16 7.242 signalp6 1.0.
#16 7.242
#16 7.242 Registering source file /tmp/onbuild/signalp-6.0b.fast.tar.gz for signalp6 into conda environment at:
#16 7.242 /opt/conda/envs/predector/share/signalp6-1.0-1
#16 7.242
#16 7.242 Unregistering old source files if they exist.
#16 7.244 bash: /opt/conda/envs/predector/lib/libtinfo.so.6: no version information available (required by bash)
#16 7.665 WARNING: Skipping signalp6 as it is not installed.
#16 7.665 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#16 7.708
#16 22.06 Using pip 21.3.1 from /opt/conda/envs/predector/lib/python3.6/site-packages/pip (python 3.6)
#16 22.07 Non-user install because site-packages writeable
#16 22.09 Created temporary directory: /tmp/pip-ephem-wheel-cache-jf91_676
#16 22.09 Created temporary directory: /tmp/pip-req-tracker-qdzjx93h
#16 22.09 Initialized build tracking at /tmp/pip-req-tracker-qdzjx93h
#16 22.09 Created build tracker: /tmp/pip-req-tracker-qdzjx93h
#16 22.09 Entered build tracker: /tmp/pip-req-tracker-qdzjx93h
#16 22.09 Created temporary directory: /tmp/pip-install-in4sr26a
#16 22.11 Processing /tmp/tmp305/signalp6_fast/signalp-6-package
#16 22.11 Added file:///tmp/tmp305/signalp6_fast/signalp-6-package to build tracker '/tmp/pip-req-tracker-qdzjx93h'
#16 22.11 Created temporary directory: /tmp/pip-build-env-ymbteq4i
#16 22.11 Installing build dependencies: started
#16 22.11 Running command /opt/conda/envs/predector/bin/python3 /opt/conda/envs/predector/lib/python3.6/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ymbteq4i/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
#16 22.44 Using pip 21.3.1 from /opt/conda/envs/predector/lib/python3.6/site-packages/pip (python 3.6)
#16 23.72 Collecting setuptools
#16 23.81 Downloading setuptools-59.2.0-py3-none-any.whl (952 kB)
#16 23.98 Collecting wheel
#16 24.00 Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
#16 24.36 Installing collected packages: wheel, setuptools
#16 24.38 Creating /tmp/pip-build-env-ymbteq4i/overlay/bin
#16 24.38 changing mode of /tmp/pip-build-env-ymbteq4i/overlay/bin/wheel to 755
#16 24.64 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
#16 24.64 tensorflow 2.2.0 requires scipy==1.4.1; python_version >= "3", but you have scipy 1.5.3 which is incompatible.
#16 24.64 tensorflow 2.2.0 requires tensorboard<2.3.0,>=2.2.0, but you have tensorboard 2.7.0 which is incompatible.
#16 24.64 tensorflow 2.2.0 requires tensorflow-estimator<2.3.0,>=2.2.0, but you have tensorflow-estimator 2.6.0 which is incompatible.
#16 24.64 Successfully installed setuptools-59.2.0 wheel-0.37.0
#16 24.64 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#16 24.69 Installing build dependencies: finished with status 'done'
#16 24.69 Getting requirements to build wheel: started
#16 24.69 Running command /opt/conda/envs/predector/bin/python3 /opt/conda/envs/predector/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpdp0r96x4
#16 24.88 running egg_info
#16 24.88 creating signalp6.egg-info
#16 24.88 writing signalp6.egg-info/PKG-INFO
#16 24.88 writing dependency_links to signalp6.egg-info/dependency_links.txt
#16 24.88 writing entry points to signalp6.egg-info/entry_points.txt
#16 24.88 writing requirements to signalp6.egg-info/requires.txt
#16 24.88 writing top-level names to signalp6.egg-info/top_level.txt
#16 24.88 writing manifest file 'signalp6.egg-info/SOURCES.txt'
#16 24.89 reading manifest file 'signalp6.egg-info/SOURCES.txt'
#16 24.89 writing manifest file 'signalp6.egg-info/SOURCES.txt'
#16 24.90 Getting requirements to build wheel: finished with status 'done'
#16 24.90 Created temporary directory: /tmp/pip-modern-metadata-jcsdwnv6
#16 24.90 Preparing metadata (pyproject.toml): started
#16 24.90 Running command /opt/conda/envs/predector/bin/python3 /opt/conda/envs/predector/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpet4qbbdr
#16 25.08 running dist_info
#16 25.08 creating /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info
#16 25.08 writing /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/PKG-INFO
#16 25.08 writing dependency_links to /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/dependency_links.txt
#16 25.08 writing entry points to /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/entry_points.txt
#16 25.08 writing requirements to /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/requires.txt
#16 25.08 writing top-level names to /tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/top_level.txt
#16 25.08 writing manifest file '/tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/SOURCES.txt'
#16 25.09 reading manifest file '/tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/SOURCES.txt'
#16 25.09 writing manifest file '/tmp/pip-modern-metadata-jcsdwnv6/signalp6.egg-info/SOURCES.txt'
#16 25.09 creating '/tmp/pip-modern-metadata-jcsdwnv6/signalp6.dist-info'
#16 25.12 Preparing metadata (pyproject.toml): finished with status 'done'
#16 25.12 Source in /tmp/tmp305/signalp6_fast/signalp-6-package has version 1.0, which satisfies requirement signalp6==1.0 from file:///tmp/tmp305/signalp6_fast/signalp-6-package
#16 25.12 Removed signalp6==1.0 from file:///tmp/tmp305/signalp6_fast/signalp-6-package from build tracker '/tmp/pip-req-tracker-qdzjx93h'
#16 25.12 Created temporary directory: /tmp/pip-unpack-_fv7hqs7
#16 25.12 Building wheels for collected packages: signalp6
#16 25.12 Created temporary directory: /tmp/pip-wheel-gocvui5l
#16 25.12 Destination directory: /tmp/pip-wheel-gocvui5l
#16 25.12 Building wheel for signalp6 (pyproject.toml): started
#16 25.12 Running command /opt/conda/envs/predector/bin/python3 /opt/conda/envs/predector/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpq7o40b8z
#16 25.31 running bdist_wheel
#16 25.31 running build
#16 25.31 running build_py
#16 25.31 creating build
#16 25.31 creating build/lib
#16 25.31 creating build/lib/signalp
#16 25.31 copying signalp/make_sequence_plot.py -> build/lib/signalp
#16 25.31 copying signalp/sequential_model.py -> build/lib/signalp
#16 25.31 copying signalp/utils.py -> build/lib/signalp
#16 25.31 copying signalp/predict.py -> build/lib/signalp
#16 25.31 copying signalp/main.py -> build/lib/signalp
#16 25.31 copying signalp/make_output_files.py -> build/lib/signalp
#16 25.31 copying signalp/init.py -> build/lib/signalp
#16 25.32 creating build/lib/signalp/conversion_utils
#16 25.32 copying signalp/conversion_utils/move_checkpoint_to_device.py -> build/lib/signalp/conversion_utils
#16 25.32 copying signalp/conversion_utils/init.py -> build/lib/signalp/conversion_utils
#16 25.32 creating build/lib/signalp/model_weights
#16 25.32 copying signalp/model_weights/README.md -> build/lib/signalp/model_weights
#16 25.32 running build_scripts
#16 25.32 creating build/scripts-3.6
#16 25.32 copying signalp/conversion_utils/signalp6_convert_models -> build/scripts-3.6
#16 25.32 changing mode of build/scripts-3.6/signalp6_convert_models from 644 to 755
#16 25.32 installing to build/bdist.linux-x86_64/wheel
#16 25.32 running install
#16 25.32 running install_lib
#16 25.32 creating build/bdist.linux-x86_64
#16 25.32 creating build/bdist.linux-x86_64/wheel
#16 25.32 creating build/bdist.linux-x86_64/wheel/signalp
#16 25.32 copying build/lib/signalp/make_sequence_plot.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 creating build/bdist.linux-x86_64/wheel/signalp/model_weights
#16 25.32 copying build/lib/signalp/model_weights/README.md -> build/bdist.linux-x86_64/wheel/signalp/model_weights
#16 25.32 copying build/lib/signalp/sequential_model.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 copying build/lib/signalp/utils.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 copying build/lib/signalp/predict.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 copying build/lib/signalp/main.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 copying build/lib/signalp/make_output_files.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 creating build/bdist.linux-x86_64/wheel/signalp/conversion_utils
#16 25.32 copying build/lib/signalp/conversion_utils/move_checkpoint_to_device.py -> build/bdist.linux-x86_64/wheel/signalp/conversion_utils
#16 25.32 copying build/lib/signalp/conversion_utils/init.py -> build/bdist.linux-x86_64/wheel/signalp/conversion_utils
#16 25.32 copying build/lib/signalp/init.py -> build/bdist.linux-x86_64/wheel/signalp
#16 25.32 running install_egg_info
#16 25.33 running egg_info
#16 25.33 writing signalp6.egg-info/PKG-INFO
#16 25.33 writing dependency_links to signalp6.egg-info/dependency_links.txt
#16 25.33 writing entry points to signalp6.egg-info/entry_points.txt
#16 25.33 writing requirements to signalp6.egg-info/requires.txt
#16 25.33 writing top-level names to signalp6.egg-info/top_level.txt
#16 25.33 reading manifest file 'signalp6.egg-info/SOURCES.txt'
#16 25.33 writing manifest file 'signalp6.egg-info/SOURCES.txt'
#16 25.33 Copying signalp6.egg-info to build/bdist.linux-x86_64/wheel/signalp6-1.0-py3.6.egg-info
#16 25.33 running install_scripts
#16 25.34 creating build/bdist.linux-x86_64/wheel/signalp6-1.0.data
#16 25.34 creating build/bdist.linux-x86_64/wheel/signalp6-1.0.data/scripts
#16 25.34 copying build/scripts-3.6/signalp6_convert_models -> build/bdist.linux-x86_64/wheel/signalp6-1.0.data/scripts
#16 25.35 changing mode of build/bdist.linux-x86_64/wheel/signalp6-1.0.data/scripts/signalp6_convert_models to 755
#16 25.35 creating build/bdist.linux-x86_64/wheel/signalp6-1.0.dist-info/WHEEL
#16 25.35 creating '/tmp/pip-wheel-gocvui5l/tmp33678hdb/signalp6-1.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
#16 25.35 adding 'signalp/init.py'
#16 25.35 adding 'signalp/main.py'
#16 25.35 adding 'signalp/make_output_files.py'
#16 25.35 adding 'signalp/make_sequence_plot.py'
#16 25.35 adding 'signalp/predict.py'
#16 25.35 adding 'signalp/sequential_model.py'
#16 25.35 adding 'signalp/utils.py'
#16 25.35 adding 'signalp/conversion_utils/init.py'
#16 25.35 adding 'signalp/conversion_utils/move_checkpoint_to_device.py'
#16 25.35 adding 'signalp/model_weights/README.md'
#16 25.35 adding 'signalp6-1.0.data/scripts/signalp6_convert_models'
#16 25.35 adding 'signalp6-1.0.dist-info/METADATA'
#16 25.35 adding 'signalp6-1.0.dist-info/WHEEL'
#16 25.35 adding 'signalp6-1.0.dist-info/entry_points.txt'
#16 25.35 adding 'signalp6-1.0.dist-info/top_level.txt'
#16 25.36 adding 'signalp6-1.0.dist-info/RECORD'
#16 25.36 removing build/bdist.linux-x86_64/wheel
#16 25.37 Building wheel for signalp6 (pyproject.toml): finished with status 'done'
#16 25.37 Created wheel for signalp6: filename=signalp6-1.0-py3-none-any.whl size=21772 sha256=45d4d9c2f7f44c6f3bf199e810165201e8900dc5fb45ce85dfae790fc4d5b8fb
#16 25.37 Stored in directory: /root/.cache/pip/wheels/e9/c6/cf/f3bd00dbdd0936965911047a0fe7c0f1dc88a43c4aa4168bb1
#16 25.38 Successfully built signalp6
#16 25.38 Installing collected packages: signalp6
#16 25.39
#16 25.39 changing mode of /opt/conda/envs/predector/bin/signalp6 to 755
#16 25.42 Successfully installed signalp6-1.0
#16 25.42 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#16 25.42 Removed build tracker: '/tmp/pip-req-tracker-qdzjx93h'
#16 25.48 Finished registering signalp6.
#16 25.48 Testing installation...
#16 26.91 Traceback (most recent call last):
#16 26.91 File "/opt/conda/envs/predector/lib/python3.6/site-packages/signalp/predict.py", line 215, in main
#16 26.91 model = torch.jit.load(FAST_MODEL_PATH)
#16 26.91 File "/opt/conda/envs/predector/lib/python3.6/site-packages/torch/jit/_serialization.py", line 151, in load
#16 26.91 raise ValueError("The provided filename {} does not exist".format(f)) # type: ignore[str-bytes-safe]
#16 26.91 ValueError: The provided filename /opt/conda/envs/predector/lib/python3.6/site-packages/signalp/model_weights/distilled_model_signalp6.pt does not exist
#16 26.91
#16 26.91 During handling of the above exception, another exception occurred:
#16 26.91
#16 26.91 Traceback (most recent call last):
#16 26.91 File "/opt/conda/envs/predector/bin/signalp6", line 8, in
#16 26.91 sys.exit(predict())
#16 26.91 File "/opt/conda/envs/predector/lib/python3.6/site-packages/signalp/init.py", line 6, in predict
#16 26.91 main()
#16 26.91 File "/opt/conda/envs/predector/lib/python3.6/site-packages/signalp/predict.py", line 219, in main
#16 26.91 raise FileNotFoundError(f"Fast mode requires model to be installed at {FAST_MODEL_PATH}. Missing from this installation.")
#16 26.91 FileNotFoundError: Fast mode requires model to be installed at /opt/conda/envs/predector/lib/python3.6/site-packages/signalp/model_weights/distilled_model_signalp6.pt. Missing from this installation.

executor failed running [/bin/sh -c echo && signalp3-register "/tmp/onbuild/$(basename "${SIGNALP3}")" && echo && signalp4-register "/tmp/onbuild/$(basename "${SIGNALP4}")" && echo && signalp5-register "/tmp/onbuild/$(basename "${SIGNALP5}")" && echo && signalp6-register "/tmp/onbuild/$(basename "${SIGNALP6}")" && echo && targetp2-register "/tmp/onbuild/$(basename "${TARGETP2}")" && echo && deeploc-register "/tmp/onbuild/$(basename "${DEEPLOC}")" && echo && phobius-register "/tmp/onbuild/$(basename "${PHOBIUS}")" && echo && tmhmm2-register "/tmp/onbuild/$(basename "${TMHMM}")" && echo && rm -rf -- /tmp/onbuild]: exit code: 1

BUG: Names of all my proteins in tsv file are "."

Hello,

Thanks for your tools it did well for me except one thing, when i go in my repository results and i open all my tsv files , the "name" column is all time "." and not the name of my proteins.

I got the name of my proteins only in the gff3 files.

It's an example of what i have.

Thanks you for your answer.

BUG: Theano (deeploc) heaps of stuff writes to ~/.theano

So theano caches it's runtime compilations to a directory in your homedir.
For a self contained and isolated pipeline, this isn't great, but even worse is that on most HPCs, your home directory has limited storage.
On pawsey, this meant it breaks the job very quickly.

Unfortunately, deeploc hard-codes its theano environment variables.
We need to set base_compiledir=/path/to/current/directory in THEANO_FLAGS.

I think the conda package should only set THEANO_FLAGS if it isn't already set.
That way we can leave the defaults as is, but in the pipeline just export the proper flags.

PINTA!!!

Conda register scripts should report proper errors if they fail.

What kind of feature are you proposing?

When running the conda package signalp3-register etc scripts if there is a failure, they currently just output the actual failure and die.
A better user experience would be if the errors were captured, and we make some suggestions on how users could attempt to fix or report it and then exit.

Even better would be if they caught some common install issues (e.g. signalp3 hmm requiring 32-bit libc compatibility).

Are you interested in helping us add this feature?
E.g. modifying documentation or code.

Yeah I'll probably be the one doing it.

BUG: SignalP4 segfaults using conda in WSL2

Signalp4 segfaults in the conda environment using WSL2.
I suspect that this is a quirk of WSL2 or a missing 32-bit library, but need to check again where things are working and where they aren't.

FEATURE REQUEST: Add support for Effectorp3

EffectorP3 is out and we should using it.
I think we could probably drop EP1 at that point because it's not really as useful.

Possibly could add an option for oomycete effector ranking based on EP3 and deepredeff output.
Depends on whether we can come up with a sufficiently independent dataset to evaluate it with...

Add example command to run tests for install.sh

@JWDebler Suggested that we add the complete command to run the test pipeline given the dependency install that install.sh just did.

I like this idea, but i'm not sure how to do this without just giving too much information, thus preventing people from actually reading it. Its a fine balance.
Until we can think of it, i'm leaving it as an issue to be done in the future.

effectorp3.0 support

Dear developer,

I noticed that the effectorP has updated to 3.0 https://github.com/JanaSperschneider/EffectorP-3.0

So whether the predector support the effector3.0 or not?

Thanks,
Xizhe

FEATURE REQUEST: Diversify the test set a bit

Currently the test set only contains known effectors and homologues.
It doesn't contain any transmembrane proteins or proteins with dbCAN matches.

I need to add some tricky edge cases, non-effectors, and proteins with TM domains and dbCAN matches to test it better and provide a better example of the outputs.

Add to this file:
https://github.com/ccdmb/predector/blob/master/test/test_set.fasta

FEATURE REQUEST: Add a check to `install.sh` and conda register scripts to check for correct SignalP6 model

#64 appears to be an issue with the "fast" model missing from the signalp6 distribution.

It seems like it would be easy to accidentally download the slow_sequential model rather than the fast one.
We need a check in the -register script for conda to check for the fast version.

At that point we can either 1) download the fast model (which appears to be possible without registering from the documentation), or we can raise a nicer error that is clear about what happened.

The SignalP6 models are really big, so avoiding doubling container sizes would be ideal, but It might be less of a barrier if we just deal with it internally.

FEATURE REQUEST: Use GPUs for tasks that can utilise them.

SignalP6 is very slow on CPU but reasonably quick on GPU.

Others like deepredeff, deepsig, deeploc, and signalp5 aren't prohibitively slow on CPU, but should be able to be sped up with a GPU.

This would probably require a separate install environment using GPU variants of deep-learning tools.
We'd also need to find a good way to make this customisable with configuration. I'm thinking that instead of a cpu_small kind of label, introduce a gpu_task label. In CPU environments set this to X-many cpus. In a GPU environment set this to use the GPU.

Could be a pain, but signalp6 is really slow without it, making it impractical to run for more than a few thousand proteins.

BUG: Downloading Pfam-A fails. Connection time out.

Moved here from a different issue #40 raised by @paulamool.

Something may have changed at EBI
Received a connection error for ftp://ftp.ebi.ac.uk/

"--2021-10-15 00:47:57-- ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.dat.gz
(try:20) => ‘Pfam-A.hmm.dat.gz’
Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.197.74|:21... failed: Connection timed out.
Giving up."

I changed url in nextflow.config from ftp to http and it seemed okay for a few runs (4) but getting the same error again.

pfam_hmm_url = "http://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz"
pfam_dat_url = "http://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.dat.gz"

about the TM domain prediction

Dear Darcy,

When I run the pipeline, I found that the complete protein sequences were set as the input of the TMHMM. But I thought the mature protein sequences predicted by signalP would be better than the complete protein sequences. Because the TM domain on signal peptide would have no function. What do you think about this?

Thanks,
Xizhe

BUG: active_site.dat.gz not downloadable anymore

Describe the bug
The URL for active_site.dat.gz may have changed, as it is not listed on ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/ anymore.

Add check for WSL to install.sh

It's kind of hard to see what version of wsl people are using.
It seems like wsl2 works ok, but wsl1 won't run some of the 32bit dependencies.

there are a few options for how to tell which version it is, but I can't confirm that they will always work.

the environment variable WSL_INTEROP should only be set for wsl2
uname -r will show *-microsoft-standard* for wsl2 but wsl1 will have *-Microsoft *

There may be a directory called /run/WSL in wsl2 but in wsl1 it might be /run/WSL1.

AFAIK this is the best I can do, and i've only looked at it for ubuntu.

We should try to warn people using WSL1 this this is the reason they might get an issue with phobius/signalp3.

FEATURE REQUEST: Option to not run unnecessary slow tools

Signalp6 is REALLY slow.
LOCALIZER also takes quite a bit of time.
If neither make it into the ranking model, possibly they should be made optional so that it can be faster for high-throughput stuff.

FEATURE REQUEST: Incorporate deepredeff into pipeline

Deepredeff https://github.com/ruthkr/deepredeff/ has been on my radar for a while, but I haven't gotten around to trying it yet.

We need to evaluate it first because some of their numbers are a bit suspiciously good (I think it's just because their training data doesn't include a realistic negative set, i.e. their all like nuclear proteins so easier to distinguish than hydrophobins etc.).
I'm sure it's fine but we'll see, especially while it's still un-reviewed.

Paper is here: https://www.biorxiv.org/content/10.1101/2020.07.08.193250v1.abstract

FEATURE REQUEST: secretomeP

What kind of feature are you proposing?
secretomeP analysis, which related to the Non-classical secreted proteins. It could be a reference for the effector candidates.

FEATURE REQUEST: Sanitise PHI-base input

PHI-base fastas sometimes have some weird characters in them that screw up the parsing of MMSeqs results.
Should add a step to remove or replace non-UTF8 or non-ASCII characters before MMSeqs.

New docs section: configuring nextflow and running on supercomputers.

We should have a bit of information about how to specify different profiles, and how you can setup your own config file to run in a custom way.

E.g. People could optimise it to run on their local super-computer by setting the job-scheduler, specifying how many CPUs a task can use per node etc.

Look into automatically downloading PHIbase

Obtaining PHIbase from http://www.phi-base.org/ requires users to register in order to download.
But it's also available from https://github.com/PHI-base/data without registration.

We should ask Urban et al. if we can please download the database from here automatically, and make very clear that users of the pipeline should also cite phibase.

@JamesHane @paulamool Have you met them before? Feel like asking?

BUG:Installation error ImportError: cannot import name 'Protocol'

Hello, I am trying to install predector with a conda environment in a linux server. However I am having a problem with some python packages and this is the error.

Traceback (most recent call last):
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/importlib_metadata/_compat.py", line 9, in
from typing import Protocol
ImportError: cannot import name 'Protocol'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/sandra.gonzalez/.conda/envs/predector/bin/deeploc", line 87, in
main(args.fasta, args.output, args.attention)
File "/home/sandra.gonzalez/.conda/envs/predector/bin/deeploc", line 44, in main
f_ids, f_loc, f_mem = prediction(ids, prot_seqs, batch_size)
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/DeepLoc/models.py", line 251, in prediction
deploy_fn1, network_out1 = net_tree()
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/DeepLoc/models.py", line 112, in net_tree
deterministic=True)
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/lasagne/layers/helper.py", line 197, in get_output
all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/lasagne/layers/recurrent.py", line 1120, in get_output_for
strict=True)[0]
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/theano/scan_module/scan.py", line 1006, in scan
from theano import gpuarray
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/theano/gpuarray/init.py", line 23, in
import pygpu
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/pygpu/init.py", line 7, in
from . import gpuarray, elemwise, reduction
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/pygpu/reduction.py", line 4, in
from mako.template import Template
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/mako/template.py", line 19, in
from mako import cache
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/mako/cache.py", line 7, in
from mako import util
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/mako/util.py", line 14, in
from .compat import importlib_metadata_get
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/mako/compat.py", line 68, in
import importlib_metadata # noqa
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/importlib_metadata/init.py", line 17, in
from . import _adapters, _meta
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/importlib_metadata/_meta.py", line 1, in
from ._compat import Protocol
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/importlib_metadata/_compat.py", line 16, in
from typing_extensions import Protocol # type: ignore
File "/home/sandra.gonzalez/.conda/envs/predector/lib/python3.6/site-packages/typing_extensions.py", line 159, in
class _FinalForm(typing._SpecialForm, _root=True):
AttributeError: module 'typing' has no attribute '_SpecialForm'

could you please to help me to solve it?

best

BUG: conda test crashes

Describe the bug
Building the environment with conda finishes successfully, but running the test fails.

To Reproduce
nextflow run -profile test -with-conda "/home/ubuntu/miniconda3/envs/predector" -resume ccdmb/predector

**Screenshots or copy-pasted terminal outut **

Error executing process > 'encode_seqs'

Caused by:
  Process `encode_seqs` terminated with an error exit status (64)

Command executed:

  predutils encode       --prefix "P"       --length 8       --upper       --strip "*-"       combined.fasta       combined.tsv       in/*

  predutils split_fasta       --size "5000"       --template "combined/chunk{index:0>4}.fasta"       combined.fasta                                                                                            
Command exit status:
  64                                                                                                                                                                                                           
Command output:
  (empty)                                                                                                                                                                                                      
Command error:
  usage: /home/ubuntu/miniconda3/envs/predector/bin/predutils [-h] {r2js,encode,split_fasta,tables,gff,rank,decode} ...
  /home/ubuntu/miniconda3/envs/predector/bin/predutils: error: unrecognized arguments: --upper --strip

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Ubuntu 18.04.4 LTS
Dependency management: Conda
Are you using: Nimbus VM

Extract information from PHI-Base matches

What kind of feature are you proposing?

We need a way of associating phi-base MMSeqs2 matches with the phenotypic effect.
This may not be trivial, since the "reduced-virulence" phenotype encompasses a few different meanings.

Describe the solution you'd like

I think a simple frequency of match phenotype would be sufficient to reduce the data when we take it in tandem with Pfam matches and signal peptide prediction etc.
So if we have a match to a protein involved in the secretion pathway, which might have the phenotype "reduced virulence" we could count that as +1 for reduced virulence.
Essentially, we reduce the phibase matches down to just the counts of matches to different phenotypic effects.

The other suggestion is to try to classify those phenotypic effects by the experimental details, e.g. "knock out"+"reduced virulence". This gives us the ability to distinguish NEs from Avrs in some cases.
The experimental details column is less consistent than the phenotype column though.

It's a bit tricky to be honest.
I'd love some suggestions.

BUG: Problem with "Registering source file signalp-3.0.Linux.tar.Z"

Describe the bug

The registration for signalp-3.0 failed with the command :

signalp3-register signalp-3.0.Linux.tar.Z

To Reproduce

I have recovered the sources of all the proprietary tools. I installed predector with conda, so I have a dedicated conda environment. I had errors to register the proprietary tools, by adding the path to the libraries in LD_LIBRARY_PATH, that solved the problem for all except one : signalp-3.0.

Just a note, I run "install.sh" to create the predictor environement, then I activate this environment, I added the "predector" environment's library path to LD_LIBRARY_PATH and THEN I launched the registration manually.

Here my commands :

ENVIRONMENT=conda

cd /cm/shared/apps/predector/proprietary_software_dependencies

bash install.sh "${ENVIRONMENT}" \
-3 signalp-3.0.Linux.tar.Z \
-4 signalp-4.1g.Linux.tar.gz \
-5 signalp-5.0b.Linux.tar.gz \
-6 signalp-6.0d.fast.tar.gz \
-t targetp-2.0.Linux.tar.gz \
-d deeploc-1.0.All.tar.gz \
-m tmhmm-2.0c.Linux.tar.gz \
-p phobius101_linux.tar.gz

# the registrations failed
# I activate the predector environment and then I launched the registrations manually

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/cm/shared/apps/predector/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/cm/shared/apps/predector/etc/profile.d/conda.sh" ]; then
        . "/cm/shared/apps/predector/etc/profile.d/conda.sh"
    else
        export PATH="/cm/shared/apps/predector/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<
export LD_LIBRARY_PATH=/cm/shared/apps/predector/envs/predector/lib/:$LD_LIBRARY_PATH

conda activate predector

signalp3-register signalp-3.0.Linux.tar.Z
signalp4-register signalp-4.1g.Linux.tar.gz
signalp5-register signalp-5.0b.Linux.tar.gz
signalp6-register signalp-6.0d.fast.tar.gz
targetp2-register targetp-2.0.Linux.tar.gz
deeploc-register deeploc-1.0.All.tar.gz
phobius-register phobius101_linux.tar.gz
tmhmm2-register tmhmm-2.0c.Linux.tar.gz

Registrations worked for all the tools except signalp-3.0

signalp3-register signalp-3.0.Linux.tar.Z

I have this error :

Registering source file signalp-3.0.Linux.tar.Z for signalp3 into conda environment at:
/cm/shared/apps/predector/envs/predector/share/signalp3-3.0b-1

Unregistering old source files if they exist.

tar (child): compress: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
basename: missing operand
Try 'basename --help' for more information.

EDIT :

In the "/cm/shared/apps/predector/envs/predector/share/signalp3-3.0b-1/register.sh", if I lanuch :

tar -tf "signalp-3.0.Linux.tar.Z" | head -n 1

I have the error :

tar (child): compress: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

There is a problem with the tar command for sure. I downloaded signalp-3.0.Linux.tar.Z twice, to be sure there is no problem with my first download, but I got the same error.

Expected behavior
I expect the message "signalp-3.0 is now full installed!"

Screenshots or copy-pasted terminal outut
/

Operating system (please enter the following information as appropriate):

CentOS
Conda
Are you using MacOS or Windows WSL? NO

Additional context
/

BUG: SignalP3-NN fails for long sequences.

The Signalp v3 nn model will raise an error if an input sequence is too long.
It's unclear what the threshold is.
Maybe 5000AA -ish?

Currently we just ignore the error and live with it as missing data.
The more correct approach might be to truncate the proteins first, and run it through like that.

I don't think it's critical ATM.
For future perfectionism.

FEATURE REQUEST: signalP6.0 support

Dear Darcy,

SignalP was updated to version 6.0 and should be better than the singnalP5.0. I think we could support it and might get a better result.

Thanks,
Xizhe

BUG: Singularity build fails

Describe the bug
Build crashes using singularity tag

To Reproduce

 curl -s "https://raw.githubusercontent.com/ccdmb/predector/master/install.sh" | bash -s singularity -3 signalp-3.0.Linux.tar.Z     -4 signalp-4.1g.Linux.tar.gz     -5 signalp-5.0b.Linux.tar.gz     -t targetp-2.0.Linux.tar.gz     -d deeploc-1.0.All.tar.gz     -m tmhmm-2.0c.Linux.tar.gz  -p phobius101_linux.tar.gz

**Screenshots or copy-pasted terminal outut **

INFO:    Starting build...
Getting image source signatures
Copying blob sha256:68ced04f60ab5c7a5f1d0b0b4e7572c5a4c8cce44866513d30d9df1a15277d6b
 25.84 MiB / 25.84 MiB [====================================================] 0s
Copying blob sha256:9c388eb6d33c40662539172f0d9a357287bd1cd171692ca5c08db2886bc527c3
 76.50 MiB / 76.50 MiB [====================================================] 1s
Copying blob sha256:96cf53b3a9dd496f4c91ab86eeadca2c8a31210c2e5c2a82badbb0dcf8c8f76b
 48.03 MiB / 48.03 MiB [====================================================] 0s
Copying blob sha256:4c652f7bbbd31c7bdc6db188af3e21e51ba78770396d03ad1b29c193bb334328
 13.56 MiB / 13.56 MiB [====================================================] 1s
Copying blob sha256:cceea6a4aa09dc45cf08aae7f951ac4a96d3f79dd27473c0645f714b2097d193
 429 B / 429 B [============================================================] 0s
Copying blob sha256:d6452039e513df89b320ac1063567b8f1eb9f728b776f2596dcc71e14aeca466
 898.51 MiB / 898.51 MiB [================================================] 2m4s
Copying config sha256:fba358169658187e2332dfaa8d7aac5d4d91878647c487f971a12f452ba941bc
 5.72 KiB / 5.72 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures

INFO:    Running setup scriptlet
+ set +x
singularity image-build: relocation error: /lib/x86_64-linux-gnu/libnss_files.so.2: symbol __libc_readline_unlocked version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference
FATAL:   While performing build: while running engine: exit status 127

ERROR: creating the singularity image failed

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Ubuntu 18.04.4 LTS
Dependency management: Singularity
Nimbus VM

Add check for filename uniqueness

We strip the last extension from the input filenames to make the output filenames nicer.

On the offchance that a user provides multiple files with the only unique content between the two in the extension (e.g. example.proteins1 example.proteins2) then we need to fail right at the start.

People can use the --nostrip option to use the full filename in the output, or they could change their filenames.

BUG: effectorp_v2 error when Test the pipeline

Describe the bug
Execution aborted due to an unexpected error
Error executing process > 'effectorp_v2 (1)'
Caused by:
Process effectorp_v2 (1) terminated with an error exit status (1).

To Reproduce
nextflow run -profile test -with-conda /home/xizhe/anaconda3/envs/predector -resume -r 1.0.0 ccdmb/predector

Expected behavior
figure out what's wrong

**Screenshots or copy-pasted terminal outut **
(predector) xizhe@fisher:~/tools$ nextflow run -profile test -with-conda /home/xizhe/anaconda3/envs/predector -resume -r 1.0.0 ccdmb/predector -latest
N E X T F L O W ~ version 21.04.0
Pulling ccdmb/predector ...
Launching ccdmb/predector [zen_minsky] - revision: b9d1027 [1.0.0]
WARN: It appears you have never run this project before -- Option -resume is ignored
[- ] process > validate_input:download_pfam_hmm -
executor > local (2)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (20)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (21)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (21)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (22)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (22)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (22)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (22)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (22)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (23)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (40)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (40)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (40)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (41)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (42)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (42)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (43)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (43)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (45)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (46)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (48)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
executor > local (49)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
[2d/1f914d] process > validate_input:download_pfam_dat [100%] 1 of 1 ✔
executor > local (49)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
[2d/1f914d] process > validate_input:download_pfam_dat [100%] 1 of 1 ✔
[1c/072d30] process > validate_input:download_dbcan [ 0%] 0 of 1
[19/344584] process > check_env:get_signalp3_version [100%] 1 of 1 ✔
[81/4874da] process > check_env:get_signalp4_version [100%] 1 of 1 ✔
[69/3f29ef] process > check_env:get_signalp5_version [100%] 1 of 1 ✔
[88/010ef1] process > check_env:get_targetp2_version [100%] 1 of 1 ✔
[31/b61678] process > check_env:get_tmhmm2_version [100%] 1 of 1 ✔
[3c/0ef0e1] process > check_env:get_deeploc1_version [100%] 1 of 1 ✔
[04/9937a7] process > check_env:get_phobius_version [100%] 1 of 1 ✔
[72/ed415a] process > check_env:get_effectorp1_version [100%] 1 of 1 ✔
[da/c4822e] process > check_env:get_effectorp2_version [100%] 1 of 1 ✔
[78/9b9c02] process > check_env:get_localizer_version [100%] 1 of 1 ✔
[01/a301c3] process > check_env:get_apoplastp_version [100%] 1 of 1 ✔
[4a/07baa4] process > check_env:get_deepsig_version [100%] 1 of 1 ✔
[b4/5851cc] process > check_env:get_emboss_version [100%] 1 of 1 ✔
executor > local (49)
[13/bc0b85] process > validate_input:download_pfam_hmm [ 0%] 0 of 1
[2d/1f914d] process > validate_input:download_pfam_dat [100%] 1 of 1 ✔
[1c/072d30] process > validate_input:download_dbcan [ 0%] 0 of 1
[19/344584] process > check_env:get_signalp3_version [100%] 1 of 1 ✔
[81/4874da] process > check_env:get_signalp4_version [100%] 1 of 1 ✔
[69/3f29ef] process > check_env:get_signalp5_version [100%] 1 of 1 ✔
[88/010ef1] process > check_env:get_targetp2_version [100%] 1 of 1 ✔
[31/b61678] process > check_env:get_tmhmm2_version [100%] 1 of 1 ✔
[3c/0ef0e1] process > check_env:get_deeploc1_version [100%] 1 of 1 ✔
[04/9937a7] process > check_env:get_phobius_version [100%] 1 of 1 ✔
[72/ed415a] process > check_env:get_effectorp1_version [100%] 1 of 1 ✔
[da/c4822e] process > check_env:get_effectorp2_version [100%] 1 of 1 ✔
[78/9b9c02] process > check_env:get_localizer_version [100%] 1 of 1 ✔
[01/a301c3] process > check_env:get_apoplastp_version [100%] 1 of 1 ✔
[4a/07baa4] process > check_env:get_deepsig_version [100%] 1 of 1 ✔
[b4/5851cc] process > check_env:get_emboss_version [100%] 1 of 1 ✔
[ec/f4c250] process > check_env:get_mmseqs2_version [100%] 1 of 1 ✔
[6a/05d477] process > check_env:get_hmmer_version [100%] 1 of 1 ✔
[86/0f289b] process > encode_seqs [100%] 1 of 1 ✔
[- ] process > signalp_v3_hmm [ 0%] 0 of 2
[2c/6a8bce] process > signalp_v3_nn (1) [ 0%] 0 of 2
[59/e9df70] process > signalp_v4 (1) [ 50%] 1 of 2
[47/940dc0] process > signalp_v5 (1) [ 0%] 0 of 2
[cd/becac1] process > deepsig (2) [ 0%] 0 of 2
[c6/753dce] process > phobius (2) [100%] 2 of 2 ✔
[5c/8c1aa2] process > tmhmm (2) [100%] 2 of 2 ✔
[3f/d8ddef] process > targetp (1) [ 0%] 0 of 2
[9c/5600ec] process > deeploc (2) [ 0%] 0 of 2
[1c/77b652] process > apoplastp (2) [ 50%] 1 of 2
[- ] process > localizer -
[d6/6e25a2] process > effectorp_v1 (2) [ 0%] 0 of 2
[b3/0fc240] process > effectorp_v2 (2) [ 0%] 0 of 2
[bc/aa485f] process > pepstats (1) [100%] 2 of 2 ✔
[- ] process > press_pfam_hmmer -
[- ] process > pfamscan -
[- ] process > press_dbcan_hmmer -
[- ] process > hmmscan_dbcan -
[5d/bd6865] process > mmseqs_index_proteomes (1) [100%] 2 of 2 ✔
[d3/620125] process > mmseqs_index_phibase [100%] 1 of 1 ✔
[- ] process > mmseqs_search_phibase [ 0%] 0 of 2
[bc/cb404b] process > extract_effector_seqs [100%] 1 of 1 ✔
[7b/5ce7ef] process > mmseqs_index_effectors [100%] 1 of 1 ✔
[- ] process > mmseqs_search_effectors [ 0%] 0 of 2
[- ] process > decode_seqs -
[- ] process > gff_results -
[- ] process > tabular_results -
[- ] process > rank_results -
[70/a27288] process > publish_it (4) [100%] 4 of 4
Execution aborted due to an unexpected error

Error executing process > 'effectorp_v2 (1)'

Caused by:
Process effectorp_v2 (1) terminated with an error exit status (1)

Command executed:

run () {
set -e
TMPFILE="tmp$$"
executor > local (49)
[- ] process > validate_input:download_pfam_hmm -
[2d/1f914d] process > validate_input:download_pfam_dat [100%] 1 of 1 ✔
[- ] process > validate_input:download_dbcan -
[19/344584] process > check_env:get_signalp3_version [100%] 1 of 1 ✔
[81/4874da] process > check_env:get_signalp4_version [100%] 1 of 1 ✔
[69/3f29ef] process > check_env:get_signalp5_version [100%] 1 of 1 ✔
[88/010ef1] process > check_env:get_targetp2_version [100%] 1 of 1 ✔
[31/b61678] process > check_env:get_tmhmm2_version [100%] 1 of 1 ✔
[3c/0ef0e1] process > check_env:get_deeploc1_version [100%] 1 of 1 ✔
[04/9937a7] process > check_env:get_phobius_version [100%] 1 of 1 ✔
[72/ed415a] process > check_env:get_effectorp1_version [100%] 1 of 1 ✔
[da/c4822e] process > check_env:get_effectorp2_version [100%] 1 of 1 ✔
[78/9b9c02] process > check_env:get_localizer_version [100%] 1 of 1 ✔
[01/a301c3] process > check_env:get_apoplastp_version [100%] 1 of 1 ✔
[4a/07baa4] process > check_env:get_deepsig_version [100%] 1 of 1 ✔
[b4/5851cc] process > check_env:get_emboss_version [100%] 1 of 1 ✔
[ec/f4c250] process > check_env:get_mmseqs2_version [100%] 1 of 1 ✔
[6a/05d477] process > check_env:get_hmmer_version [100%] 1 of 1 ✔
[86/0f289b] process > encode_seqs [100%] 1 of 1 ✔
[- ] process > signalp_v3_hmm [ 0%] 0 of 2
[2c/6a8bce] process > signalp_v3_nn (1) [100%] 1 of 1
[59/e9df70] process > signalp_v4 (1) [100%] 1 of 1
[47/940dc0] process > signalp_v5 (1) [ 0%] 0 of 1
[- ] process > deepsig (2) -
[c6/753dce] process > phobius (2) [100%] 2 of 2 ✔
[5c/8c1aa2] process > tmhmm (2) [100%] 2 of 2 ✔
[- ] process > targetp (1) -
[- ] process > deeploc (2) -
[1d/955948] process > apoplastp (1) [100%] 1 of 1
[- ] process > localizer -
[d6/6e25a2] process > effectorp_v1 (2) [ 0%] 0 of 1
[b3/0fc240] process > effectorp_v2 (2) [100%] 1 of 1, failed: 1
[bc/aa485f] process > pepstats (1) [100%] 2 of 2 ✔
[- ] process > press_pfam_hmmer -
[- ] process > pfamscan -
[- ] process > press_dbcan_hmmer -
[- ] process > hmmscan_dbcan -
[5d/bd6865] process > mmseqs_index_proteomes (1) [100%] 2 of 2 ✔
[d3/620125] process > mmseqs_index_phibase [100%] 1 of 1 ✔
[- ] process > mmseqs_search_phibase [ 0%] 0 of 2
[bc/cb404b] process > extract_effector_seqs [100%] 1 of 1 ✔
[7b/5ce7ef] process > mmseqs_index_effectors [100%] 1 of 1 ✔
[- ] process > mmseqs_search_effectors [ 0%] 0 of 2
[- ] process > decode_seqs -
[- ] process > gff_results -
[- ] process > tabular_results -
[- ] process > rank_results -
[70/a27288] process > publish_it (4) [100%] 4 of 4
Execution aborted due to an unexpected error

Error executing process > 'effectorp_v2 (1)'

Caused by:
Process effectorp_v2 (1) terminated with an error exit status (1)

Command executed:

run () {
set -e
TMPFILE="tmp$$"
EffectorP2.py -s -i "$1" -o "${TMPFILE}" 1>&2
cat "${TMPFILE}"

  rm -f "${TMPFILE}"

}
export -f run

CHUNKSIZE="$(decide_task_chunksize.sh in.fasta "4" 100)"

parallel --halt now,fail=1 --joblog log.txt -j "4" -N "${CHUNKSIZE}" --line-buffer --recstart '>' --cat run < in.fasta | cat > out.txt

predutils r2js --pipeline-version "1.0.0" -o out.ldjson effectorp2 out.txt

Command exit status:
1

Command output:
(empty)

Command error:
Calculate statistics of protein properties
Calculate statistics of protein properties
Calculate statistics of protein properties
Calculate statistics of protein properties

EffectorP 2.0 is running for 1 proteins given in FASTA file /tmp/par4d2Tr

Call pepstats...
Done.

Scan pepstats file
Done.

Ensemble classification
Error calling WEKA: Calling WEKA returned 1
parallel: This job failed:
run $PARALLEL_TMP

Work dir:
/home/xizhe/tools/work/8d/bd054bec8b41122d8dee62ff353374

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-112-generic x86_64)
Dependency management: Conda
Are you using MacOS or Windows WSL? NO

The ranking description of outputs should be a table.

The ranking table description is very long, and a bit hard to read.
https://github.com/ccdmb/predector/blob/dev/docs/output.md

It might be better to include it as a table, rather than the current enumerated list.

GitHub markdown supports tables. Described here https://github.github.com/gfm/#tables-extension-

I think the format: column index | column title | type | description would be good. Most of this could be extracted from the current list.

BUG: SignalP 6 too slow on Supercomputer.

SignalP 6 works well enough on cloud instances and laptops etc, but it seems to take forever on Zeus.
I suspect it has something to do with the network file-system.

See if we can do something about that.
They also create a lot of files and plots we don't need without any way to turn it off.
That will be killing performance, but shouldn't be as much as what I'm seeing.
Should email them and ask for an option to disable per-protein plots etc.

BUG: Installer silently fails while registering signalp3

I'm aware of an issue that seems to have arisen with recent linux updates.
In my register scripts is a step calling chown -R a+r . just as a precautionary thing to make sure all files were readable by everyone.
In some cases the group user was not set as readable for example.

This now appears to yield an exit code of 1, without any explanation of what's gone wrong.
I'm not sure what it is yet.

This issue here is just to let anyone confused about it know that I'm working on it

Installation fails when testing DeepLoc

I have been trying to install Predector with mamba and after solving an issue related to using an older version of mamba, the installation succeeds, but registration of DeepLoc fails with the following error:

AttributeError: module 'typing' has no attribute '_SpecialForm'

It appears that you need Python >3.6 for that so when I removed the pinning of the specific Python version in the yaml file (conda-forge::python=3.6.13), it installed Python v3.7.12 and the DeepLoc registered successfully.

FEATURE REQUEST: Handle tar de-compression specification properly in conda register scripts

#70 reported an issue when registering signalp-3

It appears to be a problem with tar not recognising the old-style compression (with the .Z suffix).

Normal gzip should be able to handle this, so it might be as simple as specifying the -z flag to tar to force it to use gunzip.

I need to test different distros to see which have uncompress and/or gzip available by default.

FEATURE REQUEST: Please add Prediction result to the *-ranked.tsv

Hi Darcy,

I found that Only the Probability scores of ApoplastP were shown in the rank list. But the score does not indicate whether it is an apoplast effector. I think we should add the Prediction result of ApoplastP to the *-ranked.tsv.

Have a nice weekend!
Xizhe

BUG: docker build crashes

Describe the bug
Using the install instructions for docker crashes

To Reproduce

 curl -s "https://raw.githubusercontent.com/ccdmb/predector/master/install.sh" | bash -s docker -3 signalp-3.0.Linux.tar.Z     -4 signalp-4.1g.Linux.tar.gz     -5 signalp-5.0b.Linux.tar.gz     -t targetp-2.0.Linux.tar.gz     -d deeploc-1.0.All.tar.gz     -m tmhmm-2.0c.Linux.tar.gz  -p phobius101_linux.tar.gz

**Screenshots or copy-pasted terminal outut **

## Setting up docker image.

main: line 532: : command not found
(23) Failed writing body

ERROR: creating the docker image failed

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Ubuntu 18.04.4 LTS
Dependency management: Docker
Nimbus VM

Add a wrapper script for building the environment

What kind of feature are you proposing?

Currently, registering the proprietary source files involves running a few commands, and they're different for conda, docker, and singularity.
It would be good to make this process a single command that can take all source files and build it while you make a coffee.

Describe the solution you'd like
Probably the best solution would be to have something like the nextflow install method.

e.g.

curl -s "https://whereever.com/predector-env.sh" | sh - --signalp3 ./signalp3.tar.gz ...

I'm unsure if you can pipe the script to stdin of sh like that and still provide command line parameters, but it should be possible.

The reason that I like this idea, is that potentially it could make the build process look the same for each of the environments.

curl -s "https://whereever.com/predector-env-conda.sh" | sh - --signalp3 ./signalp3.tar.gz ...
curl -s "https://whereever.com/predector-env-docker.sh" | sh - --signalp3 ./signalp3.tar.gz ...
curl -s "https://whereever.com/predector-env-singularity.sh" | sh - --signalp3 ./signalp3.tar.gz ...

For docker the command is almost identical, but it would simplify things for conda and singularity.
I'm a bit unsure about this. I think the full version should be documented somewhere, but this might be a good "easy-install" method.

Implement effector classification, clustering, and ranking strategy

This is required for the version 1 release.
Progress is being tracked in the project classifier and ranking....

SignalP6 threadlock

Hi Darcy,
Getting the following error when running predictor:

Error executing process > 'signalp_v6 (2)'

Caused by:
  Process `signalp_v6 (2)` terminated with an error exit status (137)

Command executed:

  mkdir -p tmpdir
  
     run () {
         set -e
         TMPDIR="${PWD}/tmp$$"
         mkdir -p "${TMPDIR}"
  
  export OMP_NUM_THREADS=1
         signalp6           --fastafile "$1"           --output_dir "${TMPDIR}"           --format txt           --organism eukarya           --mode fast         1>&2
  
         cat "${TMPDIR}/prediction_results.txt"
  
         rm -rf -- "${TMPDIR}"
     }
     export -f run
  
     CHUNKSIZE="$(decide_task_chunksize.sh in.fasta "4" "100")"
  
     parallel         --halt now,fail=1         --joblog log.txt         -j "4"         -N "${CHUNKSIZE}"         --line-buffer          --recstart '>'         --cat          run     < in.fasta     | cat > out.txt
  
     predutils r2js       --pipeline-version "1.1.0-alpha"       signalp6 "out.txt"     > "out.ldjson"

Command exit status:
  137

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Matplotlib created a temporary config/cache directory at tmp162/matplotlib-hbbo0rts because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  Matplotlib created a temporary config/cache directory at tmp163/matplotlib-5dprkzdr because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  Matplotlib created a temporary config/cache directory at tmp158/matplotlib-qvjqlyvk because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  Matplotlib created a temporary config/cache directory at tmp156/matplotlib-1ryo928c because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  
  
  
  
  Predicting:   0%|          | 0/100 [00:00<?, ?batch/s]/opt/conda/envs/predector/lib/python3.7/site-packages/torch/nn/modules/module.py:1051: UserWarning: where received a uint8 condition tensor. This behavior is deprecated and will be removed in a future version of PyTorch. Use a boolean condition instead. (Triggered internally at  /tmp/pip-req-build-xi8rb33b/aten/src/ATen/native/TensorCompare.cpp:255.)
    return forward_call(*input, **kwargs)
  
  Predicting:   0%|          | 0/100 [00:00<?, ?batch/s]environment: line 1:   169 Killed                  signalp6 --fastafile "$1" --output_dir "${TMPDIR}" --format txt --organism eukarya --mode fast 1>&2
  parallel: This job failed:
  run $PARALLEL_TMP

Work dir:
  /ppgdata/fredrick/assembly_data/ascochyta/FINAL_Assemblies/genome_annotation/work/1d/86a3226589c58fd35a4b5e72410b3c

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

BUG: docker (long install) test fails

Describe the bug
Installed docker version the long way. Installation succeeds, but the test fails.

To Reproduce
nextflow run -profile test,docker -resume ccdmb/predector
If you try to run it several times, it crashes at different bits, depending on which process is run first. But they all seem to relate to environment variables.

Screenshots or copy-pasted terminal outut

Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'check_env:get_apoplastp_version'

Caused by:
  Process `check_env:get_apoplastp_version` terminated with an error exit status (125)

Command executed:

  if ! which ApoplastP.py > /dev/null
      then
          echo -e "Could not find the program 'ApoplastP.py' in your environment path.
  " 1>&2

          echo "Please install ApoplastP using the conda environment." 1>&2

          exit 127
      fi

      VERSION=$(ApoplastP.py -h | grep "^# ApoplastP [[:digit:]]" | sed 's/^# ApoplastP \([[:digit:]]*\.*[^[:space:]]*\).*$/\1/')

Command exit status:
  125

Command output:
  (empty)

Command error:
  Unable to find image 'predector/predector:0.0.1-dev.1' locally
  docker: Error response from daemon: pull access denied for predector/predector, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
  See 'docker run --help'.

Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'check_env:get_signalp3_version'

Caused by:
  Process `check_env:get_signalp3_version` terminated with an error exit status (125)

Command executed:

  if ! which signalp3 > /dev/null
      then
          echo -e "Could not find the program 'signalp3' in your environment path.
  " 1>&2

          if which signalp > /dev/null
          then
              echo "You do have 'signalp' installed, but because we run multiple versions of signalp, we require executables to be available in the format 'signalp3', 'signalp4', 'signalp5' etc." 1>&2
          fi

          echo "Please either link signalp to signalp3 or install signalp using the conda environment." 1>&2

          exit 127
      fi

      VERSION=$(signalp3 -v | sed 's/\([^,]*\).*/\1/')                                                                                                                                                         
Command exit status:
  125                                                                                                                                                                                                          
Command output:
  (empty)                                                                                                                                                                                                      
Command error:
  Unable to find image 'predector/predector:0.0.1-dev.1' locally
  docker: Error response from daemon: pull access denied for predector/predector, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
  See 'docker run --help'.

Error executing process > 'check_env:get_localizer_version'

Caused by:
  Process `check_env:get_localizer_version` terminated with an error exit status (125)

Command executed:

  if ! which LOCALIZER.py > /dev/null
      then
          echo -e "Could not find the program 'LOCALIZER.py' in your environment path.
  " 1>&2

          echo "Please install LOCALIZER using the conda environment." 1>&2

          exit 127
      fi

      VERSION=$(LOCALIZER.py -h | grep "^# LOCALIZER [[:digit:]]" | sed 's/^# LOCALIZER \([[:digit:]]*\.*[^[:space:]]*\).*$/\1/')                                                                              
Command exit status:
  125                                                                                                                                                                                                          
Command output:
  (empty)

Command error:
  Unable to find image 'predector/predector:0.0.1-dev.1' locally
  docker: Error response from daemon: pull access denied for predector/predector, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
  See 'docker run --help'.

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Ubuntu 18.04.4 LTS
Dependency management: Docker
Are you using: Nimbus VM

Deepsig fails with missing tensorflow module

Error message

Error executing process > 'deepsig (2)'

Caused by:
  Process `deepsig (2)` terminated with an error exit status (1)

Command executed:

  CHUNKSIZE="$(decide_task_chunksize.sh in.fasta "4" 100)"
  
  run () {
      set -e
      OUT="tmp$$"
      deepsig.py -f $1 -k euk -o "${OUT}" 1>&2
      cat "${OUT}"
      rm -f "${OUT}"
  }
  
  export -f run
  
  
  parallel         --halt now,fail=1         --joblog log.txt         -j "4"         -N "${CHUNKSIZE}"         --line-buffer          --recstart '>'         --cat run     < in.fasta     | cat > out.txt
  
  predutils r2js         --pipeline-version "1.1.0-beta"         -o out.ldjson         deepsig out.txt

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
  Traceback (most recent call last):
    File "/opt/conda/envs/predector/bin/deepsig.py", line 15, in <module>
      from deepsiglib.helpers import printDate
    File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 13, in <module>
      from keras.preprocessing.sequence import pad_sequences
    File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/__init__.py", line 21, in <module>
      from tensorflow.python import tf2
  ModuleNotFoundError: No module named 'tensorflow'
  Traceback (most recent call last):
    File "/opt/conda/envs/predector/bin/deepsig.py", line 15, in <module>
      from deepsiglib.helpers import printDate
    File "/opt/conda/envs/predector/share/deepsig-0f1e1d9-0/deepsiglib/helpers.py", line 13, in <module>
      from keras.preprocessing.sequence import pad_sequences
    File "/opt/conda/envs/predector/lib/python3.7/site-packages/keras/__init__.py", line 21, in <module>
      from tensorflow.python import tf2
  ModuleNotFoundError: No module named 'tensorflow'
  parallel: This job failed:
  run $PARALLEL_TMP

BUG: Conda registration of signalp-4.1g.Linux.tar.gz & phobius101_linux.tar.gz

Describe the bug
"ERROR: One or more of the proprietary package registrations failed." signalp-4.1g.Linux.tar.gz
"ERROR: One or more of the proprietary package registrations failed." phobius101_linux.tar.gz

**Screenshots or copy-pasted terminal output **
Two provided below, separated by line:

Registering source file signalp-4.1g.Linux.tar.gz for signalp4 into conda environment at:
/home/user/.conda/envs/predector/share/signalp4-4.1g-2

Unregistering old source files if they exist.

patching file signalp
Finished registering signalp4.
Testing installation...
Cwd.c: loadable library and perl binaries are mismatched (got handshake key 0xdb80080, needed 0xde00080)

phobius-register phobius101_linux.tar.gz
Registering source file phobius101_linux.tar.gz for phobius into conda environment at:
/home/user/.conda/envs/predector/share/phobius-1.01-2

Unregistering old source files if they exist.

patching file phobius.pl
Finished registering phobius.
Testing installation...
Fcntl.c: loadable library and perl binaries are mismatched (got handshake key 0xdb80080, needed 0xde00080)

Operating system (please enter the following information as appropriate):

OS/Linux distribution: Linux on HPC
Dependency management: Conda
HPC

BUG: Mamba environment installation problem

Hi,
I am having problems installing Predator. Its says that I already have a Predator environment but I do not. I also give with the -c flag the path to where to install the environment and it give me the same error. Could you help me please?

I run curl -s "https://raw.githubusercontent.com/ccdmb/predector/1.2.5/install.sh" | bash -s mamba -3 signalp-3.0.Linux.tar.Z -4 signalp-4.1g.Linux.tar.gz -5 signalp-5.0b.Linux.tar.gz -6 signalp-6.0g.fast.tar.gz -t targetp-2.0.Linux.tar.gz -d deeploc-1.0.All.tar.gz -m tmhmm-2.0c.Linux.tar.gz -p phobius101_linux.tar.gz

Setting up conda environment.

conda-forge/linux-64 Using cache
conda-forge/noarch Using cache
bioconda/linux-64 Using cache
bioconda/noarch Using cache
pkgs/main/linux-64 [====================] (00m:00s) No change
pkgs/main/noarch [====================] (00m:00s) No change
pkgs/r/linux-64 [====================] (00m:00s) No change
pkgs/r/noarch [====================] (00m:00s) No change
darcyabjones/noarch [====================] (00m:00s) No change
predector/noarch [====================] (00m:00s) No change
darcyabjones/linux-64 [====================] (00m:00s) No change
predector/linux-64 [====================] (00m:00s) No change

Looking for: ['predector::apoplastp=1.0.1', 'predector::deeploc==1.0=h6bb024c_3', 'predector::deepsig=0f1e1d9', 'predector::effectorp1=1.0', 'predector::effectorp2=2.0', 'predector::effectorp3==3.0=py_2', 'predector::localizer=1.0.4', 'predector::phobius==1.01=4', 'predector::predectorutils=0.8.2', 'predector::signalp3==3.0b=3', 'predector::signalp4==4.1g=3', 'predector::signalp5==5.0b=2', 'predector::signalp6==6.0g=1', 'predector::targetp2==2.0=2', 'predector::tmhmm==2.0c=3', 'predector::r-deepredeff=0.1.1', 'conda-forge::gawk', 'conda-forge::gzip', 'conda-forge::wget', 'conda-forge::parallel', 'conda-forge::r-optparse', 'h5py==2.10.0[build=nompi_]', 'tensorflow==2.2.0[build=mkl_]', 'conda-forge::keras=2.4.3', 'conda-forge::numpy=1.19.5', 'bioconda::perl-moose=2.2011', 'conda-forge::perl=5.26.2', 'bioconda::emboss=6.6.0', 'bioconda::hmmer=3.3.2', 'bioconda::pfam_scan=1.6', 'bioconda::mmseqs2=13.45111', 'conda-forge::pytorch-cpu=1.9.0', 'conda-forge::python=3.6.13', 'conda-forge::r-base=4.1.1', 'conda-forge::sqlite', 'conda-forge::openjdk=11.0.1', 'conda-forge::theano=1.0.5', 'conda-forge::xz', 'conda-forge::zlib', 'conda-forge::bzip2', 'pip']

Encountered problems while solving.
Problem: nothing provides __glibc >=2.17,<3.0.a0 needed by pytorch-1.9.0-cpu_py36h1c7b8ea_2
Problem: nothing provides __glibc >=2.17,<3.0.a0 needed by pytorch-1.10.0-cpu_py37h2761dfd_1

ERROR: creating the conda environment failed
Usually this happens if you already have an environment with the name or path prefix that we tried to use.
You can change the name we use with the '-n' flag, or the path prefix with the '-c' flag.
Otherwise you can remove the old environment with:
conda env remove --name predector
conda env remove --prefix path/to/your/env

Operating system (please enter the following information as appropriate):
-I am using conda 4.11.0 ; and CentOS Linux release 8.2.2004

ccdmb / predector Goto Github PK

predector's People

Contributors

Stargazers

Watchers

Forkers

predector's Issues

Command error: Calculate statistics of protein properties Calculate statistics of protein properties Calculate statistics of protein properties Calculate statistics of protein properties

Error message

Setting up conda environment.

Recommend Projects

Recommend Topics

Recommend Org

Command error:
Calculate statistics of protein properties
Calculate statistics of protein properties
Calculate statistics of protein properties
Calculate statistics of protein properties