genotoul-bioinfo / dgenies Goto Github PK

Hi @genotoul-bioinfo,

I would like to know if it is possible to run dgenies using only the Linux command-line.

Thanks in advance,
Francisco.

I tried using D-genies from the following URL.
https://dgenies.toulouse.inra.fr/run

In option section,
I can select 3 mode.
"Few repeats"
"Some repeats"
"Many repeats"

What is the difference between the above three? The results are slightly different for the three?

After launching a batch we can follow its progress with the log page. It would be nice to know which query and reference each line corresponds to.

Hi there,

I am trying to run d-genies on my Mac OS Catalina 10.15.6, and locally installed it with the option --user. Everything appears to install okay, but using the command dgenies run in standalone mode yields me nothing instead of launching the program as expected. I have tried installing it through both pip and gitclone and I have the same issue in both cases. I have python v 3.6.

m@MacBook-Pro-4 dgenies % dgenies run
zsh: command not found: dgenies

Can you tell me the default location for the executable file so that I can add it to my path? I can't seem to find it in any of the current bin folders in my path. Apologies for the rookie mistake, I am inexperienced with installing programs like this on my local computer.

Thank you for your help!

Add the possibility to upload query and target bed files to plot elements on the border of the dot plot such as repeat or gene content or specific locations of interest.

Use built-in function secure_filename from werkzeug.utils instead of the handmade one in dgenies.

Hi,

After "Your job was completed successfully. Please click here to show results." with the .paf and .idx file as input.
I get the following error:
Invalid contig for query: XX050 May be you invert query and target files?

I was wondering why i am getting this error? I have tried both query and target file as index.

Thanks

Add possibility to have a banner/header/footer in order to display messages, notice and links without editing package code itself

Describe the bug

When running D-Genies for the first time, if more than one thread/worker is set, the database initialization produces the following error. It appends when D-Genies is run with on more thread and needs to create tables in database.

[2022-01-26 13:14:16 +0100] [27941] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 3830, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.OperationalError: table "job" already exists

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/gunicorn/util.py", line 412, in import_app
    app = app(*args, **kwargs)
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/__init__.py", line 69, in launch
    from dgenies import views

  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/views.py", line 15, in <module>
    from dgenies.lib.job_manager import JobManager
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/lib/job_manager.py", line 34, in <module>
    from dgenies.database import Job
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/database.py", line 145, in <module>
    Job.create_table()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 5037, in create_table
    db.create_table(cls)
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 3914, in create_table
    return self.execute_sql(*qc.create_table(model_class, safe))
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 3837, in execute_sql
    self.commit()
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 3656, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 135, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/venv/dgenies/lib/python3.6/site-packages/peewee.py", line 3830, in execute_sql
    cursor.execute(sql, params or ())
peewee.OperationalError: table "job" already exists
[2022-01-26 13:14:16 +0100] [27941] [INFO] Worker exiting (pid: 27941)

Expected behavior
The database must be create and D-Genies must not produce error

Workaround

Start D-Genies with one thread/worker once when it appends, and then run with as many thread you want.

Additional context

Tested with gunicorn in webserver mode

If you give two files with the same names as input dgenies aligns subject versus subject.

Improving the adjustment of the chromosomes name to the display space.
Modify the way of display the name on zoom.

Describe the solution you'd like
add the possibility to have batch lines starting with '#' which will be comment lines and will not be ran.

Add the ability to undo a running job

Describe the bug
dgenies clear produces error

To Reproduce

$ dgenies clear
dgenies clear
Nothing to do.
Traceback (most recent call last):
  File "/bin/dgenies", line 343, in <module>
    if args.crons:
AttributeError: 'NoneType' object has no attribute 'crons'

Expected behavior
No error

Computer (please complete the following information):

• python3.5

Describe the bug
If email is forgotten when submitting, dgenies is then stuck at 'Uploading files...' when submitting a second time after adding the email.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Run'
2. Click on 'Load example', or add local files as Target & Query fasta
3. Click on 'Submit'
4. You get the 'Email is required!' message
5. Add an email in 'E-mail' field
6. Click on 'Submit' again
7. You are stuck at 'Uploading files...'

Expected behavior
The analysis must start as if we do the 1, 2, 5, 3 steps

Computer (please complete the following information):

• OS: linux
• Browsers: Chromium 94.0.4606.81, Firefox 94.0.1

Happy New Year 2019 to the D-GENIES team!

System info
I am running D-GENIES on Ubuntu 18.04.1 LTS inside VirtualBox 6.6.0 on my MacOSX laptop with 8G RAM. So my RAM available on the virtual machine is 6.5GB

Several of my pairwise genome alignment runs locally on the Apache2 HTTP web server complete successfully. I am using gzip fasta files that are originally up to 2GB large. But so far I've only tested with max input file size of ~ 650MB & ~ 650MB pair of files.

Problems
However, some of the runs fail
A. with no explicit reason provided, or
B. because I ran out of memory for that run.

Modification tried, but with with no success
I saw your post for resolved issue #3, here about altering application.properties.
I bumped up input file size to 4GB, but had to reduce memory available to 6.5GB.

How to free up disk space?
Furthermore, it appears I have a lot of disk space that is occupied by files that have appeared only after I installed and stared using D-GENIES (I think):
572K anand/.cache/pip/http/6/7/2
948K anand/.cache/pip/http/6/7
256K anand/.cache/pip/http/6/8/4/6/f
260K anand/.cache/pip/http/6/8/4/6
264K anand/.cache/pip/http/6/8/4
Interestingly, I am unable to individually list these files / folders!

My questions are:

1. How can I free up my disk space?
2. Is there any work-around for using lower RAM? i can only use local machine since university cluster does not support Virtual Machines.
3. Is there an empirical guesstimate for RAM required based on input file size?
4. For input files that will require more RAM than I have on my local machine, is there any work-around?

Since I am a newbie to D-GENIES, VirtualBox and APACHE2, my questions may be rudimentary. But thanks very much for helping with my troubleshooting.

Cheers,
Anand

Ava mode is available when only target input is filled

Hello,
When I used the online D-GENIES, an unexpected error has occurred on upload. And then, it showed "Please contact the support".

Describe the bug

Got following message when installing dgenies with python 3.10.

	Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-7f5563xt/peewee_70af0ed678bd4596af0e43380e5dfe5b/setup.py", line 64, in <module>
        version=__import__('peewee').__version__,
      File "/tmp/pip-install-7f5563xt/peewee_70af0ed678bd4596af0e43380e5dfe5b/peewee.py", line 124, in <module>
        from collections import Callable
	ImportError: cannot import name 'Callable' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)

Seems related to peewee module

To Reproduce

Edit requirement.txt as follow and install dgenies with pip install . command

Flask==1.0.*
Flask-Mail==0.9.*
Jinja2~=2.11.3
peewee==2.10.2
numpy
requests~=2.20.1
biopython>=1.70
python-crontab>=2.2.*,<=2.6.*
psutil~=5.6.6
tendo==0.2.*
matplotlib>=2.1.*,<3.5
drmaa==0.7.*
intervaltree==2.1.*
Markdown==2.6.*
pyyaml~=5.4.1

Computer (please complete the following information):

Tested with python:3.10-slim Docker image with build-essential package

I am aligning a genome assembly to itself to identify regions which have been repeated, and I know a large portion of my assembly is duplicated and has around 80-90% identity, but this does not display on D-Genies? I only see the regions that match 100% (which I also see on mummer, using either -q or -r)

Is there a way to get around this on D-genies?

Hello again,

How does Dgenies compute identity percentage?
I have a PAF file between 2 draft assemblies, with an average identity of 97%. However, in Dgenies, the dot plot shows up in yellow, as if the identity was between 0 and 25%. I did not find anything special with that particular PAF file produced by minimap2. It looks perfectly standard.

I can provide the PAF file if requested.
I am on Ubuntu 18.04.

Thank you, and thanks for Dgenies it's really useful to have such an easy way of doing dotplots.

I want to run a dot plot in D-Genies. But, when I submit the job, it prepares the sequences for long time and does not progress further. I have obtained output earlier within 2 mins. But now, it does not show a progress at all. Kindly advise possible reasons for this and how to resolve this.

Describe the bug
I am running D-Genies inside a docker container. Everything worked perfectly. But since a few days, I got many method not allowed http error messages. Most of the time, when I tried to upload files, but also sometimes for images, like success.svg.

I compared an old and new docker images and recognized that werkzeug was updated to 2.1.0 2 days ago (2022-03-28). After switching back to 2.0.3, everything works again.

Screenshots

Computer (please complete the following information):

• OS: linux, docker
• Browser: Chrome, Firefox, Edge
• Version: latest

Hello!

Thank you for the great tool. I've been using D-Genies to help visualize de novo genomes and it looks like the website may be down. I haven't been able to access it for a week or so and have reset my system/used other peoples computers. I was able to use the website and view results as of this past August. Is the website still maintained?

Thanks!
Dustin

Hello,
I've made a conda package for dgenies (bioconda/bioconda-recipes#21621). I have one problem with it: the configuration directory gets installed in $PATH_TO_CONDA_ENV/lib/python3.6/site-packages/etc/dgenies/
And it seems like there is not way to tell dgenies to look into this directory
Would it be possible to make it more flexible, like using a DGENIES_CONF_DIR env variable if present?

Dear sir/madam,

First, thank you for creating this valuable tool.

I installed it successfully on one of our servers without root access and it works fine.
Several dotplots were created already for genomes up to 1 gbp with both zipped and unzipped fasta as I changed the max sizes in the application.properties file.

The problem I encounter is that when I use the output of a 10x genomics supernova raw.fasta output the mapping works successfully but upon display chromium fails after a while.

Now I wonder if this is just a timeout because the loading of the plot takes too long. If so, than this seems inherent to using chromium since it is impossible to increase the timeout setting -,-

Or is it the 10x supernova output that is somehow complicating matters?

If it is a timeout issue, would it be possible to implement firefox?

Your views are much appreciated.

Sincerely,

EDIT:
My bad, I just ran it on another server that had firefox as default browser and it opened firefox by itself. It was indeed a timeout and resolved by using firefox.

Dear,

Thank you for the excellent plotting software.

The setting max_nb_lines controls how many highest scoring alignments to include in the dot plot, and is fixed in the source code to be 100000. For fragmented assemblies with numerous contigs, this value is grossly insufficient.

I would love to see this parameter exposed, either:

• In the webform as an input box
• In the configuration file application.propertites

When I experimented with this parameter for this use case, I noticed that there are additional performance improvements necessary for the use case when lots of contigs are involved:
#15

Haibao

Hi,

The INSTALL.md file states:

Configuration file location:

Linux:
/etc/dgenies/application.properties if installed with root access
~/.dgenies/application.properties else

On HPC clusters, it is very common to install software centrally in user space (and therefore not to install it as root). Due to security reasons (the cluster is shared by 3000 users), we cannot allow software to install files in /var or /etc. But on the other hand, it is also not very suitable to have the config file in the users home, as I would like to do a central installation in a location like

/cluster/apps/python/3.6.1

such that all users can use the software.

Would it be possible to add an option to specify a path, where the config files are stored ?

Which locations in the code would I need to change to set another path for storing the configuration files ?

Computer (please complete the following information):

• OS: CentOS Linux release 7.4.1708 (Core)

Hello,

I have a maf file that Dgenies refuse to use because it is greater that the "1 Gb limit". Is there a way to increase that limit?

Thank you

Minimap2 produces a lot of alignments using the default -f parameter for repeated genomes
-f FLOAT filter out top FLOAT fraction of repetitive minimizers [0.0002]
Knowing the genome repeat content enables to increase this parameter and speed up the alignment process by decreasing the number of produced alignments.
This will enable to run dgenies from the demo website for repeated genomes.

Describe the bug
I found the website of the dgenies can't be connected (http://dgenies.toulouse.inra.fr/install).
I can't find any other place to read the manual. Can you check the website? Thank you!

Describe the bug
Crash at start

To Reproduce
dgenies run

Expected behavior
no crash

Screenshots

$ dgenies run
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
Traceback (most recent call last):
  File "/usr/local/bin/dgenies", line 341, in <module>
    run(*args)
  File "/usr/local/bin/dgenies", line 171, in run
    app = launch(mode=mode, debug=debug)
  File "/usr/local/lib/python3.7/dist-packages/dgenies/__init__.py", line 40, in launch
    UPLOAD_FOLDER = config_reader.upload_folder
AttributeError: 'AppConfigReader' object has no attribute 'upload_folder'

Computer (please complete the following information):
Debian Buster.
Python 3.7.3
Installed via pip3 as root

Describe the bug

When launched with mod_wsgi, the crontab is populated as follow (you need to access the website once to populate the crontab - lazy loading from apache):

$ sudo crontab -l -u dgenies

0 1 * * * /usr/bin/python3 /usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/bin/clean_jobs.py > /home/dgenies/clean.log 2>&1 # dgenies
* * * * * /usr/lib/python3.6/site-packages/dgenies/bin/start_local_scheduler.sh /usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies /usr/local/venv/dgenies/bin/python3 /home/dgenies/.local_scheduler_pid /home/dgenies/logs/local_scheduler.log > /dev/null 2>&1 & # dgenies

This problem doesn't exists when running dgenies with flask:

# We clear the crontab
sudo -u dgenies /usr/local/venv/dgenies/bin/dgenies clear -c

# We run dgenies with flask
sudo -u dgenies /usr/local/venv/dgenies/bin/dgenies run -m webserver --no-browser -o 0.0.0.0

It will produce following crontab:

$ sudo crontab -l -u dgenies

0 1 * * * /usr/local/venv/dgenies/bin/python3 /usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/bin/clean_jobs.py > /home/dgenies/clean.log 2>&1 # dgenies
* * * * * /usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies/bin/start_local_scheduler.sh /usr/local/venv/dgenies/lib/python3.6/site-packages/dgenies /usr/local/venv/dgenies/bin/python3 /home/dgenies/.local_scheduler_pid /home/dgenies/logs/local_scheduler.log > /dev/null 2>&1 & # dgenies

Computer (please complete the following information):

• OS: CentOS 7 + epel repository
• packages:
  • httpd
  • python3-mod_wsgi
  • dgenies in a venv /usr/local/venv/dgenies

For repeated genomes such as Zea mays, the dgenies scaffolding procedure does not generate a correct output. Contigs are in the correct order but have often wrongly orientation.

To Reproduce
You can reproduce the problem using the following files
alignment.paf
Zm-B73-REFERENCE-NAM-5.0.idx
ZmWGA05a-01.bp.p_ctg.gfa.idx

Expected behavior
All contigs should be oriented as the reference.

Screenshots
example

Hello,
when calling "dgenies run" I get the following error message
Traceback (most recent call last): File "/usr/local/bin/dgenies", line 4, in <module> __import__('pkg_resources').run_script('dgenies==1.2.0', 'dgenies') File "/home/alessandro/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 661, in run_script self.require(requires)[0].run_script(script_name, ns) File "/home/alessandro/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1432, in run_script .format(**locals()), pkg_resources.ResolutionError: Script 'scripts/dgenies' not found in metadata at '/home/alessandro/.local/lib/python3.6/site-packages/dgenies-1.2.0.dist-info'

OS is Ubuntu 18.04.1

I am not familiar with python though.
Thank you

The result email could be enhanced by adding information such as :

• aligner
• parameters (repeats)
• elapsed time

While running dgenies by using command dgenies run I am getting following output:

No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
No section: 'global'
Traceback (most recent call last):
File "/usr/local/bin/dgenies", line 341, in
run(*args)
File "/usr/local/bin/dgenies", line 171, in run
app = launch(mode=mode, debug=debug)
File "/usr/local/lib/python3.6/dist-packages/dgenies/init.py", line 40, in launch
UPLOAD_FOLDER = config_reader.upload_folder
AttributeError: 'AppConfigReader' object has no attribute 'upload_folder'

• OS: Linux (Lubuntu)
• Browser firefox

If I try to "plot alignment" on the d-genies website, I select a .paf file to use and it works fine.

I generate an index file using the python3 script on that page, works great. I name that file xyz.index. I upload xyz.index to target file or query file, I get an error File xyz.index is not supported!.

I downloaded the results of your example and saw that you name the indices .idx. I renamed xyz.indez to xyz.idx and then it worked.

Would you consider specifying on this page http://dgenies.toulouse.inra.fr/run that the index file must be named .idx to work with your tool? That might help someone like me in the future.

Thank you for writing and maintaining D-Genies!

Is your feature request related to a problem? Please describe.
It isn't really a problem, but it would be nice if I could upload a compressed backup file, as saving them compressed saves a lot of space.

Describe the solution you'd like
Be allowed to upload compressed files.

Additional context
The decompression is done automatically by python, the only thing that has to change is the set of allowed file extensions.

Describe the bug
dgenies enable to reverse queries on the dotplot and to download the modified query fasta file.
The expected behavior is that all the reversed queries should be correctly (as modified) oriented in the downloaded query fasta file.
But only the last reversed contigs is reversed in the download fasta file.

To Reproduce
Steps to reproduce the behavior:

1. upload target and query fasta files
2. generate the dotplot and reverse some alignments
3. download the query fasta file
4. upload the target and modified query file

Expected behavior
The expected behavior is that all the reversed queries should be correctly oriented in the downloaded query fasta file.

Computer (please complete the following information):

• OS: linuyx
• Browser: firefow
• Version: 109.0 (64 bits)

Describe the bug
job submission fail with default slurm native_spec in job_manager.py

native_specs = "--mem-per-cpu={0} --mincpus={1} -N 1=1 --time={2}"

To Reproduce
Use a recent slurm

Solution
Use the default proposed in comment in applications.properties

native_specs = "--mem-per-cpu={0} --mincpus={1} --time={2}"

It would be nice to have a selector in the user interface to automatically add swapped chromosome background colors such as in
https://www.biorxiv.org/content/10.1101/2022.03.31.486393v1.full.pdf Fig 3.

Hi,

Thanks for the really useful tool. Apologies if I have missed this somewhere obvious, but can you let me know what settings you use within D-GENIES for the minimap2 and mashmap searches? I would like to pre-generate the PAF files to upload, but want to make sure that the results will be consistent with uploading the assemblies directly.

Many thanks,

Rich

Things that can be useful:

• Provide user the command line used for alignment. It can be something displayed when loading the plot, or a file that can be downloaded in the viewer menu.
• Also provide software version if possible.
• Add it in the backup file (but ignore it when backup is reuploaded)

Describe the bug
minimap2 key must exists in tools.yaml

To Reproduce
Steps to reproduce the behavior:

1. In tools.yaml Replace minimap2 key by something else like minimap2-2.11and set the minimap2 path in exec subkey llike following:

minimap-2.11:
  exec: /usr/local/lib/python3.9/site-packages/dgenies-1.2.0.2-py3.9.egg/dgenies/bin/minimap2

2. Start dgenies
3. Try to run a job by choosing minimap2-2.11 as aligner
4. You got the following log

[2021-11-25 15:22:29,062] ERROR in app: Exception on /status/sjHBq_20211125152224 [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 2317, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1840, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1743, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.9/site-packages/flask/_compat.py", line 36, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1838, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1824, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.9/site-packages/dgenies-1.2.0.2-py3.9.egg/dgenies/views.py", line 303, in status
    job = JobManager(id_job)
  File "/usr/local/lib/python3.9/site-packages/dgenies-1.2.0.2-py3.9.egg/dgenies/lib/job_manager.py", line 80, in __init__
    self.tool = self.tools[tool] if tool is not None else None
KeyError: 'minimap2'

Expected behavior
Run starts

In order to simplify dgenies usage it would by interesting to be able to give a tabulated file with the URL of the different genomes to compare and be able to run them all together. The monitoring would be performed on the same page for all the jobs.

"New mapping VS Plot existing one" graphs could become "Analysis type" with : alignment, plot, batch

Describe the bug
In webserver mode, when asking for Query fasta file, D-Genies try to send an email when send_mail_status = False in configuration file application.properties

127.0.0.1 - - [30/Mar/2022 13:14:39] "POST /get-fasta-query/7Oiu3_20220207162341 HTTP/1.1" 200 -
Exception in thread Thread-46:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 1254, in run
    self.function(*self.args, **self.kwargs)
  File "/.../dgenies/src/dgenies/lib/functions.py", line 319, in sort_fasta
    Functions.send_fasta_ready(mailer, job_name, sample_name, compress)
  File "/.../dgenies/src/dgenies/lib/functions.py", line 266, in send_fasta_ready
    mailer.send_mail([Functions.get_mail_for_job(job_name)], "Job %s - Download fasta" % job_name, message,
  File "/.../dgenies/src/dgenies/lib/mailer.py", line 60, in send_mail
    self._send_async_email(msg)
  File "/.../dgenies/dgenies/src/dgenies/lib/mailer.py", line 33, in _send_async_email
    self.mail.send(msg)
  File "/.../venv/lib/python3.8/site-packages/flask_mail.py", line 491, in send
    with self.connect() as connection:
  File "/.../venv/lib/python3.8/site-packages/flask_mail.py", line 144, in __enter__
    self.host = self.configure_host()
  File "/.../venv/lib/python3.8/site-packages/flask_mail.py", line 158, in configure_host
    host = smtplib.SMTP(self.mail.server, self.mail.port)
  File "/usr/lib/python3.8/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib/python3.8/smtplib.py", line 339, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib/python3.8/smtplib.py", line 310, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/lib/python3.8/socket.py", line 808, in create_connection
    raise err
  File "/usr/lib/python3.8/socket.py", line 796, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

To Reproduce
Steps to reproduce the behavior:

1. Launch D-Genies in webserver mode with send_mail_status = False in application.properties file
2. Go to a result page, do a run if needed.
3. Click on 'sort contigs' button
4. Click on 'export' and choose 'Query fasta'
5. See error on server side

Expected behavior
Do not send a email and allow to download the file when ready from result page.

Describe the bug
After starting the job and uploading the files, the refreshing job page says "This job does not exists!". It continues refreshing but no progress information is displayed and it does not appear to be doing anything.

To Reproduce
Steps to reproduce the behavior:
I am not sure how to reproduce the bug per se. I installed dgenies via conda i.e.

conda create -n dgenies -c bioconda dgenies

I had to manually edit the application.properties file to allow a larger file size, as it suggests in one issue.
I then run dgenies run to start the server. After this, the browser window pops up and I select my files, which are uploaded (green progress bar shows upload). I used minimap2 default.

Then I get the page saying that the job does not exist.

Expected behavior
I expect the job to exist/complete after submission.

Screenshots

Computer (please complete the following information):

• OS: Mac OSX 10.15.7
• Browser chrome 90.0.4430.85
• Version 1.2.0 (conda)

Additional context
I am uploading a gzipped hg38.fa.gz, and comparing it to a large gzipped fully diploid human assembly.

It's possible that too large of a file will break dgenies? I cannot see what is going on under the hood, I am not sure where there is e.g. logging to check.

Add an error message if the paf file does not correspond to the minimap2 format.