hivdb / sierra Goto Github PK

View Code? Open in Web Editor NEW

14.0 8.0 7.0 67.85 MB

HIVDB Genotypic Resistance Interpretation Program

License: GNU General Public License v3.0

Shell 0.37% Java 98.40% Makefile 0.43% Dockerfile 0.79%

sierra's Introduction

Sierra Project - HIVDB Genotypic Resistance Interpretation Program

Prerequisites

Here're the full development prerequisites of Sierra project:

JDK 17 (tested with Oracle Java; also work with OpenJDK).
PostAlign Alignment Program. It is an open source codon-aware alignment program written by Philip Tzou (philiptz(at)stanford.edu) of the Stanford HIVDB team. One can retrieve the source code and binary file from the GitHub repository.
Docker.

Following system environment is required for running sequence alignment:

POSTALIGN_PROGRAM: path to the PostAlign command

For host a production server of Sierra project, only Docker is required.

Dependency Lists

Full dependency list of each sub-project can be found in build.gradle file under each sub-project directory. Here's an unfinished list of main dependencies:

Note: These dependencies are automatically installed by Gradle which is shipped with this repository. You don't need to install them manually.

Installation

Start Sierra with Docker

Docker images are released publicly for each version of Sierra since 2.2.6. To pull and start a Sierra instance (this is how we host our production server):

docker pull hivdb/sierra:latest
docker run -it --publish=8111:8080 hivdb/sierra dev

After started the instance, the local Sierra web service is accessable through this URL:

http://localhost:8111/sierra/rest/graphql

Before Sierra 3.2, the entry-point was:

http://localhost:8111/WebApplications/rest/graphql

Development with Eclipse IDE

Sierra project uses Gradle to manage the dependencies, build and test. The easiest way to install the whole project is through Eclipse. Here're the steps:

Use Git to clone this repository to a local path;
In Eclipse, click "File" > "Import...", expand the "Gradle" group, select "Gradle Project", then click "Next >";
Eclipse may show a welcome dialog before it starting the import wizard. After the welcome dialog, type sierra local path from step #1 into "Project root directory". Click "Next >";
Use all default configurations until finish dialog. Click "Finish" then you have installed sierra in Eclipse.

There may be an issue that you can't find "Gradle Project" in step #2. In this case you can install the latest "Buildship Gradle Integration" with "Help" > "Eclipse Marketplace...". then you can import a Gradle project in Eclipse.

Development server

To start a development server:

make dev

Donation

If you find Sierra useful and wish to donate to the HIVDB team, you can do so through Stanford Make a Gift form. Your contribution will be greatly appreciated.

release

./scripts/release_stable_version <customized version name>

sierra's People

Contributors

Stargazers

Watchers

Forkers

owenlancaster caspater bonsalldavid philiptzou arodcasanova gmbzero luterc

sierra's Issues

HIVDB 8.5/8.6 not working in HIVDB program

Error! GraphQL error: Exception while fetching data (/sequenceReadsAnalysis[0]/drugResistance[1]/levels) : Expect one DrugSusc data for DOR but received 0 GraphQL error: Exception while fetching data (/sequenceReadsAnalysis[0]/drugResistance[1]/drugScores) : Expect one DrugSusc data for DOR but received 0

Error encountered while browsing the Sierra container using local host

Hi, followed by the readme intructions, I've managed to pull down the latest docker image to local IP address (172.30.10.31), then I ran the following code:

docker run -d --publish=8111:8080 hivdb/sierra dev

Then I tried to access the local Sierra web service via our local weblink (http://172.30.10.31:8111/sierra/rest/graphql) but failed with # HTTP error code 405 - Method Not Allowed.

Does anyone know if there's anything wrong with my code or the local site. Thanks in advance!

`TestResources` should not be included except test environment

Check the comment in f6a436b.

@arodcasanova

converting .json to .csv output files command line

Hi,

I'm using command line sierrapy to submit .fasta sequences, do you have a script to convert the .json output to the .csv outputs that are generated using the machine readable setting of the online interface? i.e. sequenceSummaries.csv.

This will really help with automating our reports.

Thanks.

Reference Genome

Hello,

I was wondering abou what reference HIV-1 version does sierra uses. According to the releases notes presented in https://hivdb.stanford.edu/page/release-notes/, the platform uses the Consensus B sequence present in the LANL HIV database. However, a new version of the Consensus sequence seems to be available in LANL HIV according to this article: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8843389/

In the present day, is the reference sequence the Consensus B sequence generated in early 2000's or the recent Consensus B (2021)?

Thank you

hivdb capsid program error

Page: https://hivdb.stanford.edu/hivdb-capsid/by-sequences/
Sequence: https://www.ncbi.nlm.nih.gov/nuccore/KF426383.1?report=fasta

Error message: Graphql 400

"type": "DataFetchingException",
      "message": "Exception while fetching data (/sequenceAnalysis[0]/subtypes[8]/displayWithoutDistance) : java.lang.reflect.InvocationTargetException",
      "locations": [
        {
          "line": 65,
          "column": 5,
          "sourceName": null
        }
      ],

Request to /WebApplications/rest/hivdb-version/rules-comparison/8.3:8.4/NRTI.tsv failed

It seems a 500 error.

Apobec.generateComment()'s output for 0 mutations is grammatically weird.

See Utilities -> ApobecTest.testZeroMuts() to observe current behavior.

Current output: "The following 0 APOBEC muts were present in the sequence: ."
Possible revisoin: "There are no mutations present in this sequence."

Cannot access API locally using Docker

Dear Sierra developers,

First, thanks for your work on this awesome project. I am trying to run a local Sierra instance in order to query drug resistance scores for sequences that must remain private. In short, I was able to execute the query for the database version, but not for the sequence analysis example.

The Docker startup command on README.md did not work for me (connection refused), but a slight modification did (mapping container port to local port):

docker run -it -p 8080:8080 hivdb/sierra dev

Running in this manner, I was able to query for the version as in the minimal working example.

However, I am unable to execute the sequence analysis query. I've captured the HTTP request made from the GraphiQL interface to ensure that I was making a valid submission to the Sierra API, using Postman. I submit an identical JSON Body to http://localhost:8080/WebApplications/rest/graphql, and receive the following error:

  "errors": [
    {
      "type": "DataFetchingException",
      "message": "Exception while fetching data (/sequenceAnalysis) : null",

This same request, when made to https://hivdb.stanford.edu/graphql, returns the desired result. The version query made to this URL also works. I've attached the full JSON error in case it helps.

I hope what I've written makes sense; any assistance you can provide in getting this query working would be greatly appreciated. Thanks in advance for your response!

Best,
Stephen

error.txt

Graphql, adjustedAlignedAAs, Invalid targetStrain #5

"type": "DataFetchingException",
"message": "Exception while fetching data (/sequenceAnalysis[0]/alignedGeneSequences[0]/adjustedAlignedAAs) : Strain modifier for target strain HI is not defined",

Refactor `MutationComboScores.matchComboScore`

As this discussion stated, we should refactor this method to avoid mutating parameter matchedMutsListMap.

docker hivdb/sierra:3.2.2 error

WARNING [main] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime A provider edu.stanford.hivdb.web.GraphQLService registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider edu.stanford.hivdb.web.GraphQLService will be ignored.

WARNING [main] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime A provider edu.stanford.hivdb.web.HivdbVersionService registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider edu.stanford.hivdb.web.HivdbVersionService will be ignored.

WARNING [main] org.glassfish.jersey.internal.inject.Providers.checkProviderRuntime A provider edu.stanford.hivdb.web.HIV2GraphQLService registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider edu.stanford.hivdb.web.HIV2GraphQLService will be ignored.

Add AWS Lambda support for sequence alignment and others

URL option with Sierappy isn't working.

Commands:
docker run -it --publish=8111:8080 hivdb/sierra dev
sierrapy --url http://localhost:8080/WebApplications/rest/graphql fasta sampleid.fasta

Result from Sierrapy:

0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/home/cloud/miniconda3/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/cloud/miniconda3/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/cloud/miniconda3/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/cloud/miniconda3/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/home/cloud/miniconda3/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5daddea070>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /WebApplications/rest/graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5daddea070>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cloud/miniconda3/bin/sierrapy", line 33, in <module>
    sys.exit(load_entry_point('sierrapy', 'console_scripts', 'sierrapy')())
  File "/mnt/f/src/sierrapy/python/sierrapy/cmds.py", line 5, in main
    cli(obj={})
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/mnt/f/src/sierrapy/python/sierrapy/commands/fasta.py", line 114, in fasta
    for idx, partial in enumerate(chunked(result, sharding)):
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/more_itertools/recipes.py", line 93, in take
    return list(islice(iterable, n))
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/tqdm/std.py", line 1178, in __iter__
    for obj in iterable:
  File "/mnt/f/src/sierrapy/python/sierrapy/sierraclient.py", line 182, in iter_sequence_analysis
    yield from self._sequence_analysis(partial, query)
  File "/mnt/f/src/sierrapy/python/sierrapy/sierraclient.py", line 93, in _sequence_analysis
    result: Dict[str, Any] = self.execute(
  File "/mnt/f/src/sierrapy/python/sierrapy/sierraclient.py", line 69, in execute
    result: Dict[str, Any] = self.client.execute(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/client.py", line 403, in execute
    return self.execute_sync(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/client.py", line 220, in execute_sync
    with self as session:
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/client.py", line 702, in __enter__
    return self.connect_sync()
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/client.py", line 686, in connect_sync
    self.session.fetch_schema()
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/client.py", line 881, in fetch_schema
    execution_result = self.transport.execute(parse(get_introspection_query()))
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/gql/transport/requests.py", line 220, in execute
    response = self.session.request(
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/home/cloud/miniconda3/lib/python3.8/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /WebApplications/rest/graphql (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f5daddea070>: Failed to establish a new connection: [Errno 111] Connection refused'))

CodonTranslator should throw an exception on non-triplet nucleotides.

See Utilities -> CodonTranslatorTest.testGenerateControlString("TTTT", "Asn")

Current output: " ."
Suggested output: throw exception

SequenceAnalysis in GraphQL should accept headers with trailing space characters

DataFetchingException was raised if sequence was submitted with a header with trailing spaces. For example, "K12345 ". This should be allowed.

URL

issue : I've started the mirror,but the local Sierra web service is not accessable through this URL:http://localhost:8080/WebApplications/rest/graphql

itcast@ubuntu:~$ sudo docker run -it --publish=8080:8080 hivdb/sierra dev
Using CATALINA_BASE: /usr/share/tomcat
Using CATALINA_HOME: /usr/share/tomcat
Using CATALINA_TMPDIR: /usr/share/tomcat/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
09-Dec-2020 01:08:53.902 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/9.0.35
09-Dec-2020 01:08:53.910 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: May 5 2020 20:36:20 UTC
09-Dec-2020 01:08:53.910 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.35.0
09-Dec-2020 01:08:53.910 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
09-Dec-2020 01:08:53.911 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.4.0-56-generic
09-Dec-2020 01:08:53.911 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
09-Dec-2020 01:08:53.911 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-14-oracle
09-Dec-2020 01:08:53.912 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 14.0.1+7
09-Dec-2020 01:08:53.917 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
09-Dec-2020 01:08:53.917 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /usr/share/tomcat9/apache-tomcat-9.0.35
09-Dec-2020 01:08:53.917 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/share/tomcat9/apache-tomcat-9.0.35
09-Dec-2020 01:08:53.962 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
09-Dec-2020 01:08:53.963 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
09-Dec-2020 01:08:53.963 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
09-Dec-2020 01:08:53.963 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties
09-Dec-2020 01:08:53.964 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
09-Dec-2020 01:08:53.964 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
09-Dec-2020 01:08:53.965 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
09-Dec-2020 01:08:53.966 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
09-Dec-2020 01:08:53.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms512M
09-Dec-2020 01:08:53.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx1024M
09-Dec-2020 01:08:53.967 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
09-Dec-2020 01:08:53.968 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/share/tomcat
09-Dec-2020 01:08:53.971 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/share/tomcat
09-Dec-2020 01:08:53.972 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/share/tomcat/temp
09-Dec-2020 01:08:53.972 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache Tomcat Native library which allows using OpenSSL was not found on the java.library.path: [/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib]
09-Dec-2020 01:08:54.597 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
09-Dec-2020 01:08:54.677 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [1,131] milliseconds
09-Dec-2020 01:08:54.762 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
09-Dec-2020 01:08:54.763 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.35]
09-Dec-2020 01:08:54.788 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/usr/share/tomcat9/apache-tomcat-9.0.35/webapps/WebApplications.war]
09-Dec-2020 01:08:57.731 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
09-Dec-2020 01:09:02.173 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/usr/share/tomcat9/apache-tomcat-9.0.35/webapps/WebApplications.war] has finished in [7,384] ms
09-Dec-2020 01:09:02.183 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
09-Dec-2020 01:09:02.223 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [7,545] milliseconds