Some users notice that the containers they are using not working as expected.
This is solved by manually restart all the containers.
Maybe some of the containers or R sessions is corrupted?

rLog, VST, and logCPM.

Transformed data is used for EDA and for DE using limma-trend. limma-voom and DESeq2 uses counts data.

While iDEP says the limma-voom and limma-trend method can handle 2 factors with 2 or more levels, it only outputs all possible pairwise comparisons between the combination of factors and not any interaction tests. It would be nice to be able to customize the contrasts made, even if just switching the order of the groups in a pairwise comparison so it is logical instead of alphabetical.

Allow user to download an executable script, which can give them same result as they saw in iDep web site.

No Design Yet

At the end of last tab shows R session info ( R versions, package versions). It is not working on the web server.

1. The issue:
   I received a lot error messages from downnotifier.com that iDEP was down for 2 minutes. Sometimes up to 10 times a day.
   Site back online for http://bioinformatics.sdstate.edu/idep/
   Hello,
   We found http://bioinformatics.sdstate.edu/idep/ to be up again.
   Your site was down for 2 minutes.

2. Replicate the issue.
   Start a new web Brower and visit iDEP. If iDEP loads within a few seconds, close the browser window. Not just the tab, close the entire browser.
   Repeat the process and sometimes you will see the site loads so slowly that it takes 1 or 2 minutes. When this happens, all users seem to be affected, namely subsequent requests to the server is also slow.

3. Possible causes:
   During the slow loading time, the nginx container does not seem to relay the connection to other webapp containers. Subsequent requests from other So it is likely a load balancing issue.
   Memory issue? Network latency? R package?

Using biocLite("BiocUpgrade") to upgrade from Bioconductor 3.4 to 3.6 causes problems.
Trying to remove all R packages and re-install.

Remove all installed packages:
https://www.r-bloggers.com/how-to-remove-all-user-installed-packages-in-r/

Install all packages using the librarySetup.R file in the background:
nohup R CMD BATCH librarySetup.R output &

Shiny plot is base64 by default. Which cannot be copy as an image.
Try change it to some image type, so user can easily copy it by right click the image.

https://groups.google.com/forum/#!topic/shiny-discuss/hdpBcAcPicU

Shiny-server can deny request of iframe by setting

frame_options deny;

So user don't need click 'reset' button manually

If one process use who cpu resource other process will be waiting until the resource is available.
Docker-compose version 3 seems to provide set a limitation of resource. iDep is needed to re-define docker-compoase.yml with version3.

System should able to directly download read count from ARCHS4 database using GSE id or GSM id.

In analyzing T47D cell line data, up- and down- regulation is the opposite? Limma-voom and limma trend.

Also, check the fold change values shown in DEG2 page?

Some R packages like sva can detect and remove batch effects from RNA-seq data.

https://github.com/iscb-dc-rsg/2016-summer-workshop/tree/master/3B-Hughitt-RNASeq-Coex-Network-Analysis/tutorial
https://bioconductor.org/packages/release/bioc/vignettes/sva/inst/doc/sva.pdf

Running these commands, nginx fails to run with the following output. Is there a problem with the nginx configuration file? I have not modified any of the files, only ran the following. Thanks!

git clone https://github.com/iDEP-SDSU/idep.git
The following command starts containers from docker-compose.yml. The R server listening for HTTP connections on port 3838
docker-compose up -d

Gives this output:

C:\Users\mikel\idep\idep (master -> origin) λ docker-compose.exe up Starting idep_webapp_1 ... done Starting idep_nginx_1 ... done Attaching to idep_webapp_1, idep_nginx_1 webapp_1 | [2018-02-14 20:43:52.043] [INFO] shiny-server - Shiny Server v1.5.7.877 (Node.js v6.10.3) webapp_1 | [2018-02-14 20:43:52.048] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf" nginx_1 | 2018/02/14 20:43:52 [emerg] 1#1: host not found in upstream "idep_webapp_2:3838" in /etc/nginx/nginx.conf:19 webapp_1 | [2018-02-14 20:43:52.104] [WARN] shiny-server - Running as root unnecessarily is a security risk! You could be running more securely as non-root. nginx_1 | nginx: [emerg] host not found in upstream "idep_webapp_2:3838" in /etc/nginx/nginx.conf:19 webapp_1 | [2018-02-14 20:43:52.121] [INFO] shiny-server - Starting listener on 0.0.0.0:3838 idep_nginx_1 exited with code 1

Sometimes user upload a read count or FPKM file, but two column are exactly the same for all genes due to error in the merging process. Identify these.

To shut down the web server:

1.        Log in to resapp003 as super user or use sudo
   

2.        cd   /dockers/idep
   

3.        docker-compose down
   

To start the web server after upgrade:

1.        Log in to resapp003 as super user or use sudo
   

2.        cd   /dockers/idep
   

3.        docker-compose up -d --scale webapp=15
   

The cause of the problem this morning:
I wanted to reduce the docker instances to 8, as we are having 8 cores. By simply using
docker-compose up -d --scale webapp=8
The Ngix docker instance is started but soon killed. It is the load balancer that handle web traffic.
The reason is that the Ngix docker definition file specifies 15 instances. Lines idep_webapp_9 to idep_webapp_15 needs in the source code below need to be deleted and the Ngix image needs to be rebuilt. https://github.com/iDEP-SDSU/idep/blob/master/nginx/nginx.conf

We want iDep could generate a report when user finish analysis.
This report should contains same results as what user seen in the web site.

1. use R mark down design the layout of the report.
2. R mark down file should not contains any algorithm related code.

It takes 2 minutes for DEG1 to run with DESeq2. The experiment has 2 factors, one is treated as batch factor

When the server is busy running, if the user clicks many tags server will disconnect. Maybe it can be resolved in Shiny Server Pro?

A histogram helps people better in reshaping the data

The heatmaps are red-green. Users should be able to change it to blue-yellow or other colors.

System should able to pass #44 result to #43

Problem:
Comment: missing for entire sample group, use median for all samples
Actual: Used the median of the result after first replacement

Example:

Reason:
When calculate 'sample median', we already replaced NA values use group median.
This cause the 'sample median' no-longer be the original sample median.

Please confirm which way is the one we want implement.

Hi,
when I try to download the customary R code from the iDEP server I get the following error -

ERROR: An error has occurred. Check your logs or contact the app author for clarification.

The link for the download is this:
http://bioinformatics.sdstate.edu/idep/session/6bd2ddca1847866ef426d7810fb29f75/download/downloadRcodeMarkdown?w=

is it correct?

thanks
Assa

System should able to search GSM id or GSE id based on given description.

Besides Google analytics, is there any other way to track usage (CPU, memory etc.)?

If one R session runs at 100% CPU for a while (for example the RUN PREDA at Chromosome tab), other users cannot connect to http://137.216.132.65/idep/ This is a big problem if we have a lot of computing-intensive processes. Limitation of the free version of Shiny Server?

All text shows in UI should able to change into another language easily

To improve reproducibility in research, it is desirable for users to know what happened to their data.
Shiny Server can also host R Markdown. Is it possible to let users interact with R markdown document?
Maybe not a legitimate question. But I want users to leave the site with a documentation of R codes and results, even their notes!

The http://ge-lab.org/idep/ needs 7 seconds to load.
Try to move the library( ) scripts from the very beginning of the server.R to places where these packages are needed.

Showing sample name on PCA plot

wget https://bioconductor.org/packages/3.6/bioc/src/contrib/QUBIC_1.6.0.tar.gz
tar xvf QUBIC_1.6.0.tar.gz
Add the following line to cluster.cc under src folder:
#include <cstdio>

Save the file, then
tar -zcvf QUBIC.tar.gz QUBIC

From R, run:
install.packages("QUBIC.tar.gz")

Clicking on the “?” in the top left box in any tab should automatically open in a new tab or pop-up box. Right now, it takes you in the same tab to the documentation page, but when you try to go back to your analysis, everything is lost and you have to start over! [at least, in Google Chrome. Have you checked for compatibility with all major browsers?]

STRINGdb is used but not in 'librarySetup.R'

read.csv() is a wrapper around the more general read.table() function. That latter function has argument check.names which is documented as:
check.names: logical. If ‘TRUE’ then the names of the variables in the
data frame are checked to ensure that they are syntactically
valid variable names. If necessary they are adjusted (by
‘make.names’) so that they are, and also to ensure that there
are no duplicates.

https://stackoverflow.com/questions/10441437/why-am-i-getting-x-in-my-column-names-when-reading-a-data-frame

In addition to the parameters listed in the R tab, it would be extremely useful if the base R codes used could be output. This could help researchers learn R and allowed experienced programmers to quickly recreate/modify the plots.

If you see this error during installing R library, it means your vm or local machine doesn't have a sufficient memory space to compile code. In this case, you need to add more memory space.

Put databases and flat files on servers and enable users to run iDEP on their own computer from RStudio using runGitHub such as:

runGitHub("shiny_example", "rstudio")

Some issues with the site can be solved by restart the shiny server or even the docker container.
To restart Shiny server, make changes to the restart.txt file and upload using git pull.
To restart the container:
cd /dockers/idep/
docker-compose restart
docker-compose ps

Other docker commands:

acess docker terminal

docker exec -it idep_web_1 bash

start R and install packages

R

install.packages("WGCNA")

For FPKM:

and read count:

we both have 'incProgress'.

But this is not exist in 'other data type' case:

Please confirm if we need it here.