wyang17 / squire Goto Github PK
View Code? Open in Web Editor NEWSoftware for Quantifying Interspersed Repeat Expression
License: Other
Software for Quantifying Interspersed Repeat Expression
License: Other
Hi,
I am looking into TE expression upon drug treatment using stranded RNA-seq of two samples.
Here are the commands I used
squire Count -c <clean_dir> -f <fetch_dir> -r 75 -n <sample fastq> -p 4 -b hg38 -s 2
squire Call -1 -2 -A treated -B control -i squire_count -o -p 4 -f pdf
The number of TE copies in "DESeq2_TE_only.txt" is not the same across samples while the number of genes in "DESeq2_RefSeq_only.txt" is the same. (To avoid the case where one TE copy/gene generates two transcripts in the opposite strand, I trimmed the transcript strand information in row names and count the number of TE copy and gene.)
I expect to see all TE copies in "DESeq2_TE_only.txt" but only a subset of TE copies are shown for each sample and the number of TE copies is discordant across samples. Is there a way for me to have SQuIRE output with all TE copies?
Hi, I got the following errors when running squire Map and Call.
It looks like similar problems to #19 , but I am not sure how to fix it. Please would you let me know how to solve these problems?
I got the above error, but I confirmed I could get bam file by squire map. So I tried squire call and got the following error.
I have been working with SQuIRE a few months performing different analysis with the samples of my laboratory.
Now, I am going to start analyzing samples of different laboratories for increasing the number of samples and my statistical power.
I have been reading about it and the developers of DESEQ2 (the part of SQuIRE that performs the differential expression analysis) recommend to introduce the batch variable so the algorithm can take into account the differences between samples produced by its origin.
How can I introduce this variable if I am using SQuIRE? Is necessary to perform this after the mathematical correction performed by this sofware?
Hi I wonder if you could add explanations about the format of the files generated by SQuIRE Count. This would be helpful for people who want to perform customized analysis besides differential expression on their own.
I think a good example of format explanations is from StringTie: http://ccb.jhu.edu/software/stringtie/index.shtml?t=manual#output
Specifically for me, I am wondering what each column in *_refGenecounts.txt means. For example,
chr17 26138685 26195811 Axin1 11.196273 + 264 NM_001159598,NM_009733
I think 11.196273 is the TPM, and 264 is the read count? And how did you deal with multiple mapped reads that align to genes?
Thanks.
Hello,
I am trying to apply SQuiRE on my data but I have access on a Slurm scheduler and not on an SGE.
So, I have changed the scheduler commands but I am getting the following error:
sbatch: error: Invalid numeric value "wd" for cpus-per-task.
Which apparently is the SGE command for cpus.
Here are my sbatch commands:
##!/bin/bash --login #$ -cwd #SBATCH --job-name=fetch #SBATCH -p compute #SBATCH -n 1 --core-spec=40 #SBATCH --tasks-per-node=40
BTW, I am trying to run Fetch.
Thank you very much in advance,
Vasilis.
Hello,
I was wondering if anyone else has the same problem with me, regarding the volcano and the PCA graphs. The pdf files that are generated are empty (4KB). This happens only in the volcano and PCA. All the other graphs are fine!
Thank you very much in advance,
Vasilis.
I want to find out the differential expressed retrotransposons among two groups of samples. So I firstly ran SQuIRE pipeline on each sample. However, I found the rows in the squire_count/*_TEcounts.txt are not alignable, and neither column (tx_chr, tx_start, tx_stop) nor TE_ID is unique. Just according to my naive understanding, I'm going to do the downstream analysis as below:
On each *_TEcounts.txt file, group by the "TE_ID" column and sum up the "tot_counts" (or shall I use "tot_reads" instead? what's the difference?) in every group. So I can get a table uniq_TEcounts.txt with unique TE_ID and the tot_counts_sum for each TE_ID;
Merge uniq_TEcounts.txt of all samples together, aligned based on the unique TE_ID. If TE_ID are missed in any files, fill with 0;
Using DESeq to compare the tot_counts_sum among two condition groups, assigning the sizeFactors with the total refGenecounts (sum up the 7th column of *_refGenecounts.txt).
Do you think is the analysis correct?
I am running squire Map step for paired end data. However, when i list the files for each read (read 1 and 2), it doesn't work properly. It seems not to be picking up the files, and only picks one (the last one on the list). What could be wrong?
I separated the fastq files with comma (,) as directed.
Hi,
I've tried to run 'squire fetch' in a slurm cluster but it never finished (run over 14 days and got killed). If I understand the code correctly, the 'squire fecth' command would download all the necessary fasta and bed files, but there was no output in the output directory in my run.
Below is the command I used and the log file.
Script:
_#!/bin/bash
#SBATCH -n 8
#SBATCH --time=14-24:00:00
#SBATCH -J fetch
#SBATCH -o fetch.%A.out
#SBATCH -e fetch.%A.err
#SBATCH --mem=320g
source activate squire
squire Fetch -c -b mm10 -f -r -g -k -p 8 -o ./ -v >squire_Fetch_mm10.log 2>squire_Fetch_mm10.err.log_
Log:
_start time is:2020-11-25 21:13:15.239034
Fetch.pyc
index=False
fetch_folder=./
rmsk=True
verbosity=True
pthreads=8
keep=True
build=mm10
func=<function main at 0x7f0c84f4a578>
fasta=True
gene=True
chrom_info=True
Downloading Compressed Chromosome files..._
Any help would be appreciated.
Thanks!
Weida
Hi,
I am trying to run the squire Fetch
command but when it reaches the STAR index build it fails right away returning segmentation fault. It's not due to insufficient memory because it fails the moment it reaches the command. I am trying to build hg38 genome.
I installed squire in conda following the instructions.
Any idea why this is happening?
Thanks
Finished running expectation-maximization calculation after iteration:38 2018-06-22 16:50:47.094679
Writing counts 2018-06-22 16:50:47.094714
Traceback (most recent call last):
File "/users/cpacyna/miniconda3/envs/squire_git/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/users/cpacyna/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/users/cpacyna/SQuIRE/squire/Count.py", line 2171, in main
RepClass.writeRep(aligned_libsize,counts_temp,basename,strandedness,iteration)
File "/users/cpacyna/SQuIRE/squire/Count.py", line 1205, in writeRep
self.fpkm += outline.fpkm
UnboundLocalError: local variable 'outline' referenced before assignment
Hi ,
May I how many samples are simulated, if more than one samples are simulated, is it ok to use the avearge value to calc the true positive ratio and other ratios?
By the way, may I know if it's convenient to share the scripts of the data simulation?
Thanks a lot for your kind guidance!
Best,
Hi,
What is the default method SQuIRE uses when multiple inputs are passed to "squire map"? I recently tried passing in three FASTQs and assumed the program would produce three BAM files corresponding to each of the inputs, but instead only one output BAM was produced (with the name I specified using the --name flag).
Thanks so much!
Hi,
I am beginning to work with Squire to analyze L1 mobile elements in knockout cell lines. I am specifically interested in obtaining antisense expression on L1 (that should include transcripts from L1 antisense promoter and "run on" transcripts from nearby gene promoters).
I have some trouble in getting that information from strand output (tx_strand and TE_strand). According to SQuIRE Count output explanation, TE_strand is orientation of TE insertion and tx_strand the strand of TE transcription. My guess is that I can get antisense L1 transcripts by choosing counts where TE_strand is different from tx_strand (TE_strand != tx_strand). Am I right?
Thank you for your support and for this useful software,
Guille.
As SQuIRE has been published, the repository is not seeing much change right now, and the latest release is several commits behind master
, it would be nice to have a new release.
Hello,
I'm doing an RNA-seq analysis on 80 files taken from CARNES 2015 paper. At the mapping phase, 4 files fail, with the following error message:
fetch_folder=squire_fetch
name=False
extra=None
read_length=115
verbosity=True
pthreads=4
read1=/beegfs/data/eugloh/CARNES/bug_files/B1_W1_F2_TCGAAG_L002_fp.fastq
read2=None
build=dm6
func=<function main at 0x7f1c979f7a28>
trim3=0
map_folder=squire_map_115
Aligning FastQ files 2019-04-24 16:04:36.165775
Apr 24 16:04:36 ..... started STAR run
Apr 24 16:04:36 ..... loading genome
Apr 24 16:06:16 ..... processing annotations GTF
Apr 24 16:06:23 ..... inserting junctions into the genome indices
Apr 24 16:07:26 ..... started 1st pass mapping
Apr 24 16:19:59 ..... finished 1st pass mapping
Apr 24 16:19:59 ..... inserting junctions into the genome indices
Apr 24 16:20:32 ..... started mapping
Apr 24 16:35:10 ..... finished successfully
[bam_sort_core] merging from 32 files...
Aborted
Traceback (most recent call last):
File "/beegfs/home/eugloh/miniconda3/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/beegfs/data/eugloh/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/beegfs/data/eugloh/SQuIRE/squire/Map.py", line 378, in main
align_unpaired(read1,pthreads,trim3,index,outfile,gtf,gzip,prefix, read_length,extra_fapath)
File "/beegfs/data/eugloh/SQuIRE/squire/Map.py", line 180, in align_unpaired
sp.check_call(["/bin/sh", "-c", sortcommand])
File "/beegfs/home/eugloh/miniconda3/envs/squire/lib/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/sh', '-c', 'samtools sort -@ 4 squire_map_115/B1_W1_F2_TCGAAG_L002_fpAligned.out.bam squire_map_115/B1_W1_F2_TCGAAG_L002_fp']' returned non-zero exit status 134
I work on single-end data and reads of length 115. I don't believe the error is due to a RAM issue because I work on a powerful enough processing cluster.
Those 4 files are not particularly larger than the other 76, and I can't seem to find the root of the problem.
Have you ever encountered this issue, and if so, what was the cause, and how to fix it? I'm stuck here and would really appreciate your help.
Thanks in advance.
Eugenie
Hello,
I am using SQuIRE for the TE expression analysis in Drosophila (dm6).
I find that there are at least three duplications: CG40439, CG46302, and CG17715.
Coincidentally (?), they were duplicated in chr2L and chr2R.
Here are the links for their documented info on FlyBase:
CG40439 http://flybase.org/reports/FBgn0058439
CG46302 http://flybase.org/reports/FBgn0284080
CG17715 http://flybase.org/reports/FBgn0041004
Any idea?
Hi,
I have been trying to run Fetch on my samples but I guess I must be doing something wrong in the argument.sh file.
Could you provide an example on how to fill out the argument file especially the sample names ? I keep getting "command not found" at the line 14 of the argument.sh file. Sorry, for that I am pretty new in the programming field but I find your tool pretty interesting for what I am doing.
best,
Salim.
Hi wyang17,
I am having an issue with running the fetch command as shown below.
Traceback (most recent call last): File "/home/svu/phaei/.conda/miniconda/4.9/envs/squire/bin/squire", line 11, in <module> load_entry_point('SQuIRE', 'console_scripts', 'squire')() File "/hpctmp/phaei/FACT/SQuIRE/squire/cli.py", line 156, in main subargs.func(args = subargs) File "/hpctmp/phaei/FACT/SQuIRE/squire/Fetch.py", line 231, in main raise Exception("Was not able to download chromosome file from UCSC" + "\n", file = sys.stderr) TypeError: exceptions.Exception does not take keyword arguments
I am not sure how to resolve this issue.
regards
Hi,
thanks for developing this cool tool!
I'm trying to use mm10 (obtained using fetch ) to quantify some RNA-seq libraries, then I noticed a weird warning at the Count procedure :
Warning: gene "Gm20747" (on chrY) has reference transcripts on both strands?
I figured out this was caused by the gene ID collision in the mm10_refGene.gtf
chrY squire_fetch/mm10_refGene.genepred transcript 21164554 21166898 . + . gene_id "Gm20747"; transcript_id "NM_001025241"; gene_name "Gm20747";
chrY squire_fetch/mm10_refGene.genepred transcript 53415957 53418307 . - . gene_id "Gm20747"; transcript_id "NM_001025241_2"; gene_name "Gm20747";
chrY squire_fetch/mm10_refGene.genepred transcript 73313695 73316051 . - . gene_id "Gm20747"; transcript_id "NM_001025241_3"; gene_name "Gm20747";
chrY squire_fetch/mm10_refGene.genepred transcript 81799150 81801497 . - . gene_id "Gm20747"; transcript_id "NM_001025241_4"; gene_name "Gm20747";
since there are quite a few such cases, wondering how does this affect the stranded libraries?
Possible to get a quick answer here before I go and check the code?
Another question is: do you have any descriptions of the input data preparation for each step somewhere ( such as the Clean procedure. too lazy to read the code, sorry :) )? Then I can quickly get some scripts to fix an Ensembl data converter.
Thanks!
Hello,
I am processing some fastq files coming out of trimmomatics trimming. They have different read lengths. What setting should I use for the squire Map and Count? The maximum or the minimum? Or shall I trim the reads to the minimum length and run it with a certain read length setting?
Mx
Dear all,
please would you let me know : would it be possible to have a docker container for SQuIRE ?
thanks !
bogdan
hello there,
After running successfully the tool until Squire Count, yay! I was going over the Count output tables and I noticed that the same individual (Same TE family, same Chrom:Start-END) was divided into several different TE_IDs based on the milliDiv parameter.
I went to the supplementary data from the paper and it states:
milliDiv = Base mismatches in parts per thousand (from RepeatMasker)
However, I do not understand why that parameter was chosen to be part of the ID and why it is used to separate individuals as "different". If anyone could explain more in detail that parameter and why use it as part of the unique identifier of a TE, that would be great.
Thank you!
Adrián
Hi,
I have been getting the following messages over and over again when I run squire count. It seems that it keeps counting but does not end...Any idea on what might be going on ?
Many thanks,
Salim.
Creating unique and multiple alignment bedfiles 2019-06-27 16:59:38.076091
Intersecting bam file with TE bedfile 2019-06-27 16:59:38.077534
Combining adjacent TEs with same read alignment 2019-06-27 17:19:13.274866
Getting genomic coordinates of read2019-06-27 18:16:52.544021
Identifying and labeling unique and multi reads2019-06-27 18:36:43.369293
counting unique alignments 2019-06-27 19:58:01.933640
counting multi alignments 2019-06-27 20:06:09.891890
Adding Tag information to aligned TEs 2019-06-27 20:21:03.918243
Calculating multialignment assignments 2019-06-27 20:21:05.750406
Running expectation-maximization calculation for iteration:1 2019-06-27 20:23:02.584813
Average change in TE count:0.0 2019-06-27 20:23:05.730731
Max change in TE count:0 2019-06-27 20:23:05.732306
Number changed TE:0 2019-06-27 20:23:05.733453
Number TEs changed by at least 1 count:0 2019-06-27 20:23:05.734695
Number TEs changed by at least 1 count with at least 10 counts:0 2019-06-27 20:23:05.735947
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:0 2019-06-27 20:23:05.737184
Expectation maximization changed the average likelihood by: 0.0511343014077 2019-06-27 20:24:45.524809
Number of reads with average TE likelihoods changed by at least 0.1: 1897485 2019-06-27 20:24:45.526482
Running expectation-maximization calculation for iteration:2 2019-06-27 20:24:45.527646
Average change in TE count:1.07411444581 2019-06-27 20:24:48.826892
Max change in TE count:95024.75679 2019-06-27 20:24:48.828276
Number changed TE:351560 2019-06-27 20:24:48.829507
Number TEs changed by at least 1 count:12190 2019-06-27 20:24:48.830639
Number TEs changed by at least 1 count with at least 10 counts:3828 2019-06-27 20:24:48.831712
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3773 2019-06-27 20:24:48.832848
Expectation maximization changed the average likelihood by: 0.0208707660624 2019-06-27 20:26:28.475871
Number of reads with average TE likelihoods changed by at least 0.1: 777028 2019-06-27 20:26:28.477260
Running expectation-maximization calculation for iteration:3 2019-06-27 20:26:28.478517
Average change in TE count:0.435416359278 2019-06-27 20:26:31.153396
Max change in TE count:72363.5924624 2019-06-27 20:26:31.154932
Number changed TE:47833 2019-06-27 20:26:31.156224
Number TEs changed by at least 1 count:1601 2019-06-27 20:26:31.157534
Number TEs changed by at least 1 count with at least 10 counts:953 2019-06-27 20:26:31.158797
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:892 2019-06-27 20:26:31.160008
Expectation maximization changed the average likelihood by: 0.00814132308952 2019-06-27 20:28:10.913557
Number of reads with average TE likelihoods changed by at least 0.1: 51618 2019-06-27 20:28:10.915097
Running expectation-maximization calculation for iteration:4 2019-06-27 20:28:10.916369
Average change in TE count:0.164178617785 2019-06-27 20:28:13.599662
Max change in TE count:30982.8882079 2019-06-27 20:28:13.601114
Number changed TE:45770 2019-06-27 20:28:13.602582
Number TEs changed by at least 1 count:874 2019-06-27 20:28:13.603975
Number TEs changed by at least 1 count with at least 10 counts:632 2019-06-27 20:28:13.605180
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:566 2019-06-27 20:28:13.606357
Expectation maximization changed the average likelihood by: 0.00429504488505 2019-06-27 20:29:53.296998
Number of reads with average TE likelihoods changed by at least 0.1: 50188 2019-06-27 20:29:53.298564
Running expectation-maximization calculation for iteration:5 2019-06-27 20:29:53.300400
Average change in TE count:0.0785211764923 2019-06-27 20:29:55.968532
Max change in TE count:11067.9082562 2019-06-27 20:29:55.970595
Number changed TE:44859 2019-06-27 20:29:55.972181
Number TEs changed by at least 1 count:599 2019-06-27 20:29:55.974156
Number TEs changed by at least 1 count with at least 10 counts:481 2019-06-27 20:29:55.976046
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:414 2019-06-27 20:29:55.977677
Expectation maximization changed the average likelihood by: 0.00311617217645 2019-06-27 20:31:35.842340
Number of reads with average TE likelihoods changed by at least 0.1: 45879 2019-06-27 20:31:35.843969
Running expectation-maximization calculation for iteration:6 2019-06-27 20:31:35.845344
Average change in TE count:0.0492869501621 2019-06-27 20:31:38.541818
Max change in TE count:6061.79276584 2019-06-27 20:31:38.543195
Number changed TE:44325 2019-06-27 20:31:38.544597
Number TEs changed by at least 1 count:441 2019-06-27 20:31:38.545872
Number TEs changed by at least 1 count with at least 10 counts:383 2019-06-27 20:31:38.547271
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:306 2019-06-27 20:31:38.548735
Expectation maximization changed the average likelihood by: 0.00275636485596 2019-06-27 20:33:17.999590
Number of reads with average TE likelihoods changed by at least 0.1: 45898 2019-06-27 20:33:18.001036
Running expectation-maximization calculation for iteration:7 2019-06-27 20:33:18.002219
Average change in TE count:0.038318265443 2019-06-27 20:33:20.678041
Max change in TE count:4859.82343395 2019-06-27 20:33:20.679473
Number changed TE:43975 2019-06-27 20:33:20.680943
Number TEs changed by at least 1 count:345 2019-06-27 20:33:20.682083
Number TEs changed by at least 1 count with at least 10 counts:303 2019-06-27 20:33:20.683332
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:233 2019-06-27 20:33:20.684473
Expectation maximization changed the average likelihood by: 0.00271953945072 2019-06-27 20:35:00.202770
Number of reads with average TE likelihoods changed by at least 0.1: 45971 2019-06-27 20:35:00.204421
Running expectation-maximization calculation for iteration:8 2019-06-27 20:35:00.205657
Average change in TE count:0.0359083173197 2019-06-27 20:35:02.892197
Max change in TE count:6683.84204686 2019-06-27 20:35:02.893611
Number changed TE:43736 2019-06-27 20:35:02.894816
Number TEs changed by at least 1 count:276 2019-06-27 20:35:02.896042
Number TEs changed by at least 1 count with at least 10 counts:251 2019-06-27 20:35:02.897082
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:181 2019-06-27 20:35:02.898140
Expectation maximization changed the average likelihood by: 0.0028531869672 2019-06-27 20:36:42.500407
Number of reads with average TE likelihoods changed by at least 0.1: 45993 2019-06-27 20:36:42.502106
Running expectation-maximization calculation for iteration:9 2019-06-27 20:36:42.503385
Average change in TE count:0.0382347414545 2019-06-27 20:36:45.166871
Max change in TE count:9054.59090785 2019-06-27 20:36:45.168059
Number changed TE:43516 2019-06-27 20:36:45.169250
Number TEs changed by at least 1 count:214 2019-06-27 20:36:45.170452
Number TEs changed by at least 1 count with at least 10 counts:195 2019-06-27 20:36:45.171551
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:141 2019-06-27 20:36:45.172629
Expectation maximization changed the average likelihood by: 0.00308161005019 2019-06-27 20:38:24.852120
Number of reads with average TE likelihoods changed by at least 0.1: 46036 2019-06-27 20:38:24.853556
Running expectation-maximization calculation for iteration:10 2019-06-27 20:38:24.854923
Average change in TE count:0.0433444850587 2019-06-27 20:38:27.540708
Max change in TE count:11700.2977783 2019-06-27 20:38:27.541917
Number changed TE:43326 2019-06-27 20:38:27.543153
Number TEs changed by at least 1 count:167 2019-06-27 20:38:27.544349
Number TEs changed by at least 1 count with at least 10 counts:153 2019-06-27 20:38:27.545431
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:104 2019-06-27 20:38:27.546703
Expectation maximization changed the average likelihood by: 0.00329804410289 2019-06-27 20:40:06.692700
Number of reads with average TE likelihoods changed by at least 0.1: 46075 2019-06-27 20:40:06.694309
Running expectation-maximization calculation for iteration:11 2019-06-27 20:40:06.695695
Average change in TE count:0.0499389090483 2019-06-27 20:40:09.365934
Max change in TE count:13930.1494273 2019-06-27 20:40:09.367182
Number changed TE:43146 2019-06-27 20:40:09.368407
Number TEs changed by at least 1 count:137 2019-06-27 20:40:09.369604
Number TEs changed by at least 1 count with at least 10 counts:124 2019-06-27 20:40:09.370739
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:83 2019-06-27 20:40:09.371919
Expectation maximization changed the average likelihood by: 0.00340465606123 2019-06-27 20:41:48.626337
Number of reads with average TE likelihoods changed by at least 0.1: 46110 2019-06-27 20:41:48.627810
Running expectation-maximization calculation for iteration:12 2019-06-27 20:41:48.629091
Average change in TE count:0.0537709260366 2019-06-27 20:41:51.292370
Max change in TE count:14858.1805188 2019-06-27 20:41:51.293685
Number changed TE:42968 2019-06-27 20:41:51.295011
Number TEs changed by at least 1 count:123 2019-06-27 20:41:51.296200
Number TEs changed by at least 1 count with at least 10 counts:115 2019-06-27 20:41:51.297567
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:73 2019-06-27 20:41:51.298779
Expectation maximization changed the average likelihood by: 0.00333866856605 2019-06-27 20:43:30.729615
Number of reads with average TE likelihoods changed by at least 0.1: 46139 2019-06-27 20:43:30.730967
Running expectation-maximization calculation for iteration:13 2019-06-27 20:43:30.732207
Average change in TE count:0.053363197281 2019-06-27 20:43:33.378818
Max change in TE count:14028.8332855 2019-06-27 20:43:33.380110
Number changed TE:42741 2019-06-27 20:43:33.381390
Number TEs changed by at least 1 count:111 2019-06-27 20:43:33.382869
Number TEs changed by at least 1 count with at least 10 counts:105 2019-06-27 20:43:33.384073
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:64 2019-06-27 20:43:33.385400
Expectation maximization changed the average likelihood by: 0.00312510886026 2019-06-27 20:45:12.833152
Number of reads with average TE likelihoods changed by at least 0.1: 46158 2019-06-27 20:45:12.834808
Running expectation-maximization calculation for iteration:14 2019-06-27 20:45:12.836113
Average change in TE count:0.0493298247059 2019-06-27 20:45:15.512704
Max change in TE count:11826.4783365 2019-06-27 20:45:15.514117
Number changed TE:42497 2019-06-27 20:45:15.515527
Number TEs changed by at least 1 count:98 2019-06-27 20:45:15.516983
Number TEs changed by at least 1 count with at least 10 counts:95 2019-06-27 20:45:15.518354
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:59 2019-06-27 20:45:15.519520
Expectation maximization changed the average likelihood by: 0.00284940382948 2019-06-27 20:46:54.466403
Number of reads with average TE likelihoods changed by at least 0.1: 46194 2019-06-27 20:46:54.468013
Running expectation-maximization calculation for iteration:15 2019-06-27 20:46:54.469371
Average change in TE count:0.0435483704919 2019-06-27 20:46:57.118977
Max change in TE count:9132.90551704 2019-06-27 20:46:57.120383
Number changed TE:42270 2019-06-27 20:46:57.121696
Number TEs changed by at least 1 count:94 2019-06-27 20:46:57.122968
Number TEs changed by at least 1 count with at least 10 counts:89 2019-06-27 20:46:57.124119
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:49 2019-06-27 20:46:57.125346
Expectation maximization changed the average likelihood by: 0.00258426128013 2019-06-27 20:48:36.133542
Number of reads with average TE likelihoods changed by at least 0.1: 46199 2019-06-27 20:48:36.134934
Running expectation-maximization calculation for iteration:16 2019-06-27 20:48:36.136223
Average change in TE count:0.0375320520522 2019-06-27 20:48:38.782909
Max change in TE count:6684.88294109 2019-06-27 20:48:38.784338
Number changed TE:41976 2019-06-27 20:48:38.785682
Number TEs changed by at least 1 count:88 2019-06-27 20:48:38.787236
Number TEs changed by at least 1 count with at least 10 counts:82 2019-06-27 20:48:38.788352
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:43 2019-06-27 20:48:38.789489
Expectation maximization changed the average likelihood by: 0.00236110482072 2019-06-27 20:50:17.968019
Number of reads with average TE likelihoods changed by at least 0.1: 46217 2019-06-27 20:50:17.969448
Running expectation-maximization calculation for iteration:17 2019-06-27 20:50:17.970551
Average change in TE count:0.0321293912345 2019-06-27 20:50:20.614179
Max change in TE count:5726.53391534 2019-06-27 20:50:20.615441
Number changed TE:41687 2019-06-27 20:50:20.616596
Number TEs changed by at least 1 count:83 2019-06-27 20:50:20.617772
Number TEs changed by at least 1 count with at least 10 counts:79 2019-06-27 20:50:20.619035
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:41 2019-06-27 20:50:20.620212
Expectation maximization changed the average likelihood by: 0.00218294079568 2019-06-27 20:51:59.762587
Number of reads with average TE likelihoods changed by at least 0.1: 46220 2019-06-27 20:51:59.764062
Running expectation-maximization calculation for iteration:18 2019-06-27 20:51:59.765339
Average change in TE count:0.0273788685319 2019-06-27 20:52:02.424787
Max change in TE count:5246.40423245 2019-06-27 20:52:02.426066
Number changed TE:41379 2019-06-27 20:52:02.427363
Number TEs changed by at least 1 count:74 2019-06-27 20:52:02.428502
Number TEs changed by at least 1 count with at least 10 counts:71 2019-06-27 20:52:02.429586
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:35 2019-06-27 20:52:02.430861
Expectation maximization changed the average likelihood by: 0.00204207345146 2019-06-27 20:53:40.322640
Number of reads with average TE likelihoods changed by at least 0.1: 46230 2019-06-27 20:53:40.324206
Running expectation-maximization calculation for iteration:19 2019-06-27 20:53:40.325536
Average change in TE count:0.0232583614708 2019-06-27 20:53:42.927558
Max change in TE count:4612.47756964 2019-06-27 20:53:42.928926
Number changed TE:41006 2019-06-27 20:53:42.930052
Number TEs changed by at least 1 count:66 2019-06-27 20:53:42.931286
Number TEs changed by at least 1 count with at least 10 counts:65 2019-06-27 20:53:42.932621
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:30 2019-06-27 20:53:42.933836
Expectation maximization changed the average likelihood by: 0.00192967916094 2019-06-27 20:55:20.763344
Number of reads with average TE likelihoods changed by at least 0.1: 46241 2019-06-27 20:55:20.764685
Running expectation-maximization calculation for iteration:20 2019-06-27 20:55:20.766005
Average change in TE count:0.0196431497373 2019-06-27 20:55:23.369147
Max change in TE count:3918.57418016 2019-06-27 20:55:23.370629
Number changed TE:40630 2019-06-27 20:55:23.371972
Number TEs changed by at least 1 count:55 2019-06-27 20:55:23.373450
Number TEs changed by at least 1 count with at least 10 counts:55 2019-06-27 20:55:23.374566
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:26 2019-06-27 20:55:23.375936
Expectation maximization changed the average likelihood by: 0.00183923556257 2019-06-27 20:57:01.474360
Number of reads with average TE likelihoods changed by at least 0.1: 46241 2019-06-27 20:57:01.475955
Running expectation-maximization calculation for iteration:21 2019-06-27 20:57:01.477240
Average change in TE count:0.0165341638683 2019-06-27 20:57:04.076442
Max change in TE count:3238.33391922 2019-06-27 20:57:04.077851
Number changed TE:40264 2019-06-27 20:57:04.079071
Number TEs changed by at least 1 count:55 2019-06-27 20:57:04.080064
Number TEs changed by at least 1 count with at least 10 counts:55 2019-06-27 20:57:04.081209
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:26 2019-06-27 20:57:04.082221
Expectation maximization changed the average likelihood by: 0.00176591813656 2019-06-27 20:58:41.822545
Number of reads with average TE likelihoods changed by at least 0.1: 46250 2019-06-27 20:58:41.824085
Running expectation-maximization calculation for iteration:22 2019-06-27 20:58:41.825459
Average change in TE count:0.0138651489821 2019-06-27 20:58:44.426406
Max change in TE count:2618.59973131 2019-06-27 20:58:44.427860
Number changed TE:39815 2019-06-27 20:58:44.429300
Number TEs changed by at least 1 count:55 2019-06-27 20:58:44.430579
Number TEs changed by at least 1 count with at least 10 counts:55 2019-06-27 20:58:44.431925
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:25 2019-06-27 20:58:44.433193
Expectation maximization changed the average likelihood by: 0.00170605754024 2019-06-27 21:00:21.903940
Number of reads with average TE likelihoods changed by at least 0.1: 46252 2019-06-27 21:00:21.905416
Running expectation-maximization calculation for iteration:23 2019-06-27 21:00:21.906577
Average change in TE count:0.01159880618 2019-06-27 21:00:24.503112
Max change in TE count:2082.2009462 2019-06-27 21:00:24.504509
Number changed TE:39347 2019-06-27 21:00:24.505776
Number TEs changed by at least 1 count:51 2019-06-27 21:00:24.506938
Number TEs changed by at least 1 count with at least 10 counts:51 2019-06-27 21:00:24.508277
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:23 2019-06-27 21:00:24.509553
Expectation maximization changed the average likelihood by: 0.00165694910525 2019-06-27 21:02:02.171758
Number of reads with average TE likelihoods changed by at least 0.1: 46248 2019-06-27 21:02:02.173264
Running expectation-maximization calculation for iteration:24 2019-06-27 21:02:02.174513
Average change in TE count:0.00970922054459 2019-06-27 21:02:04.768649
Max change in TE count:1634.66148864 2019-06-27 21:02:04.770092
Number changed TE:38801 2019-06-27 21:02:04.771515
Number TEs changed by at least 1 count:48 2019-06-27 21:02:04.772769
Number TEs changed by at least 1 count with at least 10 counts:47 2019-06-27 21:02:04.774048
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:20 2019-06-27 21:02:04.775345
Expectation maximization changed the average likelihood by: 0.00161711258005 2019-06-27 21:03:42.508227
Number of reads with average TE likelihoods changed by at least 0.1: 46257 2019-06-27 21:03:42.509565
Running expectation-maximization calculation for iteration:25 2019-06-27 21:03:42.510829
Average change in TE count:0.0081616192768 2019-06-27 21:03:45.128548
Max change in TE count:1271.04622332 2019-06-27 21:03:45.129995
Number changed TE:38336 2019-06-27 21:03:45.131224
Number TEs changed by at least 1 count:47 2019-06-27 21:03:45.132609
Number TEs changed by at least 1 count with at least 10 counts:46 2019-06-27 21:03:45.133893
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:17 2019-06-27 21:03:45.134903
Expectation maximization changed the average likelihood by: 0.0015844442798 2019-06-27 21:05:22.187435
Number of reads with average TE likelihoods changed by at least 0.1: 46269 2019-06-27 21:05:22.188765
Running expectation-maximization calculation for iteration:26 2019-06-27 21:05:22.190001
Average change in TE count:0.00688586579968 2019-06-27 21:05:24.790287
Max change in TE count:981.265316363 2019-06-27 21:05:24.791672
Number changed TE:37869 2019-06-27 21:05:24.792907
Number TEs changed by at least 1 count:43 2019-06-27 21:05:24.794107
Number TEs changed by at least 1 count with at least 10 counts:41 2019-06-27 21:05:24.795340
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:15 2019-06-27 21:05:24.796453
Expectation maximization changed the average likelihood by: 0.00155762887416 2019-06-27 21:07:01.908840
Number of reads with average TE likelihoods changed by at least 0.1: 46278 2019-06-27 21:07:01.910399
Running expectation-maximization calculation for iteration:27 2019-06-27 21:07:01.911900
Average change in TE count:0.00583103922745 2019-06-27 21:07:04.512607
Max change in TE count:823.981362897 2019-06-27 21:07:04.513862
Number changed TE:37456 2019-06-27 21:07:04.515010
Number TEs changed by at least 1 count:42 2019-06-27 21:07:04.516292
Number TEs changed by at least 1 count with at least 10 counts:42 2019-06-27 21:07:04.517547
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:15 2019-06-27 21:07:04.518693
Expectation maximization changed the average likelihood by: 0.00153577668836 2019-06-27 21:08:42.014963
Number of reads with average TE likelihoods changed by at least 0.1: 46280 2019-06-27 21:08:42.016686
Running expectation-maximization calculation for iteration:28 2019-06-27 21:08:42.018155
Average change in TE count:0.00499618689615 2019-06-27 21:08:44.614680
Max change in TE count:761.115133474 2019-06-27 21:08:44.616074
Number changed TE:36967 2019-06-27 21:08:44.617588
Number TEs changed by at least 1 count:39 2019-06-27 21:08:44.618847
Number TEs changed by at least 1 count with at least 10 counts:39 2019-06-27 21:08:44.620091
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:12 2019-06-27 21:08:44.621284
Expectation maximization changed the average likelihood by: 0.00151754606352 2019-06-27 21:10:21.590170
Number of reads with average TE likelihoods changed by at least 0.1: 46269 2019-06-27 21:10:21.591664
Running expectation-maximization calculation for iteration:29 2019-06-27 21:10:21.592839
Average change in TE count:0.00429722115661 2019-06-27 21:10:24.189884
Max change in TE count:702.23078489 2019-06-27 21:10:24.191362
Number changed TE:36586 2019-06-27 21:10:24.192623
Number TEs changed by at least 1 count:38 2019-06-27 21:10:24.194015
Number TEs changed by at least 1 count with at least 10 counts:38 2019-06-27 21:10:24.195130
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:11 2019-06-27 21:10:24.196402
Expectation maximization changed the average likelihood by: 0.00150227908722 2019-06-27 21:12:00.881477
Number of reads with average TE likelihoods changed by at least 0.1: 46272 2019-06-27 21:12:00.883071
Running expectation-maximization calculation for iteration:30 2019-06-27 21:12:00.884296
Average change in TE count:0.00374537930245 2019-06-27 21:12:03.483473
Max change in TE count:647.262833851 2019-06-27 21:12:03.484961
Number changed TE:36232 2019-06-27 21:12:03.486272
Number TEs changed by at least 1 count:38 2019-06-27 21:12:03.487640
Number TEs changed by at least 1 count with at least 10 counts:38 2019-06-27 21:12:03.488938
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:12 2019-06-27 21:12:03.490123
Expectation maximization changed the average likelihood by: 0.00148929562068 2019-06-27 21:13:40.612210
Number of reads with average TE likelihoods changed by at least 0.1: 46278 2019-06-27 21:13:40.613594
Running expectation-maximization calculation for iteration:31 2019-06-27 21:13:40.614695
Average change in TE count:0.00327274655353 2019-06-27 21:13:43.208198
Max change in TE count:596.091603337 2019-06-27 21:13:43.209662
Number changed TE:35861 2019-06-27 21:13:43.211016
Number TEs changed by at least 1 count:39 2019-06-27 21:13:43.212465
Number TEs changed by at least 1 count with at least 10 counts:38 2019-06-27 21:13:43.213860
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:11 2019-06-27 21:13:43.215073
Expectation maximization changed the average likelihood by: 0.00147823697945 2019-06-27 21:15:20.141891
Number of reads with average TE likelihoods changed by at least 0.1: 46272 2019-06-27 21:15:20.143444
Running expectation-maximization calculation for iteration:32 2019-06-27 21:15:20.144722
Average change in TE count:0.00287664556681 2019-06-27 21:15:22.761304
Max change in TE count:548.562483738 2019-06-27 21:15:22.762776
Number changed TE:35492 2019-06-27 21:15:22.764130
Number TEs changed by at least 1 count:38 2019-06-27 21:15:22.765285
Number TEs changed by at least 1 count with at least 10 counts:38 2019-06-27 21:15:22.766503
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:10 2019-06-27 21:15:22.767754
Expectation maximization changed the average likelihood by: 0.00146877295966 2019-06-27 21:16:59.012601
Number of reads with average TE likelihoods changed by at least 0.1: 46275 2019-06-27 21:16:59.014077
Running expectation-maximization calculation for iteration:33 2019-06-27 21:16:59.015336
Average change in TE count:0.00255849149255 2019-06-27 21:17:01.611622
Max change in TE count:504.499536169 2019-06-27 21:17:01.613099
Number changed TE:35219 2019-06-27 21:17:01.614275
Number TEs changed by at least 1 count:37 2019-06-27 21:17:01.615635
Number TEs changed by at least 1 count with at least 10 counts:34 2019-06-27 21:17:01.617011
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:9 2019-06-27 21:17:01.618155
Expectation maximization changed the average likelihood by: 0.00146086925031 2019-06-27 21:18:38.198824
Number of reads with average TE likelihoods changed by at least 0.1: 46278 2019-06-27 21:18:38.200441
Running expectation-maximization calculation for iteration:34 2019-06-27 21:18:38.201781
Average change in TE count:0.00228485720661 2019-06-27 21:18:40.787136
Max change in TE count:463.715127795 2019-06-27 21:18:40.788534
Number changed TE:34960 2019-06-27 21:18:40.790056
Number TEs changed by at least 1 count:30 2019-06-27 21:18:40.791405
Number TEs changed by at least 1 count with at least 10 counts:30 2019-06-27 21:18:40.792605
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:8 2019-06-27 21:18:40.793772
Expectation maximization changed the average likelihood by: 0.00145376151948 2019-06-27 21:20:17.578131
Number of reads with average TE likelihoods changed by at least 0.1: 46281 2019-06-27 21:20:17.579756
Running expectation-maximization calculation for iteration:35 2019-06-27 21:20:17.581020
Average change in TE count:0.00204286844238 2019-06-27 21:20:20.175911
Max change in TE count:426.0167221 2019-06-27 21:20:20.177406
Number changed TE:34694 2019-06-27 21:20:20.178878
Number TEs changed by at least 1 count:29 2019-06-27 21:20:20.180646
Number TEs changed by at least 1 count with at least 10 counts:29 2019-06-27 21:20:20.182240
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:8 2019-06-27 21:20:20.183842
Expectation maximization changed the average likelihood by: 0.00144761579982 2019-06-27 21:21:56.326294
Number of reads with average TE likelihoods changed by at least 0.1: 46281 2019-06-27 21:21:56.328044
Running expectation-maximization calculation for iteration:36 2019-06-27 21:21:56.329484
Average change in TE count:0.00184992582663 2019-06-27 21:21:58.929435
Max change in TE count:391.211709706 2019-06-27 21:21:58.930967
Number changed TE:34522 2019-06-27 21:21:58.932178
Number TEs changed by at least 1 count:29 2019-06-27 21:21:58.933564
Number TEs changed by at least 1 count with at least 10 counts:29 2019-06-27 21:21:58.934677
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:8 2019-06-27 21:21:58.935858
Expectation maximization changed the average likelihood by: 0.00144237608353 2019-06-27 21:23:35.015266
Number of reads with average TE likelihoods changed by at least 0.1: 46285 2019-06-27 21:23:35.016892
Running expectation-maximization calculation for iteration:37 2019-06-27 21:23:35.018193
Average change in TE count:0.00167405868438 2019-06-27 21:23:37.614310
Max change in TE count:359.110754416 2019-06-27 21:23:37.615647
Number changed TE:34279 2019-06-27 21:23:37.617111
Number TEs changed by at least 1 count:28 2019-06-27 21:23:37.618481
Number TEs changed by at least 1 count with at least 10 counts:28 2019-06-27 21:23:37.619823
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:7 2019-06-27 21:23:37.621102
Expectation maximization changed the average likelihood by: 0.00143757349531 2019-06-27 21:25:14.171650
Number of reads with average TE likelihoods changed by at least 0.1: 46287 2019-06-27 21:25:14.173085
Running expectation-maximization calculation for iteration:38 2019-06-27 21:25:14.174479
Average change in TE count:0.00151707123458 2019-06-27 21:25:16.756255
Max change in TE count:329.530127048 2019-06-27 21:25:16.757755
Number changed TE:34094 2019-06-27 21:25:16.759005
Number TEs changed by at least 1 count:27 2019-06-27 21:25:16.760180
Number TEs changed by at least 1 count with at least 10 counts:27 2019-06-27 21:25:16.761461
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:7 2019-06-27 21:25:16.762856
Expectation maximization changed the average likelihood by: 0.00143348606341 2019-06-27 21:26:52.637959
Number of reads with average TE likelihoods changed by at least 0.1: 46294 2019-06-27 21:26:52.639509
Running expectation-maximization calculation for iteration:39 2019-06-27 21:26:52.640830
Average change in TE count:0.00139494603957 2019-06-27 21:26:55.231428
Max change in TE count:302.293258583 2019-06-27 21:26:55.232736
Number changed TE:33854 2019-06-27 21:26:55.234004
Number TEs changed by at least 1 count:27 2019-06-27 21:26:55.235217
Number TEs changed by at least 1 count with at least 10 counts:27 2019-06-27 21:26:55.236214
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:8 2019-06-27 21:26:55.237649
Expectation maximization changed the average likelihood by: 0.00142969378107 2019-06-27 21:28:30.887263
Number of reads with average TE likelihoods changed by at least 0.1: 46295 2019-06-27 21:28:30.888909
Running expectation-maximization calculation for iteration:40 2019-06-27 21:28:30.890255
Average change in TE count:0.00127242305743 2019-06-27 21:28:33.478790
Max change in TE count:277.231740712 2019-06-27 21:28:33.480352
Number changed TE:33700 2019-06-27 21:28:33.481751
Number TEs changed by at least 1 count:24 2019-06-27 21:28:33.483030
Number TEs changed by at least 1 count with at least 10 counts:24 2019-06-27 21:28:33.484199
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:8 2019-06-27 21:28:33.485444
Expectation maximization changed the average likelihood by: 0.00142628373186 2019-06-27 21:30:09.711251
Number of reads with average TE likelihoods changed by at least 0.1: 46295 2019-06-27 21:30:09.712683
Running expectation-maximization calculation for iteration:41 2019-06-27 21:30:09.713898
Average change in TE count:0.00116724009883 2019-06-27 21:30:12.289737
Max change in TE count:254.185910391 2019-06-27 21:30:12.291183
Number changed TE:33517 2019-06-27 21:30:12.292622
Number TEs changed by at least 1 count:22 2019-06-27 21:30:12.293963
Number TEs changed by at least 1 count with at least 10 counts:22 2019-06-27 21:30:12.295178
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:30:12.296488
Expectation maximization changed the average likelihood by: 0.00142337798465 2019-06-27 21:31:47.928252
Number of reads with average TE likelihoods changed by at least 0.1: 46294 2019-06-27 21:31:47.929694
Running expectation-maximization calculation for iteration:42 2019-06-27 21:31:47.931117
Average change in TE count:0.00109452561911 2019-06-27 21:31:50.536766
Max change in TE count:233.005140503 2019-06-27 21:31:50.538262
Number changed TE:33376 2019-06-27 21:31:50.539589
Number TEs changed by at least 1 count:24 2019-06-27 21:31:50.540832
Number TEs changed by at least 1 count with at least 10 counts:23 2019-06-27 21:31:50.541951
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:6 2019-06-27 21:31:50.543186
Expectation maximization changed the average likelihood by: 0.00142065736362 2019-06-27 21:33:25.750015
Number of reads with average TE likelihoods changed by at least 0.1: 46297 2019-06-27 21:33:25.751474
Running expectation-maximization calculation for iteration:43 2019-06-27 21:33:25.752920
Average change in TE count:0.00099637559883 2019-06-27 21:33:28.335385
Max change in TE count:213.547891076 2019-06-27 21:33:28.336670
Number changed TE:33225 2019-06-27 21:33:28.337887
Number TEs changed by at least 1 count:22 2019-06-27 21:33:28.339108
Number TEs changed by at least 1 count with at least 10 counts:22 2019-06-27 21:33:28.340319
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:33:28.341507
Expectation maximization changed the average likelihood by: 0.00141819008395 2019-06-27 21:35:04.250397
Number of reads with average TE likelihoods changed by at least 0.1: 46301 2019-06-27 21:35:04.251953
Running expectation-maximization calculation for iteration:44 2019-06-27 21:35:04.253509
Average change in TE count:0.000926423704609 2019-06-27 21:35:06.826193
Max change in TE count:195.681613834 2019-06-27 21:35:06.827485
Number changed TE:33063 2019-06-27 21:35:06.828760
Number TEs changed by at least 1 count:21 2019-06-27 21:35:06.829895
Number TEs changed by at least 1 count with at least 10 counts:21 2019-06-27 21:35:06.831068
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:35:06.832260
Expectation maximization changed the average likelihood by: 0.00141595475478 2019-06-27 21:36:42.625183
Number of reads with average TE likelihoods changed by at least 0.1: 46300 2019-06-27 21:36:42.626480
Running expectation-maximization calculation for iteration:45 2019-06-27 21:36:42.627744
Average change in TE count:0.000857332461115 2019-06-27 21:36:45.235734
Max change in TE count:179.282531765 2019-06-27 21:36:45.237186
Number changed TE:32950 2019-06-27 21:36:45.238385
Number TEs changed by at least 1 count:20 2019-06-27 21:36:45.239725
Number TEs changed by at least 1 count with at least 10 counts:20 2019-06-27 21:36:45.240866
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:36:45.242041
Expectation maximization changed the average likelihood by: 0.00141389240555 2019-06-27 21:38:20.439304
Number of reads with average TE likelihoods changed by at least 0.1: 46299 2019-06-27 21:38:20.440875
Running expectation-maximization calculation for iteration:46 2019-06-27 21:38:20.442064
Average change in TE count:0.00079227159918 2019-06-27 21:38:23.032100
Max change in TE count:164.235342427 2019-06-27 21:38:23.033566
Number changed TE:32806 2019-06-27 21:38:23.034896
Number TEs changed by at least 1 count:20 2019-06-27 21:38:23.036262
Number TEs changed by at least 1 count with at least 10 counts:20 2019-06-27 21:38:23.037439
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:38:23.038813
Expectation maximization changed the average likelihood by: 0.00141202189043 2019-06-27 21:39:59.002635
Number of reads with average TE likelihoods changed by at least 0.1: 46299 2019-06-27 21:39:59.004148
Running expectation-maximization calculation for iteration:47 2019-06-27 21:39:59.005681
Average change in TE count:0.000737021598594 2019-06-27 21:40:01.585467
Max change in TE count:150.432858562 2019-06-27 21:40:01.587045
Number changed TE:32690 2019-06-27 21:40:01.588332
Number TEs changed by at least 1 count:19 2019-06-27 21:40:01.589623
Number TEs changed by at least 1 count with at least 10 counts:19 2019-06-27 21:40:01.590968
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:40:01.592212
Expectation maximization changed the average likelihood by: 0.0014105331083 2019-06-27 21:41:37.499622
Number of reads with average TE likelihoods changed by at least 0.1: 46301 2019-06-27 21:41:37.501094
Running expectation-maximization calculation for iteration:48 2019-06-27 21:41:37.502522
Average change in TE count:0.000691164155386 2019-06-27 21:41:40.099543
Max change in TE count:137.775616455 2019-06-27 21:41:40.100983
Number changed TE:32590 2019-06-27 21:41:40.102239
Number TEs changed by at least 1 count:18 2019-06-27 21:41:40.103668
Number TEs changed by at least 1 count with at least 10 counts:18 2019-06-27 21:41:40.104874
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:41:40.106228
Expectation maximization changed the average likelihood by: 0.00140902344907 2019-06-27 21:43:15.230267
Number of reads with average TE likelihoods changed by at least 0.1: 46301 2019-06-27 21:43:15.231965
Running expectation-maximization calculation for iteration:49 2019-06-27 21:43:15.233167
Average change in TE count:0.000641234525855 2019-06-27 21:43:17.826328
Max change in TE count:126.171469585 2019-06-27 21:43:17.827611
Number changed TE:32471 2019-06-27 21:43:17.828853
Number TEs changed by at least 1 count:18 2019-06-27 21:43:17.830155
Number TEs changed by at least 1 count with at least 10 counts:18 2019-06-27 21:43:17.831382
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:43:17.832524
Expectation maximization changed the average likelihood by: 0.00140759854425 2019-06-27 21:44:53.425303
Number of reads with average TE likelihoods changed by at least 0.1: 46301 2019-06-27 21:44:53.427046
Running expectation-maximization calculation for iteration:50 2019-06-27 21:44:53.428362
Average change in TE count:0.000600351862039 2019-06-27 21:44:56.006958
Max change in TE count:115.636994799 2019-06-27 21:44:56.008249
Number changed TE:32412 2019-06-27 21:44:56.009369
Number TEs changed by at least 1 count:18 2019-06-27 21:44:56.010648
Number TEs changed by at least 1 count with at least 10 counts:18 2019-06-27 21:44:56.011854
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:44:56.012960
Expectation maximization changed the average likelihood by: 0.00140642628317 2019-06-27 21:46:31.696950
Number of reads with average TE likelihoods changed by at least 0.1: 46305 2019-06-27 21:46:31.698427
Running expectation-maximization calculation for iteration:51 2019-06-27 21:46:31.699636
Average change in TE count:0.000573752666182 2019-06-27 21:46:34.289699
Max change in TE count:109.789331198 2019-06-27 21:46:34.291071
Number changed TE:32307 2019-06-27 21:46:34.292474
Number TEs changed by at least 1 count:15 2019-06-27 21:46:34.293680
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:46:34.295124
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:4 2019-06-27 21:46:34.296546
Expectation maximization changed the average likelihood by: 0.00140523894495 2019-06-27 21:48:09.237551
Number of reads with average TE likelihoods changed by at least 0.1: 46305 2019-06-27 21:48:09.239093
Running expectation-maximization calculation for iteration:52 2019-06-27 21:48:09.240193
Average change in TE count:0.000526706387801 2019-06-27 21:48:11.824855
Max change in TE count:104.227874967 2019-06-27 21:48:11.826166
Number changed TE:32203 2019-06-27 21:48:11.827426
Number TEs changed by at least 1 count:15 2019-06-27 21:48:11.828777
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:48:11.829995
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:4 2019-06-27 21:48:11.831208
Expectation maximization changed the average likelihood by: 0.00140419509835 2019-06-27 21:49:47.053050
Number of reads with average TE likelihoods changed by at least 0.1: 46305 2019-06-27 21:49:47.054662
Running expectation-maximization calculation for iteration:53 2019-06-27 21:49:47.056061
Average change in TE count:0.000504564376863 2019-06-27 21:49:49.643315
Max change in TE count:98.9397984083 2019-06-27 21:49:49.645005
Number changed TE:32122 2019-06-27 21:49:49.646208
Number TEs changed by at least 1 count:16 2019-06-27 21:49:49.647578
Number TEs changed by at least 1 count with at least 10 counts:16 2019-06-27 21:49:49.648741
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:5 2019-06-27 21:49:49.650101
Expectation maximization changed the average likelihood by: 0.0014032649676 2019-06-27 21:51:24.990779
Number of reads with average TE likelihoods changed by at least 0.1: 46309 2019-06-27 21:51:24.992293
Running expectation-maximization calculation for iteration:54 2019-06-27 21:51:24.993717
Average change in TE count:0.000466809371099 2019-06-27 21:51:27.581046
Max change in TE count:93.9127083258 2019-06-27 21:51:27.582298
Number changed TE:32040 2019-06-27 21:51:27.583673
Number TEs changed by at least 1 count:15 2019-06-27 21:51:27.584948
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:51:27.586126
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:51:27.587343
Expectation maximization changed the average likelihood by: 0.00140233818386 2019-06-27 21:53:02.233440
Number of reads with average TE likelihoods changed by at least 0.1: 46309 2019-06-27 21:53:02.235030
Running expectation-maximization calculation for iteration:55 2019-06-27 21:53:02.236210
Average change in TE count:0.000437849765234 2019-06-27 21:53:04.821629
Max change in TE count:89.1346483843 2019-06-27 21:53:04.822910
Number changed TE:31984 2019-06-27 21:53:04.824169
Number TEs changed by at least 1 count:15 2019-06-27 21:53:04.825346
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:53:04.826540
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:53:04.827724
Expectation maximization changed the average likelihood by: 0.00140153874811 2019-06-27 21:54:39.764457
Number of reads with average TE likelihoods changed by at least 0.1: 46309 2019-06-27 21:54:39.765956
Running expectation-maximization calculation for iteration:56 2019-06-27 21:54:39.767071
Average change in TE count:0.000415334713331 2019-06-27 21:54:42.345675
Max change in TE count:84.5940985575 2019-06-27 21:54:42.347076
Number changed TE:31912 2019-06-27 21:54:42.348339
Number TEs changed by at least 1 count:15 2019-06-27 21:54:42.349616
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:54:42.350881
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:54:42.352026
Expectation maximization changed the average likelihood by: 0.00140078299085 2019-06-27 21:56:17.404752
Number of reads with average TE likelihoods changed by at least 0.1: 46309 2019-06-27 21:56:17.406339
Running expectation-maximization calculation for iteration:57 2019-06-27 21:56:17.407591
Average change in TE count:0.000386093667688 2019-06-27 21:56:19.999590
Max change in TE count:80.2799716667 2019-06-27 21:56:20.001108
Number changed TE:31859 2019-06-27 21:56:20.002373
Number TEs changed by at least 1 count:15 2019-06-27 21:56:20.003596
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:56:20.004633
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:56:20.005852
Expectation maximization changed the average likelihood by: 0.00140014443752 2019-06-27 21:57:54.399653
Number of reads with average TE likelihoods changed by at least 0.1: 46310 2019-06-27 21:57:54.401133
Running expectation-maximization calculation for iteration:58 2019-06-27 21:57:54.402369
Average change in TE count:0.000364730718422 2019-06-27 21:57:56.981938
Max change in TE count:76.1816105884 2019-06-27 21:57:56.983213
Number changed TE:31799 2019-06-27 21:57:56.984396
Number TEs changed by at least 1 count:15 2019-06-27 21:57:56.985557
Number TEs changed by at least 1 count with at least 10 counts:15 2019-06-27 21:57:56.986643
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:57:56.987782
Expectation maximization changed the average likelihood by: 0.00139948292248 2019-06-27 21:59:31.709725
Number of reads with average TE likelihoods changed by at least 0.1: 46310 2019-06-27 21:59:31.711113
Running expectation-maximization calculation for iteration:59 2019-06-27 21:59:31.712369
Average change in TE count:0.000341869519353 2019-06-27 21:59:34.283456
Max change in TE count:72.2887815133 2019-06-27 21:59:34.284743
Number changed TE:31710 2019-06-27 21:59:34.285978
Number TEs changed by at least 1 count:14 2019-06-27 21:59:34.287090
Number TEs changed by at least 1 count with at least 10 counts:14 2019-06-27 21:59:34.288144
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 21:59:34.289216
Expectation maximization changed the average likelihood by: 0.00139893433134 2019-06-27 22:01:09.090080
Number of reads with average TE likelihoods changed by at least 0.1: 46310 2019-06-27 22:01:09.091645
Running expectation-maximization calculation for iteration:60 2019-06-27 22:01:09.092865
Average change in TE count:0.000326840366087 2019-06-27 22:01:11.676242
Max change in TE count:68.5916660654 2019-06-27 22:01:11.677653
Number changed TE:31637 2019-06-27 22:01:11.678886
Number TEs changed by at least 1 count:14 2019-06-27 22:01:11.680155
Number TEs changed by at least 1 count with at least 10 counts:14 2019-06-27 22:01:11.681417
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:3 2019-06-27 22:01:11.682543
Hello,
This may be a trivial one but I can't get the squire Call run with two groups of samples. Here is how my code looks like:
squire Call --group1 data_analysis/squire_count/Liv1_Hi272_Mut_* -2 data_analysis/squire_count/Liv2_Hi272_Cont_* -A mutant -B control -o data_analysis/squire_call/ -p 24 -v
The "squire_count" folder contains all the count file:
Then I get the following error:
Traceback (most recent call last): File "/scratch/gencore/conda3/envs/squire/bin/squire", line 11, in <module> load_entry_point('SQuIRE', 'console_scripts', 'squire')() File "/scratch/gencore/ma5877/TE/SQuIRE/squire/cli.py", line 156, in main subargs.func(args = subargs) File "/scratch/gencore/ma5877/TE/SQuIRE/squire/Call.py", line 270, in main group1_list=get_groupfiles(group1,gene_files,subF_files,TE_files,subfamily,count_folder) File "/scratch/gencore/ma5877/TE/SQuIRE/squire/Call.py", line 88, in get_groupfiles TE_files.append(find_file(count_folder,"_TEcounts.txt",sample,1,True)) File "/scratch/gencore/ma5877/TE/SQuIRE/squire/Call.py", line 43, in find_file raise Exception("No " + pattern + " file") Exception: No _TEcounts.txt file
But "_TEcounts.txt" file is there in the "squire_count" folder. Am I providing the wrong script? Please help!
Hi there,
It seems that this tool is mainly though to use with already published genomes and uses these fetch and clean in published datasets before the mapping and counting, however I have a custom genome with a custom TE de novo annotation (together with an annotation made with maker) and I was wondering If I could use this tool and just fake the fetch and clean step or if I will face some incompatibilities in downstream analysis.
Thank you!
I am trying to run SquiRE call. However, I am getting errors. I have attached the error below. Could anyone please help me fix the error.
count_folder=/home/arpatil/TE_counts
projectname=TEs_T1D_Normal_Squire
output_format=pdf
pthreads=12
call_folder=/home/arpatil/dete
condition2=Normal
table_only=False
group1=T1D_*
func=<function main at 0x7fa278b34140>
group2=Normal_*
subfamily=False
verbosity=True
condition1=T1D
Traceback (most recent call last):
File "/home/arpatil/miniconda3/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/home/arpatil/Tools/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/home/arpatil/Tools/SQuIRE/squire/Call.py", line 286, in main
create_TE_dict(TE_combo,TE_dict,threshold)
File "/home/arpatil/Tools/SQuIRE/squire/Call.py", line 134, in create_TE_dict
milliDiv = int(line[12])
ValueError: invalid literal for int() with base 10: 'Sample'
I tried editing Call.py line 134 by changing milliDiv = int(line[12])
to milliDiv = int(float(line[12]))
. However, I still keep getting error.
Hi!
The "loop_map.sh" script has the following lines:
for fastq in $fastq_folder/${samplename}*$r1suffix
do
read1+="${fastq},"
This concatenation means that squire map is called with an increasing number of libraries each time, and BAM files for the libraries have all the reads from all the libraries, instead only those from its respective FASTQ file.
Is there any reason for this?
I'm not an expert in Python so forgive me if I'm wrong, but I've tried to decompose the Call.py function to understand how the counttables used in the DESeq analysis are generated.
Within the create_subfamily_dict function, the count assigned into the dictionary is the uniq_counts rather than the total counts used elsewhere:
def create_subfamily_dict(infilepath,count_dict):
TE_classes=["LTR","LINE","SINE","Retroposon","DNA","RC"]
with open(infilepath,'r') as infile:
for line in infile:
line = line.rstrip()
line = line.split("\t")
taxo = line[2]
count=line[6]
if any(x in taxo for x in TE_classes):
if count=="tot_counts":
continue
else:
count = str(int(round(float(line[5]))))
sample = line[0]
if taxo not in count_dict:
count_dict[taxo] = {sample:count}
else:
count_dict[taxo][sample]=count
Can you explain why this is the case?
Hello,
I managed to arrive to mapping step, however it stops because of an error (please see below). Any help would be appreciated.
Thanks,
Martina
squire Map --read1 /data/MB998/HepC_Uwe/Uwe_Influenca/FASTqs/A549_DMSO_12h_dNS1_3_20141211_NoIndex_L001_R1_001.fastq.gz --read2 FALSE --read_length 100 --fetch_folder /data/MB998/SOFTWARE/SQuIRE/fetch_folder --build hg38 --verbosity
start time is:2019-04-08 18:52:14.682391
Map.pyc
fetch_folder=/data/MB998/SOFTWARE/SQuIRE/fetch_folder
name=False
extra=None
read_length=100
verbosity=True
pthreads=1
read1=/data/MB998/HepC_Uwe/Uwe_Influenca/FASTqs/A549_DMSO_12h_dNS1_3_20141211_NoIndex_L001_R1_001.fastq.gz
read2=FALSE
build=hg38
func=<function main at 0x7f6a967c5578>
trim3=0
map_folder=squire_map
Traceback (most recent call last):
File "/home/mb998/miniconda/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/data/MB998/SOFTWARE/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/data/MB998/SOFTWARE/SQuIRE/squire/Map.py", line 301, in main
index = find_file(fetch_folder,"_STAR",build, 1,True)
File "/data/MB998/SOFTWARE/SQuIRE/squire/Map.py", line 95, in find_file
raise Exception("No " + pattern + " file")
Exception: No _STAR file
Hi, I get the following error when running squire count
squire Count -r 76 -b hg19 -p 16 -v -e 10
Warning: gene "CDY1" (on chrY) has reference transcripts on both strands?
Warning: gene "TTTY3" (on chrY) has reference transcripts on both strands?
Warning: gene "TTTY3B" (on chrY) has reference transcripts on both strands?
Creating temporary files2019-06-27 08:42:57.385653
Creating unique and multiple alignment bedfiles 2019-06-27 08:42:57.385925
Identifying properly paired reads 2019-06-27 08:42:57.385953
Intersecting bam files with TE bedfile 2019-06-27 08:43:39.311783
Splitting into read1 and read 2 2019-06-27 08:44:24.749062
Combining adjacent TEs with same read alignment 2019-06-27 08:44:26.709173
Getting genomic coordinates of read2019-06-27 08:44:49.983114
Identifying and labeling unique and multi reads2019-06-27 08:44:57.855092
Matching paired-end mates and merging coordinates2019-06-27 08:45:03.586699
join: multi-character tab ‘$\t’
Traceback (most recent call last):
File "/mnt/RNA-SEQ/miniconda3/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/mnt/RNA-SEQ/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/mnt/RNA-SEQ/SQuIRE/squire/Count.py", line 1743, in main
match_reads(paired_tempfile1_ulabeled,paired_tempfile2_ulabeled,strandedness,paired_matched_tempfile,paired_unmatched1, paired_unmatched2,debug) #match pairs between paired files
File "/mnt/RNA-SEQ/SQuIRE/squire/Count.py", line 494, in match_reads
sp.check_call(["/bin/sh","-c",joincommand])
File "/mnt/RNA-SEQ/miniconda3/envs/squire/lib/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/sh', '-c', "join -j 12 -t $'\t' -o 1.1,1.2,1.3,1.4,1.5,1.6,1.7,1.8,1.9,1.10,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,2.10 /mnt/RNA-SEQ/MCF-7/squire_count/CTRL_1.fastq_paired_ulabeled_1.tmpx3ba1x_newread_v1 /mnt/RNA-SEQ/MCF-7/squire_count/CTRL_1.fastq_paired_ulabeled_2.tmpke3IRJ_newread_v1 > /mnt/RNA-SEQ/MCF-7/squire_count/CTRL_1.fastq_paired_matched.tmpeea9kD_10k_v1"]' returned non-zero exit status 1
Hi @wyang17
I have encountered the following ZeroDivisionError during SQuIRE Count stage. I have more than 20 different RNA-seq samples, and only this one has this issue every time I run this.
Could you please resolve this issue?
start time is:2020-07-02 13:33:52.016270
Count.pyc
Script Arguments
=================
count_folder=8_squire/squire_count
EM=auto
clean_folder=/scratch/twlab/hlee/genomes/danRer10/squire/squire_clean
fetch_folder=/scratch/twlab/hlee/genomes/danRer10/squire/squire_fetch
name=Kidney_rep1
tempfolder=False
verbosity=True
pthreads=8
strandedness=2
read_length=61
build=danRer10
func=<function main at 0x2b70404309b0>
map_folder=8_squire/squire_map
Quantifying Gene expression 2020-07-02 13:33:52.253733
Running Guided Stringtie on each bamfile Kidney_rep1 2020-07-02 13:33:52.385027
Warning: gene "adob" (on chr17) has reference transcripts on both strands?
Warning: gene "nr4a2a" (on chr9) has reference transcripts on both strands?
Creating temporary files2020-07-02 13:37:17.839948
Creating unique and multiple alignment bedfiles 2020-07-02 13:37:17.853702
Identifying properly paired reads 2020-07-02 13:37:17.853732
Intersecting bam files with TE bedfile 2020-07-02 13:59:53.159556
Splitting into read1 and read 2 2020-07-02 14:06:45.765105
Combining adjacent TEs with same read alignment 2020-07-02 14:07:18.157487
Getting genomic coordinates of read2020-07-02 14:11:31.226259
Identifying and labeling unique and multi reads2020-07-02 14:13:49.211829
Matching paired-end mates and merging coordinates2020-07-02 14:15:44.396913
Adding properly paired reads that have mates outside of TE into matched file2020-07-02 14:27:09.296185
Removing single-end reads that have matching paired-end mates at other alignment locations2020-07-02 14:27:26.819686
Identifying and labeling unique and multi fragments2020-07-02 14:27:43.282618
Identifying multi read pairs with one end unique2020-07-02 14:28:51.554478
counting unique alignments 2020-07-02 14:29:19.007752
counting multi alignments 2020-07-02 14:30:50.949737
Adding Tag information to aligned TEs 2020-07-02 14:31:48.145425
Calculating multialignment assignments 2020-07-02 14:31:49.172457
Running expectation-maximization calculation for iteration:1 2020-07-02 14:31:58.770414
Average change in TE count:0.0 2020-07-02 14:32:00.995930
Max change in TE count:0 2020-07-02 14:32:00.995979
Number changed TE:0 2020-07-02 14:32:00.995991
Number TEs changed by at least 1 count:0 2020-07-02 14:32:00.995997
Number TEs changed by at least 1 count with at least 10 counts:0 2020-07-02 14:32:00.996004
Number TEs changed by at least 1 count with at least 10 counts and > 1pct total count:0 2020-07-02 14:32:00.996010
Expectation maximization changed the average likelihood by: 0.113478553779 2020-07-02 14:32:10.475061
Number of reads with average TE likelihoods changed by at least 0.1: 151086 2020-07-02 14:32:10.475121
Running expectation-maximization calculation for iteration:2 2020-07-02 14:32:10.475132
Traceback (most recent call last):
File "/home/hyungjoo.lee/bin/miniconda3/envs/squire/bin/squire", line 11, in <module>
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/home/hyungjoo.lee/bin/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/home/hyungjoo.lee/bin/SQuIRE/squire/Count.py", line 1844, in main
TE_changecount = RepClass.calcmultiRep(iteration)
File "/home/hyungjoo.lee/bin/SQuIRE/squire/Count.py", line 948, in calcmultiRep
self.oldmulti_plus_perkb = self.old_counts_plus/((self.length_plus - avg_fraglength +1)/1000)
ZeroDivisionError: float division by zero
I am getting errors while running the call step in SquIRE. The counts were generated successfully without any errors. However, in the call step, I am getting errors. I have attached the errors below:
The following object is masked from ‘package:base:
apply
Warning message:
multiple methods tables found for ‘pos’
[1] "/home/arpatil/dete/TEs_T1D_Normal_Squire_gene_TE_counttable.txt"
[2] "/home/arpatil/dete/TEs_T1D_Normal_Squire_coldata.txt"
[3] "/home/arpatil/dete"
[4] "TEs_T1D_Normal_Squire"
[5] "4"
[6] "T1D"
[7] "Normal"
[8] "20"
Error in read.table(file = file, header = header, sep = sep, quote = quote, :
duplicate 'row.names' are not allowed
Calls: as.matrix -> read.delim -> read.table
Execution halted
Traceback (most recent call last):
File "/home/arpatil/miniconda3/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/home/arpatil/Tools/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/home/arpatil/Tools/SQuIRE/squire/Call.py", line 337, in main
create_rscript(counttable,coldata,outfolder,output_format,projectname,verbosity,str(pthreads),prefilter,condition1,condition2,label_no)
File "/home/arpatil/Tools/SQuIRE/squire/Call.py", line 198, in create_rscript
sp.check_call(["/bin/sh","-c",Rcommand])
File "/home/arpatil/miniconda3/envs/squire/lib/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/sh', '-c', 'Rscript /home/arpatil/dete/TEs_T1D_Normal_Squire_R_script.tmpOGNGUJ /home/arpatil/dete/TEs_T1D_Normal_Squire_gene_TE_counttable.txt /home/arpatil/dete/TEs_T1D_Normal_Squire_coldata.txt /home/arpatil/dete TEs_T1D_Normal_Squire 4 T1D Normal 20']' returned non-zero exit status 1
I have the count's table generated. For ex, I got the following files: TEs_T1D_Normal_Squire_coldata.txt, TEs_T1D_Normal_Squire _counttable.txt, TEs_T1D_Normal_Squire _combo.txt files generated successfully. But, it is not giving the differentially expressed TEs output.
Could anyone please help me in addressing this error? Thanks!
Hello, I would like to give you a question about the use of SQuIRE for the comparison of multiple RNA-seq data generated with different libraries.
I am performing the differential analysis of samples of data obtained with different libraries (first and unstranded). In the count step I indicate the type of specific library to correctly perform the TE quantification but when executing the differential analysis I obtain few significant differences due to the differences between both libraries.
When I remove the unstranded samples of the analysis it finish successfully and I have more consistent data. Is there a way to include this batch effect in squire?
I'm trying to process the public sample (https://www.ncbi.nlm.nih.gov/sra/?term=SRR6325536). The Count process never ends (now the subprocess stringtie is running more than 80 hours). I tried to redo the process for 3 times (in distinct machines) but the issue persists. No log messages was reported by SQuiRE. Thanks.
Hi, I want to find TE expression of novel TE insertions detected in WES data. Do you have any suggestions on how to look for those specific TE insertions expression?
Hi Wan,
I was wondering if there is a way (a parameter or a script inside the package) that it could allow us to "correct" the total number of reads in the counting phase, by taking account reads that are mapped across the intron-TE boundaries.
Thank you very much in advance,
Vasilis.
Hello,
I am really sorry for bothering you again, but I have a new problem when I am trying to run the fetch phase.
It looks like the arguments are fine, however, I am getting an error that has to do with the connection with UCSC.
`Downloading Compressed Chromosome files...
Traceback (most recent call last):
File "/scratch/x.v.l.01/yes/envs/squire/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/scratch/x.v.l.01/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/scratch/x.v.l.01/SQuIRE/squire/Fetch.py", line 212, in main
urllib.urlretrieve(chrom_loc1, filename=chrom_name_compressed)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/urllib.py", line 98, in urlretrieve
return opener.retrieve(url, filename, reporthook, data)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/urllib.py", line 245, in retrieve
fp = self.open(url, data)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/urllib.py", line 213, in open
return getattr(self, name)(url)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/urllib.py", line 350, in open_http
h.endheaders(data)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/httplib.py", line 1038, in endheaders
self._send_output(message_body)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/httplib.py", line 882, in _send_output
self.send(msg)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/httplib.py", line 844, in send
self.connect()
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/httplib.py", line 821, in connect
self.timeout, self.source_address)
File "/scratch/x.v.l.01/yes/envs/squire/lib/python2.7/socket.py", line 575, in create_connection
raise err
IOError: [Errno socket error] [Errno 101] Network is unreachable`
I have tested to download a single file from UCSC it works fine, meaning that my cluster is reachable from the internet.
Any possible explanation?
Many thanks in advance.
Hi,
I'm quite excited to use SQuIRE, but I work on a non-UCSC genome, with no files to "fetch". Would it be possible to make available the formats of the "fetch" files, but also potentially an option to upload our own well-formatted files? Otherwise, is it possible to maybe tweak the code to add my own files (#notabioinformaticienhere)?
Thank you very much
Rita
Traceback (most recent call last):
File "/users/cpacyna/miniconda3/envs/squire_git/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/users/cpacyna/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/users/cpacyna/SQuIRE/squire/Count.py", line 1564, in main
filter_abund(abund_ref,genename_dict,False, read_length)
TypeError: filter_abund() takes exactly 3 arguments (4 given)
Hi,
I was wondering if it will be possible to use the 'GENCODE comprehensive catalog' as an option to run SQuIRE.
Is RefSeq been chosen for a specific reason? Like shown here http://www.biomedcentral.com/1471-2164/16/S8/S2, the GENCODE comprehensive contains more annotation and covers more genomic regions which sounds ideal for TEs quantification purposes.
Would SQuIRE work with such annotation?
Thanks
Half of my count jobs are going through, half stop with this error:
I've come across this error before but it did not affect so many files as now.
Thanks!
Traceback (most recent call last):
File "/users/cpacyna/miniconda3/envs/squire_git/bin/squire", line 11, in
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/users/cpacyna/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/users/cpacyna/SQuIRE/squire/Count.py", line 1683, in main
reduce_reads(paired_bed_tempfile1_sorted, paired_reduced_tempfile1,debug)
File "/users/cpacyna/SQuIRE/squire/Count.py", line 378, in reduce_reads
prev.line_split[15] = prev_TE_ID
UnboundLocalError: local variable 'prev_TE_ID' referenced before assignment
Hi @wyang17 ,
in line 1567 of Count.py has an issue on "genename_dict" : filter_tx(outgtf_ref, genename_dict,read_length,genecounts)
Thank you,
Hi!
Another question: in running squire Map on my paired-end RNA-Seq data, I'm getting a really large number of reads classified as Unmapped (about 70%, with 30% falling under "Unmapped other"). Is this expected? What are the possible reasons for these results, and how can I adjust?
The command I'm running is squire Map -1 $read1 -2 $read2 -f $squire_fetch -r $read_length -b hg19 -p 8 -v
Thanks!
Hello all,
I was told to use the reference genome annotation with extra 5'-UTR and 3'-UTR records when I want to analyze the ref & TE expression out of the nuclear RNA-seq data set. I assume I should do the same thing using SQuIRE pipeline, shouldn't I?
I am asking because I observed slightly different marker gene expression between the SQuIRE and the other pipeline. Still trying to figure out what the real cause is.
Thanks in advance~
Maxwell
Hi all,
I am using SQuIRE to look at TE expression in human fibroblast RNA-Seq data, but I'm slightly surprised by the results SQuIRE is producing. I am running squire Call with option -s to get subfamily level results as I do not need locus-level resolution. I find that ~81% tested TE subfamilies have mean counts > 10 and 60% > 100. With FPKM, this translates to 75% with an FPKM >2, 54% > 10 and 17% > 100. If I re-run the same data at a locus level, I find between 30-70% TE loci have counts > 10. Though I have been unable to find other data to compare to for what TE expression in fibroblasts should look like, this seems incredibly high and feels like something has gone wrong somewhere. Surely all of these TEs are not expressed in adult fibroblasts?
I am wondering whether other people see this with their data? is it to be expected with the way the read assignment algorithms work or have I made a mistake somewhere?
Is the issue with the cut-off value for when a TE is deemed 'expressed? The SQuIRE authors use a > 10 count cut-off at a locus level, so presumably it needs to be quite a lot higher than this at a subfamily level.
These are examples of the commands I am using:
squire Fetch -b 'hg38' -p 8 -r -f -c -x
squire Clean -b 'hg38'
squire Map -1 <fq file 1> -2 <fq file 2> -n -p 15 -r 75
squire Count -r 75 -n -p 15 -s 2
squire Call -1 TREATMENT1,TREATMENT2,TREATMENT3 -2 CONTROL1,CONTROL2,CONTROL3 -A Treatment -B Control -s -p 15 -N
Any advice or comments are appreciated; thanks in advance for your help.
Tom
I am getting this error in only 8 of 86 samples:
Traceback (most recent call last):
File "/home/alejandro.rubio/tools/miniconda3/envs/squire/bin/squire", line 11, in
srun: error: mulhacen9: task 0: Exited with exit code 1
load_entry_point('SQuIRE', 'console_scripts', 'squire')()
File "/home/alejandro.rubio/tools/SQuIRE/squire/cli.py", line 156, in main
subargs.func(args = subargs)
File "/home/alejandro.rubio/tools/SQuIRE/squire/Count.py", line 1560, in main
paired_end = is_paired(bamfile,basename,tempfolder,debug)
File "/home/alejandro.rubio/tools/SQuIRE/squire/Count.py", line 325, in is_paired
return paired
UnboundLocalError: local variable 'paired' referenced before assignment
Any idea?
Hi,
May I have some guidance on how to use the TE counts result file to do DE analysis, as the rows differ for different samples, and if run the call step, it will show the error "list index out of range".
Thanks so much!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.