2018-07-08 17:12:05,62] [info] Running with database db.url = jdbc:hsqldb:mem:b928854f-1b26-4f74-8a45-e72f73b30968;shutdown=false;hsqldb.tx=mvcc
[2018-07-08 17:12:11,01] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000
[2018-07-08 17:12:11,02] [info] [RenameWorkflowOptionsInMetadata] 100%
[2018-07-08 17:12:11,10] [info] Running with database db.url = jdbc:hsqldb:mem:b55a6427-b864-4e72-9858-1211b6533178;shutdown=false;hsqldb.tx=mvcc
[2018-07-08 17:12:11,42] [warn] This actor factory is deprecated. Please use cromwell.backend.google.pipelines.v1alpha2.PipelinesApiLifecycleActorFactory for PAPI v1 or cromwell.backend.google.pipelines .v2alpha1.PipelinesApiLifecycleActorFactory for PAPI v2
[2018-07-08 17:12:11,42] [warn] Couldn't find a suitable DSN, defaulting to a Noop one.
[2018-07-08 17:12:11,43] [info] Using noop to send events.
[2018-07-08 17:12:11,69] [info] Slf4jLogger started
[2018-07-08 17:12:11,87] [info] Workflow heartbeat configuration:
{
"cromwellId" : "cromid-df3d320",
"heartbeatInterval" : "2 minutes",
"ttl" : "10 minutes",
"writeBatchSize" : 10000,
"writeThreshold" : 10000
}
[2018-07-08 17:12:11,91] [info] Metadata summary refreshing every 2 seconds.
[2018-07-08 17:12:11,95] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.
[2018-07-08 17:12:11,95] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.
[2018-07-08 17:12:11,95] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.
[2018-07-08 17:12:12,71] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds.
[2018-07-08 17:12:12,73] [info] JES batch polling interval is 33333 milliseconds
[2018-07-08 17:12:12,73] [info] JES batch polling interval is 33333 milliseconds
[2018-07-08 17:12:12,73] [info] JES batch polling interval is 33333 milliseconds
[2018-07-08 17:12:12,73] [info] PAPIQueryManager Running with 3 workers
[2018-07-08 17:12:12,74] [info] SingleWorkflowRunnerActor: Submitting workflow
[2018-07-08 17:12:12,78] [info] Unspecified type (Unspecified version) workflow 068f45d8-f29a-4335-8c0e-a711391af811 submitted
[2018-07-08 17:12:12,83] [info] SingleWorkflowRunnerActor: Workflow submitted 068f45d8-f29a-4335-8c0e-a711391af811
[2018-07-08 17:12:12,83] [info] 1 new workflows fetched
[2018-07-08 17:12:12,83] [info] WorkflowManagerActor Starting workflow 068f45d8-f29a-4335-8c0e-a711391af811
[2018-07-08 17:12:12,84] [info] WorkflowManagerActor Successfully started WorkflowActor-068f45d8-f29a-4335-8c0e-a711391af811
[2018-07-08 17:12:12,84] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2018-07-08 17:12:12,84] [warn] SingleWorkflowRunnerActor: received unexpected message: Done in state RunningSwraData
[2018-07-08 17:12:12,85] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.
[2018-07-08 17:12:12,89] [info] MaterializeWorkflowDescriptorActor [068f45d8]: Parsing workflow as WDL draft-2
[2018-07-08 17:13:26,22] [info] MaterializeWorkflowDescriptorActor [068f45d8]: Call-to-Backend assignments: chip.macs2 -> Local, chip.bam2ta_ctl -> Local, chip.spp_ppr2 -> Local, chip.bwa -> Local, chip .qc_report -> Local, chip.bwa_ctl -> Local, chip.filter -> Local, chip.overlap -> Local, chip.pool_ta -> Local, chip.idr_ppr -> Local, chip.filter_ctl -> Local, chip.macs2_pr2 -> Local, chip.spp_pr1 -> Local, chip.read_genome_tsv -> Local, chip.merge_fastq_ctl -> Local, chip.macs2_ppr1 -> Local, chip.pool_ta_pr2 -> Local, chip.trim_fastq -> Local, chip.overlap_ppr -> Local, chip.bam2ta -> Local, chip. pool_ta_ctl -> Local, chip.reproducibility_idr -> Local, chip.spp_pooled -> Local, chip.fraglen_mean -> Local, chip.spr -> Local, chip.bam2ta_no_filt -> Local, chip.macs2_pr1 -> Local, chip.fingerprint -> Local, chip.xcor -> Local, chip.spp_pr2 -> Local, chip.spp -> Local, chip.merge_fastq -> Local, chip.bam2ta_no_filt_R1 -> Local, chip.reproducibility_overlap -> Local, chip.overlap_pr -> Local, chip. idr -> Local, chip.macs2_ppr2 -> Local, chip.bwa_R1 -> Local, chip.spp_ppr1 -> Local, chip.macs2_pooled -> Local, chip.idr_pr -> Local, chip.choose_ctl -> Local, chip.pool_ta_pr1 -> Local
[2018-07-08 17:13:26,34] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,35] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,36] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [preemptible, disks, cpu, time, memory] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:26,37] [warn] Local [068f45d8]: Key/s [cpu, memory, time, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
[2018-07-08 17:13:28,66] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Starting chip.read_genome_tsv
[2018-07-08 17:13:28,67] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: '!align_only && !true_rep_only'. Running conditional section
[2018-07-08 17:13:28,68] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: '!true_rep_only'. Running conditional section
[2018-07-08 17:13:28,87] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.read_genome_tsv:NA:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 17:13:29,34] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.read_genome_tsv:NA:1]: echo "Reading genome_tsv /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068 f45d8-f29a-4335-8c0e-a711391af811/call-read_genome_tsv/inputs/1532045310/mm10.tsv ..."
[2018-07-08 17:13:29,46] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.read_genome_tsv:NA:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d 8-f29a-4335-8c0e-a711391af811/call-read_genome_tsv/execution/script
[2018-07-08 17:13:29,70] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition NOT met: 'peak_caller_ == "macs2"'. Bypassing conditional section
[2018-07-08 17:13:29,70] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: 'enable_idr'. Running conditional section
[2018-07-08 17:13:29,70] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: 'enable_idr'. Running conditional section
[2018-07-08 17:13:29,70] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: 'peak_caller_ == "spp"'. Running conditional section
[2018-07-08 17:13:29,70] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: 'peak_caller_ == "spp"'. Running conditional section
[2018-07-08 17:13:29,71] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Condition met: '!align_only && !true_rep_only && enable_idr'. Running conditional section
[2018-07-08 17:13:31,99] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.read_genome_tsv:NA:1]: job id: 18903
[2018-07-08 17:13:32,00] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.read_genome_tsv:NA:1]: Status change from - to Done
[2018-07-08 17:13:32,80] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Starting chip.merge_fastq_ctl, chip.merge_fastq
[2018-07-08 17:13:33,74] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq_ctl:0:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 17:13:33,74] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq:0:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 17:13:33,85] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq_ctl:0:1]: python $(which encode_merge_fastq.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-merge_fastq_ctl/shard-0/execution/write_tsv_609523603b8830d2bf2d45a4a71d8dd7.tmp
--nth 2
[2018-07-08 17:13:33,85] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq:0:1]: python $(which encode_merge_fastq.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-merge_fastq/shard-0/execution/write_tsv_577d047f7bb8a8ea8c6ee89ee3d97c7b.tmp
--nth 2
[2018-07-08 17:13:33,85] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29 a-4335-8c0e-a711391af811/call-merge_fastq/shard-0/execution/script
[2018-07-08 17:13:33,85] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq_ctl:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8 -f29a-4335-8c0e-a711391af811/call-merge_fastq_ctl/shard-0/execution/script
[2018-07-08 17:13:36,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq:0:1]: job id: 18944
[2018-07-08 17:13:36,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq_ctl:0:1]: job id: 18946
[2018-07-08 17:13:36,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq:0:1]: Status change from - to Done
[2018-07-08 17:13:36,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.merge_fastq_ctl:0:1]: Status change from - to Done
[2018-07-08 17:13:38,92] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Starting chip.bwa, chip.bwa_ctl
[2018-07-08 17:13:39,74] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: Unrecognized runtime attribute keys: preemptible, disks, cpu, time, memory
[2018-07-08 17:13:39,74] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: Unrecognized runtime attribute keys: preemptible, disks, cpu, time, memory
[2018-07-08 17:13:39,76] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: python $(which encode_bwa.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-bwa/shard-0/inputs/1424220334/mm10_no_alt_analysis_set_ENCODE.fasta.tar
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-bwa/shard-0/inputs/-1993312639/merge_fastqs_R1_RYBP.merged.fastq.gz
--nth 4
[2018-07-08 17:13:39,76] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: python $(which encode_bwa.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-bwa_ctl/shard-0/inputs/1424220334/mm10_no_alt_analysis_set_ENCODE.fasta.tar
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-bwa_ctl/shard-0/inputs/1654728541/merge_fastqs_R1_IgG.merged.fastq.gz
--nth 4
[2018-07-08 17:13:39,77] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-43 35-8c0e-a711391af811/call-bwa_ctl/shard-0/execution/script
[2018-07-08 17:13:39,77] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8 c0e-a711391af811/call-bwa/shard-0/execution/script
[2018-07-08 17:13:41,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: job id: 19040
[2018-07-08 17:13:41,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: job id: 19041
[2018-07-08 17:13:41,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-08 17:13:41,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-08 18:03:30,01] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa_ctl:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-08 18:03:35,69] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Starting chip.filter_ctl
[2018-07-08 18:03:35,73] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 18:03:35,77] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0:1]: python $(which encode_filter.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-filter_ctl/shard-0/inputs/1613485654/IgG.merged.bam
--dup-marker picard
--mapq-thresh 30
\
ugly part to deal with optional outputs with Google JES backend
touch null
[2018-07-08 18:03:35,88] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a -4335-8c0e-a711391af811/call-filter_ctl/shard-0/execution/script
[2018-07-08 18:03:36,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0:1]: job id: 1170
[2018-07-08 18:03:36,99] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-08 18:07:57,26] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bwa:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-08 18:08:02,93] [info] WorkflowExecutionActor-068f45d8-f29a-4335-8c0e-a711391af811 [068f45d8]: Starting chip.bam2ta_no_filt, chip.filter
[2018-07-08 18:08:03,73] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 18:08:03,73] [warn] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: Unrecognized runtime attribute keys: disks, cpu, time, memory
[2018-07-08 18:08:03,74] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: python $(which encode_bam2ta.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-bam2ta_no_filt/shard-0/inputs/600163450/RYBP.merged.bam
--disable-tn5-shift
--regex-grep-v-ta 'chrM'
--subsample 0
[2018-07-08 18:08:03,74] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8- f29a-4335-8c0e-a711391af811/call-bam2ta_no_filt/shard-0/execution/script
[2018-07-08 18:08:03,76] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: python $(which encode_filter.py)
/home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-4335-8c0e-a711391af811/call-filter/shard-0/inputs/600163450/RYBP.merged.bam
--dup-marker picard
--mapq-thresh 30
\
ugly part to deal with optional outputs with Google JES backend
touch null
[2018-07-08 18:08:03,77] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: executing: /bin/bash /home/chengwsh/Encode/chip-seq-pipeline2/cromwell-executions/chip/068f45d8-f29a-433 5-8c0e-a711391af811/call-filter/shard-0/execution/script
[2018-07-08 18:08:06,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: job id: 2502
[2018-07-08 18:08:06,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: job id: 2516
[2018-07-08 18:08:06,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-08 18:08:06,97] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-07-08 18:10:26,92] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.bam2ta_no_filt:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-08 18:11:32,77] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter_ctl:0: 1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-08 18:16:25,24] [info] BackgroundConfigAsyncJobExecutionActor [068f45d8chip.filter:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-07-08 18:16:26,14] [error] WorkflowManagerActor Workflow 068f45d8-f29a-4335-8c0e-a711391af81 1 failed (during ExecutingWorkflowState): Failed to evaluate job outputs:
Bad output 'filter_ctl.nodup_bai': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.flagstat_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.dup_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.pbc_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs:
Bad output 'filter_ctl.nodup_bai': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.flagstat_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.dup_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter_ctl.pbc_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1 (StandardAsyncExecutionActor.scala:786)
at scala.util.Success.$anonfun$map$1(Try.scala:251)
at scala.util.Success.map(Try.scala:209)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfig urator.scala:43)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Failed to evaluate job outputs:
Bad output 'filter.nodup_bai': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.flagstat_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.dup_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.pbc_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs:
Bad output 'filter.nodup_bai': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.flagstat_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.dup_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
Bad output 'filter.pbc_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0
at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1 (StandardAsyncExecutionActor.scala:786)
at scala.util.Success.$anonfun$map$1(Try.scala:251)
at scala.util.Success.map(Try.scala:209)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfig urator.scala:43)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[2018-07-08 18:16:26,14] [info] WorkflowManagerActor WorkflowActor-068f45d8-f29a-4335-8c0e-a711391 af811 is in a terminal state: WorkflowFailedState
[2018-07-08 18:17:28,68] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
[2018-07-08 18:17:31,99] [info] Workflow polling stopped
[2018-07-08 18:17:32,12] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds
[2018-07-08 18:17:32,12] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds
[2018-07-08 18:17:32,15] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds
[2018-07-08 18:17:32,17] [info] Aborting all running workflows.
[2018-07-08 18:17:32,19] [info] JobExecutionTokenDispenser stopped
[2018-07-08 18:17:32,19] [info] WorkflowStoreActor stopped
[2018-07-08 18:17:32,24] [info] WorkflowLogCopyRouter stopped
[2018-07-08 18:17:32,24] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds
[2018-07-08 18:17:32,24] [info] WorkflowManagerActor stopped
[2018-07-08 18:17:32,24] [info] WorkflowManagerActor All workflows finished
[2018-07-08 18:17:32,24] [info] Connection pools shut down
[2018-07-08 18:17:32,24] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds
[2018-07-08 18:17:32,24] [info] Shutting down JobStoreActor - Timeout = 1800 seconds
[2018-07-08 18:17:32,24] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds
[2018-07-08 18:17:32,24] [info] SubWorkflowStoreActor stopped
[2018-07-08 18:17:32,24] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds
[2018-07-08 18:17:32,24] [info] Shutting down DockerHashActor - Timeout = 1800 seconds
[2018-07-08 18:17:32,24] [info] Shutting down IoProxy - Timeout = 1800 seconds
[2018-07-08 18:17:32,27] [info] DockerHashActor stopped
[2018-07-08 18:17:32,27] [info] IoProxy stopped
[2018-07-08 18:17:32,30] [info] KvWriteActor Shutting down: 0 queued messages to process
[2018-07-08 18:17:32,30] [info] WriteMetadataActor Shutting down: 0 queued messages to process
[2018-07-08 18:17:32,30] [info] CallCacheWriteActor Shutting down: 0 queued messages to process
[2018-07-08 18:17:32,30] [info] CallCacheWriteActor stopped
[2018-07-08 18:17:32,33] [info] JobStoreActor stopped
[2018-07-08 18:17:32,34] [info] ServiceRegistryActor stopped
[2018-07-08 18:17:32,36] [info] Database closed
[2018-07-08 18:17:32,36] [info] Stream materializer shut down
Workflow 068f45d8-f29a-4335-8c0e-a711391af811 transitioned to state Failed
[2018-07-08 18:17:32,46] [info] Automatic shutdown of the async connection
[2018-07-08 18:17:32,46] [info] Gracefully shutdown sentry threads.
[2018-07-08 18:17:32,46] [info] Shutdown finished.