whitwham / tears Goto Github PK
View Code? Open in Web Editor NEWStream files to and from iRODS
License: GNU General Public License v3.0
Stream files to and from iRODS
License: GNU General Public License v3.0
Suggest
-w
option for when writing at file to iRODS-w
, but allowing -w
option on command line) for next revision-w
option for following revisionWe are considering using tears -r
to feed a bandwidth-throttled ichksum -K
substitute, with additional benefit of ongoing end-to-end transfer verification.
If we went ahead with that, we would want to specify the source resource rather than let tears choose, so we can verify every copy. How big does that look?
We don't need tears -w -R foo ...
.
@kript FYI
Currently if we use tears to write to iRODS and a file of that name already exists we get the following error:
Extra error message:
Error: rcDataObjCreate failed with status -312000:OVERWRITE_WITHOUT_FORCE_FLAG
Please can we have a force flag to allow us to overwrite an existing file. Thanks.
Please can tears -h
show the version and project URL?
I already knew where to come search...
mca@cgpbar:~$ /software/solexa/pkg/tears/current/tears -h
Usage: /software/solexa/pkg/tears/current/tears [-b bytes -v] /path/to/irods/output/file < filein
or /software/solexa/pkg/tears/current/tears [-b bytes -v] -r /path/to/irods/input/file > fileout
-b bytes read/write buffer (default 1048576)
-v verbose mode
current is from 11.11.15
Test harness passes in 4.1.10 and fails in 4.1.11 (I think) and 4.1.12
#expanded from the test harness for readability
jc18@farm3-head4:/tmp$ SMALL_FILE=$(mktemp irods_test_small_XXX)
jc18@farm3-head4:/tmp$ dd if=/dev/zero of="${SMALL_FILE}" bs=1k count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 4.4354e-05 s, 23.1 MB/s
jc18@farm3-head4:/tmp$ ls -lah "${SMALL_FILE}"
-rw------- 1 jc18 team94 1.0K Jul 20 13:35 irods_test_small_jra
jc18@farm3-head4:/tmp$ irods_cwd=$(ipwd)
jc18@farm3-head4:/tmp$ echo $irods_cwd
/seq-dev/home/jc18#Sanger1-dev
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -w "${irods_cwd}/${SMALL_FILE}_tears" < "${SMALL_FILE}"
jc18@farm3-head4:/tmp$ echo $?
0
jc18@farm3-head4:/tmp$ ils
/seq-dev/home/jc18#Sanger1-dev:
1FBLYX_0.json
24174_5#888.cram
irods_test_large_QZI
irods_test_small_jra_tears
jc18@farm3-head4:/tmp$ ils -l irods_test_small_jra_tears
jc18 0 root;replicate;red;red3;irods-seq-i16-fg 1024 2018-07-20.13:36 & irods_test_small_jra_tears
jc18 1 root;replicate;green;green2;irods-seq-sr02-ddn-ra08-9-10-11 1024 2018-07-20.13:36 & irods_test_small_jra_tears
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r "${irods_cwd}/${SMALL_FILE}_tears" > $(mktemp --tmpdir irods_test_XXX)
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r "${irods_cwd}/${SMALL_FILE}_tears" > kabloom
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r /seq-dev/home/jc18#Sanger1-dev/irods_test_small_jra_tears > kabloom
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r /seq-dev/home/jc18#Sanger1-dev/irods_test_small_jra_tears
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
What is interesting also is that the HIERARCHY_ERROR
doesn't appear in any of the IES or IRES logs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.