whitwham / tears Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 5.0 45 KB

Stream files to and from iRODS

License: GNU General Public License v3.0

C 65.32% Makefile 3.27% M4 31.41%

irods

tears's People

Contributors

Stargazers

Watchers

Forkers

dkj wtsi-hgi kjsanger alanking korydraughn

tears's Issues

default write is scary

Suggest

adding a -w option for when writing at file to iRODS
retain current behaviour (of writing without -w, but allowing -w option on command line) for next revision
drop writing without the -w option for following revision

support `tears -r -R src-resource /path` ?

We are considering using tears -r to feed a bandwidth-throttled ichksum -K substitute, with additional benefit of ongoing end-to-end transfer verification.

If we went ahead with that, we would want to specify the source resource rather than let tears choose, so we can verify every copy. How big does that look?

We don't need tears -w -R foo ....

@kript FYI

Currently if we use tears to write to iRODS and a file of that name already exists we get the following error:
Extra error message:
Error: rcDataObjCreate failed with status -312000:OVERWRITE_WITHOUT_FORCE_FLAG

Please can we have a force flag to allow us to overwrite an existing file. Thanks.

more --help

Please can tears -h show the version and project URL?

I already knew where to come search...

mca@cgpbar:~$ /software/solexa/pkg/tears/current/tears -h
Usage: /software/solexa/pkg/tears/current/tears [-b bytes -v] /path/to/irods/output/file < filein 
    or /software/solexa/pkg/tears/current/tears [-b bytes -v] -r /path/to/irods/input/file > fileout

        -b bytes        read/write buffer (default 1048576)
        -v              verbose mode

current is from 11.11.15

Possible regression vs iRODS 4.1.12

Test harness passes in 4.1.10 and fails in 4.1.11 (I think) and 4.1.12

#expanded from the test harness for readability
jc18@farm3-head4:/tmp$ SMALL_FILE=$(mktemp irods_test_small_XXX)
jc18@farm3-head4:/tmp$ dd if=/dev/zero of="${SMALL_FILE}" bs=1k count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 4.4354e-05 s, 23.1 MB/s
jc18@farm3-head4:/tmp$ ls -lah "${SMALL_FILE}"
-rw------- 1 jc18 team94 1.0K Jul 20 13:35 irods_test_small_jra
jc18@farm3-head4:/tmp$ irods_cwd=$(ipwd)
jc18@farm3-head4:/tmp$ echo $irods_cwd
/seq-dev/home/jc18#Sanger1-dev
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -w  "${irods_cwd}/${SMALL_FILE}_tears" < "${SMALL_FILE}"
jc18@farm3-head4:/tmp$ echo $?
0
jc18@farm3-head4:/tmp$ ils 
/seq-dev/home/jc18#Sanger1-dev:
  1FBLYX_0.json
  24174_5#888.cram
  irods_test_large_QZI
  irods_test_small_jra_tears
jc18@farm3-head4:/tmp$ ils -l irods_test_small_jra_tears
  jc18              0 root;replicate;red;red3;irods-seq-i16-fg         1024 2018-07-20.13:36 & irods_test_small_jra_tears
  jc18              1 root;replicate;green;green2;irods-seq-sr02-ddn-ra08-9-10-11         1024 2018-07-20.13:36 & irods_test_small_jra_tears
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r  "${irods_cwd}/${SMALL_FILE}_tears" > $(mktemp --tmpdir irods_test_XXX)
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r  "${irods_cwd}/${SMALL_FILE}_tears" > kabloom
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r  /seq-dev/home/jc18#Sanger1-dev/irods_test_small_jra_tears > kabloom
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR
jc18@farm3-head4:/tmp$ /software/npg/20180702/bin/tears -f -r  /seq-dev/home/jc18#Sanger1-dev/irods_test_small_jra_tears 
Error: rcGetHostForGet failed with status -1803000:HIERARCHY_ERROR

What is interesting also is that the HIERARCHY_ERROR doesn't appear in any of the IES or IRES logs.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.