Tried to use vtag, vlock and the java client to lock a container/node in cavern but islocked was not set. Was able to lock/unlock using vtag at one time.The following shows the vtag commands when locking/unlock was working. Similar commands no longer works.

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 ivo://cadc.nrc.ca//vospace/core#islocked="true"

Check the node properties. The islocked property is there.

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59
{'creator': 'cadcregtest1',
'date': '2020-04-02T21:02:07.235',
'ispublic': 'false',
'ivo://cadc.nrc.ca//vospace/core#islocked': 'true'}

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 ivo://cadc.nrc.ca//vospace/core#islocked
'true'

Delete the islocked property.

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 ivo://cadc.nrc.ca//vospace/core#islocked=

The islocked property is deleted.

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59
{'creator': 'cadcregtest1',
'date': '2020-04-02T21:02:07.235',
'ispublic': 'false'}

The following is what the client received from cavern using the vlock command.

vlock -d --vos-debug --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 --lock

DEBUG 2020-04-02 14:23:02,391 140600699381568 vos-3.1.0 connectionpool._make_request.437 https://proto.canfar.net:443 "POST /cavern/nodes/home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 HTTP/1.1" 200 760
DEBUG 2020-04-02 14:23:02,391 140600699381568 vos-3.1.0 ws.send.486 alinga-- return response <Response [200]>
DEBUG 2020-04-02 14:23:02,392 140600699381568 vos-3.1.0 vos.update.2562 update response: b'\r\n<vos:node xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" uri="vos://canfar.net~cavern/home/cadcregtest1/atest/locktest/2020-04-02T14-01-59" xsi:type="vos:ContainerNode">\r\n vos:properties\r\n <vos:property uri="ivo://ivoa.net/vospace/core#creator" readOnly="false">cadcregtest1</vos:property>\r\n <vos:property uri="ivo://ivoa.net/vospace/core#date" readOnly="false">2020-04-02T21:02:07.235</vos:property>\r\n <vos:property uri="ivo://ivoa.net/vospace/core#ispublic" readOnly="false">false</vos:property>\r\n <vos:property uri="ivo://cadc.nrc.ca/vospace/core#islocked" readOnly="false">true</vos:property>\r\n </vos:properties>\r\n <vos:nodes />\r\n</vos:node>\r\n'

When the client set the islocked node property (using vlock), I saw the following in the cavern sever log. The last property was the islocked property being set to true. Yet when I used vtag to list the node properties, islocked wass not there, so it was not set.

253543045 cavern [request-proc-13] DEBUG NodeUtil - [nodeToPath] path: vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-02T14-01-59 -> /cephfs/cavern/home/cadcregtest1/atest/locktest/2020-04-02T14-01-59
253543045 cavern [request-proc-13] DEBUG NodeUtil - setNodeProperties: ContainerNode [appData=NodeID[-1], uri=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-02T14-01-59, parent=vos://canfar.net~cavern/home/cadcregtest1/atest/locktest NodeID[-1], properties=[ivo://ivoa.net/vospace/core#creator: cadcregtest1, ivo://ivoa.net/vospace/core#date: 2020-04-02T21:02:07.235, ivo://ivoa.net/vospace/core#ispublic: false, ivo://cadc.nrc.ca/vospace/core#islocked: true]]

vtag --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem vos:home/cadcregtest1/atest/locktest/2020-04-02T14-01-59
{'creator': 'cadcregtest1',
'date': '2020-04-02T21:02:07.235',
'ispublic': 'false'}

I tried using the java client to lock a file, it did not lock. Here is the command I executed:

./cadc-vos java -jar cadcVOSClient.jar --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem --set --target=vos://canfar.net~cavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png --lock

I saw the same debug log on cavern as when the python client was used.

190068570 cavern [request-proc-16] DEBUG NodeUtil - setNodeProperties: DataNode [appData=NodeID[-1], uri=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png, parent=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53 NodeID[-1], properties=[ivo://ivoa.net/vospace/core#creator: cadcregtest1, ivo://ivoa.net/vospace/core#date: 2020-04-05T04:00:06.011, ivo://ivoa.net/vospace/core#length: 497927, ivo://ivoa.net/vospace/core#MD5: 946e13bf8e50a4b6a8ab875d7dac614a, ivo://ivoa.net/vospace/core#type: image/png, ivo://ivoa.net/vospace/core#ispublic: false, ivo://cadc.nrc.ca/vospace/core#islocked: true]]
190068570 cavern [request-proc-16] DEBUG NodeUtil - setAttribute: ivo://ivoa.net/vospace/core#MD5 = 946e13bf8e50a4b6a8ab875d7dac614a
190068572 cavern [request-proc-16] DEBUG NodeUtil - setAttribute: ivo://ivoa.net/vospace/core#type = image/png
190068575 cavern [request-proc-16] DEBUG NodeAction - performNodeAction() elapsed time: 5ms
{"@timestamp":"2020-04-06 03:19:09.882","service":{"name":"cavern"},"thread":{"name":"request-proc-16"},"log":{"level":"info"},"phase":"end","ip":"10.233.114.0","method":"POST","path":"/cavern/nodes/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png","success":true,"duration":90,"user":"cadcregtest1"}

However the node was not locked.

./cadc-vos java -jar cadcVOSClient.jar --cert=/usr/cadc/dev/admin/test-certificates/x509_CADCRegtest1.pem --view --target=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png
data: vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png
creator: cadcregtest1
is locked:
last modified: 2020-04-05T04:00:06.011
readable by anyone: false
readable by:
readable and writable by:
size: 497927
type: image/png
encoding:
md5sum: 946e13bf8e50a4b6a8ab875d7dac614a

Here's a debug log on cavern. islocked was not present.

190430442 cavern [request-proc-11] DEBUG FileSystemNodePersistence - [get] DataNode [appData=NodeID[-1], uri=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53/something.png, parent=vos://canfar.netcavern/home/cadcregtest1/atest/locktest/2020-04-04T20-59-53 NodeID[-1], properties=[ivo://ivoa.net/vospace/core#creator: cadcregtest1, ivo://ivoa.net/vospace/core#date: 2020-04-05T04:00:06.011, ivo://ivoa.net/vospace/core#length: 497927, ivo://ivoa.net/vospace/core#MD5: 946e13bf8e50a4b6a8ab875d7dac614a, ivo://ivoa.net/vospace/core#type: image/png, ivo://ivoa.net/vospace/core#ispublic: false]]

Class LocalServiceURI.java reads the 'local' service URI from the classpath, so this configuration has to be made at built time. Instead, it should read from a properties file in the config directory at runtime.

The optimization for empty files is to create just the node but the server does not return the corresponding md5 sum in that case (d41d8cd98f00b204e9800998ecf8427e). This creates problems in vos which relies on the checksum to be present all the time.

Patch, when available, should be tested with vos before release.

In version 2.1, an optimized synctrans (download only) scenario was introduced where the transfer parameters (protocol, direction, target) are provided as URL query parameters. The response to this request is a completed transfer document with the download endpoints (URLs) included. If, however, the parameter "request=redirect" is also provided, then the service must redirect the user to the a preferred download endpoint.

Currently the service always redirects the user to the preferred download URL. It should be changed so that it only redirects when the request=redirect parameter is supplied.

Currently, one must restart the servlet container for a change in the vospace root (defined in ~/config/Cavern.properties) to take effect. It should take effect immediately.

If a user attempts to create a folder or upload a file into a VOSpace to which they are not authorized to do so, the client will throw an IllegalStateException and a 500 Server Error, which is inappropriate.

The server logs from the service that the client was connecting to correctly report the unauthorized error:
PermissionDenied (401): Write permission denied on ...

But the client interprets it as an IllegalStateException.

In the series of catch statements (line 290+) the error message string used to sendError is just the base required message and discards additional detail from the exception. For example, the AccessControlException message says which node triggered the permission failure, which is required when trying to delete or move a container tree and the lack of write permission is somewhere in the tree.

Probably most of these errors could have the message from the exception suitably appended.

This issue is to note that preliminary tests from Java and Python clients show that link nodes aren't working in cavern. More followup and investigation is needed.

I have been storing a large volume of Gemini processing results in /cavern/projects/Gemini/reductions. There are roughly 6000 sub-directories in this directory. At time of writing, to list the contents of this directory, it is taking 34 minutes:

time ls /cavern/projects/Gemini/reductions/

......

real    34m51.565s
user    0m0.010s
sys     0m0.006s

Service returns NodeNotFound when client tries to access a resource URI that does not resolve. A possible cause for this is that the path in the resource URI contains an external link, i.e., vos://cadc.nrc.ca~vault/user/a/b/external_link/c/d/foo.txt
If client is informed that the NodeNotFound was caused by the vos://cadc.nrc.ca~vault/user/a/b/external_link being an external link that cannot be resolved, it could try to resolve it itself as it currently does with the leaf links.

Best it would be to return both the external link and the remaining/unresolved path (as one link?) so that the client gets the right information to try to identify and access the linked resource.

Low priority bug / feature request:

I created a script using Julia's watch_folder function to trigger an action when a file is added to a given directory. Testing using linux commands on the science platform e.g. tough text.txt triggers events, but transferring files using vcp does not. The files appear immediately using ls but no event is generated.

If the reference to a file is through a link to a container then the delete operation does not work correctly. Like 'GetNodeAction', the 'DeleteNodeAction' needs to resolve the path to the real node before attempting any operations.

The setRequestCookie(HttpConnection) call is missing because it is not consistent with the other Http[PutPostGet] classes.

We started with the vault implementation where it didn't make sense to have it but in arc does. I should add it to arc but implement it as a property in vault as well so the flag is always preserved on the round trips (local->hospice->local). (Reference: opencadc/vostools#212)

The "ivo://cadc.nrc.ca/vospace" string must be configurable. I do not find a StandardID which could be used for this purpose. It is not defined in Standards.java file nor published in vospace capabilities file.

Files involved:

** vos/cadc-vos-server/src/main/java/ca/nrc/cadc/vos/server/transfers/TransferRunner.java
this.serviceURI = new URI("ivo://cadc.nrc.ca/vospace");

** vos/cadc-vos-server/src/main/java/ca/nrc/cadc/vos/server/transfers/TransferUtil.java
public static final String VOSPACE_RESOURCE_ID = "ivo://cadc.nrc.ca/vospace";

** vos/cadc-vos-server/src/main/java/ca/nrc/cadc/vos/server/transfers/VOSpaceTransfer.java
this.serviceURI = new URI("ivo://cadc.nrc.ca/vospace");

** vosui/src/main/java/ca/nrc/cadc/beacon/AbstractStorageItemProducer.java
new VOSpaceClient(URI.create("ivo://cadc.nrc.ca/vospace"));

** vosui/src/main/java/ca/nrc/cadc/beacon/web/resources/SecureServerResource.java
URI.create("ivo://cadc.nrc.ca/vospace");

** vosui/src/main/java/ca/nrc/cadc/beacon/web/resources/StorageServerResource.java
URI.create("ivo://cadc.nrc.ca/vospace");

I am part of the SDP group from ICRAR, headed by Andreas Wicenec, and I would like to review your VOSpace 2.1 reference implementation but I am having issues running it. Is there an instruction manual or some other piece of documentation that can guide me through the steps of building, installing and operating this software?
Thank you.

Setting read or read/write groups on a Cavern node adds the group, rather than using what is represented in the XML.

• In an existing writeable folder in Arc (https://www.canfar.net/storage/arc/list/), create a new folder
• Set the read group on it:

$ curl -Lv -H "content-type: text/xml" -d '<?xml version="1.0" encoding="UTF-8"?>
<vos:node xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" uri="vos://cadc.nrc.ca~arc<path_to_created_folder>" xsi:type="vos:ContainerNode">
 <vos:properties>
   <vos:property uri="ivo://ivoa.net/vospace/core#creator" readOnly="false"><your_username></vos:property>
   <vos:property uri="ivo://ivoa.net/vospace/core#date" readOnly="false">2023-04-06T23:05:06.592</vos:property>
   <vos:property uri="ivo://ivoa.net/vospace/core#groupmask" readOnly="false">rwx</vos:property>
   <vos:property uri="ivo://ivoa.net/vospace/core#groupread" readOnly="false">ivo://cadc.nrc.ca/gms#<group_name_1></vos:property>
   <vos:property uri="ivo://ivoa.net/vospace/core#groupmask" readOnly="false">rwx</vos:property>
   <vos:property uri="ivo://ivoa.net/vospace/core#ispublic" readOnly="false">false</vos:property>
 </vos:properties>
 <vos:nodes />
</vos:node>' https://ws-uv.canfar.net/arc/nodes/<path_to_created_folder>

This will set <group_name_1> on the node as expected.

• Update the read group:

<vos:node xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" uri="vos://cadc.nrc.ca~arc<path_to_created_folder>" xsi:type="vos:ContainerNode">
  <vos:properties>
    <vos:property uri="ivo://ivoa.net/vospace/core#creator" readOnly="false"><your_username></vos:property>
    <vos:property uri="ivo://ivoa.net/vospace/core#date" readOnly="false">2023-04-06T23:05:06.592</vos:property>
    <vos:property uri="ivo://ivoa.net/vospace/core#groupmask" readOnly="false">rwx</vos:property>
    <vos:property uri="ivo://ivoa.net/vospace/core#groupread" readOnly="false">ivo://cadc.nrc.ca/gms#<group_name_2></vos:property>
    <vos:property uri="ivo://ivoa.net/vospace/core#groupmask" readOnly="false">rwx</vos:property>
    <vos:property uri="ivo://ivoa.net/vospace/core#ispublic" readOnly="false">false</vos:property>
  </vos:properties>
  <vos:nodes />
</vos:node>' https://ws-uv.canfar.net/arc/nodes/<path_to_created_folder>

The returned XML will only contain <group_name_2> as the read group, but looking at the ACLs:

$ getfacl /cephfs/cavern<path_to_created_folder>
...
group:<group_name_1>:r-x
group:<group_name_2>:r-x
...

The cavern resource id is currently required to be set in two places:

• web.xml (in src/main/webapp/WEB-INF)
• LocalServiceURI.properties (in src/main/resources)

This should be consolidated to a single location outside of the build. It needs to be part of runtime configuration (rather than embedded in the war file) so the same image can be deployed at different locations with different serviceIDs.

Additionally, the integration tests refer to the serviceID in many locations. That needs to be cleaned up.

For example, setting the read-group on a node and all sub-nodes.

Running a transfer with username/password produces an /xfer transfer result but with the wrong authentication type. The transfer result should be sent to /auth/xfer rather than /xfer

Using cadc-vos/VOSpaceClient, once the #groupwrite attribute is set on a node, #groupread is no longer available, and can't be set/changed. Attempts to do so get a 200 return code.

Once scenario to try to see this is based on a functional browser integration test, when run against vault. :

• set groupread
• set groupwrite (both groupread and groupwrite values should be returned as part of node properties, but only groupwrite is.)
• change groupread (200 returned, no change to property set)

If a transfer is re-tried after a failure, the job update will fail because the job is already in the ERROR state, so cannot do the transition from EXECUTING to COMPLETE | ERROR on the second try.

The transfer service needs to be smarter about this somehow. The correct solution would probably be to consider the jobs as complete when the transfer document was successfully created and remove the byte transfer from the scope of the job altogether. This way /transfer wouldn't have to make any callback for job updating.

When the client initiates a transfer with the CADC token or when it does not specify security methods server does not return the expected protocol and end points. Instead, it returns a document of the form:

<?xml version="1.0" encoding="UTF-8"?>
<vos:transfer xmlns:vos="http://www.ivoa.net/xml/VOSpace/v2.0" version="2.1">
  <vos:target>vos://cadc.nrc.ca!vault/some/path/file.txt</vos:target>
  <vos:direction>pushToVoSpace</vos:direction>
  <vos:keepBytes>true</vos:keepBytes>
</vos:transfer>

This problem is specific to v. 2.1 of VOSpace.

When using detail=min for a Cavern request (which omits properties), the NodeReader is failing in the VOSpaceClient as there is no empty <properties></properties> tag.

The getOwners() method in NodeDAO assumes that the IdentityManager returns the owner object as an Integer. This will break when using any other IdentityManger when making a getNode call with detail=raw.

Instead, the IdentityManager class should provide a way get an owner's subject without having to augment that subject.

Provide a command line interface to the DatabaseNodePersistence method chown().

The VOSpaceAuthorizer should throw an AccessControlException rather than an IllegalStateException when it finds that the system is in read-only or offline mode.

Support POSIX compliant filenames in the arc VOSpace service. Tools such as vcp and the web portal fail if they encounter files will non supported characters, such as spaces.

The CRED_SERVICE_ID in cadc-vos-server/src/main/java/ca/nrc/cadc/vos/server/auth/VOSpaceAuthorizer should be configurable

Proposed patch

Move CRED_SERVICE_ID in constructor and use localAuthority.getServiceURI:

public class VOSpaceAuthorizer implements Authorizer
{
protected static final Logger LOG = Logger.getLogger(VOSpaceAuthorizer.class);

// TODO: make these configurable or find from the VOSpace capabilities
//private static final String CRED_SERVICE_ID = "ivo://cadc.nrc.ca/cred";
private static String CRED_SERVICE_ID;


 public VOSpaceAuthorizer()
{
    this(false);

    LocalAuthority localAuthority = new LocalAuthority();
    URI serviceURI = localAuthority.getServiceURI(Standards.CRED_DELEGATE_10.toString());
    CRED_SERVICE_ID = serviceURI.toString();

}

public VOSpaceAuthorizer(boolean allowPartialPaths)
{
    initState();
    this.allowPartialPaths = allowPartialPaths;

    LocalAuthority localAuthority = new LocalAuthority();
    URI serviceURI = localAuthority.getServiceURI(Standards.CRED_DELEGATE_10.toString());
    CRED_SERVICE_ID = serviceURI.toString();
}

Currently, a software optimization checks if the posted attributes against the existing ones and issues the update (and implicitly the update of the last_mod) only when there is at least one change. For some user actions, such as the touch command, posted attributes are the same with the existing ones however the last_mod time is expected to change.

See opencadc/vostools#206

Sorting a container node's children by size (content-length) causes casting exception:

Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT TOP 500 nodeID,parentID,name,type,busyState,isPublic,isLocked,ownerID,creatorID,groupRead,groupWrite,lastModified,contentType,contentEncoding,link,storageID,contentLength,contentMD5 FROM vospace2.dbo.Node WHERE parentID = 14379 AND contentLength <= ? ORDER BY contentLength DESC]; nested exception is java.sql.SQLException: Implicit conversion from datatype 'VARCHAR' to 'BIGINT' is not allowed.  Use the CONVERT function to run this query.

        at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:97)
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:607)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:641)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:670)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:678)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:710)
        at ca.nrc.cadc.vos.server.NodeDAO.getChildren(NodeDAO.java:640)
        ... 70 more
Caused by: java.sql.SQLException: Implicit conversion from datatype 'VARCHAR' to 'BIGINT' is not allowed.  Use the CONVERT function to run this query.

It will be very useful to have a feature like vdu equivalent of posix's du command for /arc. This will allow users to monitor their disk usage for something like /arc/projects/chime_frb or /arc/home/$USER/.

When transferring larger files (e.g. 9GB) to arc, if the transfer fails, the file just gets deleted. Should the client be capable of resuming an interrupted transfer?