exlibrisgroup / almaremotestorageapp Goto Github PK

It appears that the pom.xml file was not updated and a dependency is missing. Could you please update?

Here is a portion of the error messages:
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[15,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[16,35] package com.jcraft.jsch.ChannelSftp does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[17,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[18,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[19,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[20,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[21,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[22,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[27,17] cannot find symbol
symbol: class ChannelSftp
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[29,16] cannot find symbol
symbol: class ChannelSftp
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[30,32] cannot find symbol
symbol: class JSchException
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[225,32] cannot find symbol
symbol: class LsEntry
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[53,51] cannot find symbol
symbol: class UserInfo
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[53,61] cannot find symbol
symbol: class UIKeyboardInteractive
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[33,25] cannot find symbol
symbol: class JSch
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[33,41] cannot find symbol
symbol: class JSch
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[34,25] cannot find symbol
symbol: class Session
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[45,40] cannot find symbol
symbol: class ChannelSftp
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[47,26] cannot find symbol
symbol: class JSchException
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[132,50] cannot find symbol
symbol: class LsEntry
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[132,30] cannot find symbol
symbol: class LsEntry
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[191,62] cannot find symbol
symbol: variable ChannelSftp
location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[233,17] cannot find symbol
symbol: class SftpATTRS
location: class com.exlibris.ftp.SFTPUtil
[INFO] 23 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.446 s
[INFO] Finished at: 2019-11-25T08:30:06-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project AlmaRemoteStorageApp: Compilation failure: Compilation failure:
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[15,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[16,35] package com.jcraft.jsch.ChannelSftp does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[17,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[18,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[19,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[20,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[21,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[22,23] package com.jcraft.jsch does not exist
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[27,17] cannot find symbol
[ERROR] symbol: class ChannelSftp
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[29,16] cannot find symbol
[ERROR] symbol: class ChannelSftp
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[30,32] cannot find symbol
[ERROR] symbol: class JSchException
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[225,32] cannot find symbol
[ERROR] symbol: class LsEntry
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[53,51] cannot find symbol
[ERROR] symbol: class UserInfo
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[53,61] cannot find symbol
[ERROR] symbol: class UIKeyboardInteractive
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[33,25] cannot find symbol
[ERROR] symbol: class JSch
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[33,41] cannot find symbol
[ERROR] symbol: class JSch
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[34,25] cannot find symbol
[ERROR] symbol: class Session
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[45,40] cannot find symbol
[ERROR] symbol: class ChannelSftp
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[47,26] cannot find symbol
[ERROR] symbol: class JSchException
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[132,50] cannot find symbol
[ERROR] symbol: class LsEntry
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[132,30] cannot find symbol
[ERROR] symbol: class LsEntry
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[191,62] cannot find symbol
[ERROR] symbol: variable ChannelSftp
[ERROR] location: class com.exlibris.ftp.SFTPUtil
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/ftp/SFTPUtil.java:[233,17] cannot find symbol
[ERROR] symbol: class SftpATTRS
[ERROR] location: class com.exlibris.ftp.SFTPUtil

Thanks,
Kathy

Hi,

I was wondering if/when Ex Libris would update the pom.xml file to keep it up to date with the latest dependencies.

thanks,
Kathy

Hi,

It looks like the signature validation fails. I have the "webhook_secret" set in conf.json and this is the same secret as configured in the Integration Profile. Here is the message from the logs:

2019-11-25 17:13:45,386 http-nio-8080-exec-18id INFO WebhookServlet 78: un validate signature

Could you please assist in trouble shooting?

I did read in a post that webhooks are sent with:
"Content-Type: application/json;charset=UTF-8
rather than
Content-Type: application/json"
(https://sbprojects.statsbiblioteket.dk/stash/projects/SWHL/repos/web-hooker/commits/e5c0855e1fe03f476cae13aa3edc7fbeca09af92)

It appears that the request is coming with "content-type = application/json".

thanks,
Kathy

Hi,

I do not see where the AlmaRemoteStorageApp webhook code validates the post request signature. Shouldn't a shared secret used? If so, where would it be configured for the AlmaRemoteStorageApp? In conf.json?

thanks,
Kathy

I configured a Remote Storage Integration Profile named "Shared Collections Facility" in each IZ.

please confirm:

Is the name of the job (from the jobs API call) that I want to get the "publishing_job_id" from named "Inventory Remote Storage Update - Shared Collections Facility"?

Is the name of the job to get the "requests_job_id" named "Send Requests to Remote Storage"?

why I'm asking (tl;dr):

conf.json has a space for two job IDs for each IZ:

    "requests_job_id":"S123456789",
    "publishing_job_id":"S987654321",

The instructions say "Find the job ID [singular] ... use this API ... search for the integration profile name and get the id."

I got the /almaws/v1/conf/jobs for each IZ. The only place I can find the name of the Remote Storage integration profile is in the name & desc for a "Repository" job:

    "name": "Inventory Remote Storage Update - Shared Collections Facility",

So I assume that is the publishing job id.

I also found a "Fulfillment" job about "Remote Storage":

    "name": "Send Requests to Remote Storage"

That must be the requests job id, even tho it doesn't have "Shared Collections Facility" in it?

If we are going to host the remote storage app locally, what is the minimum access we need to configure in our firewall to allow the app to work? Is it http/s access from the North American IP ranges listed on https://knowledge.exlibrisgroup.com/Alma/Implementation_and_Migration/Implementation_Guides/03Technical_Requirements_for_Alma_and_Discovery_Implementation#Alma_Platform_IP_Range or is there more access than that required?

Note that we are going to use an existing SFTP server that is on the same subnet as this new app server and the SFTP server already allows SSH access from those NA Alma Platform IP Ranges. Also, if Ex Libris needs SSH access and an account to assist us in installing, configuring and monitoring this application we are happy to provide that as we did routinely in the Voyager days.

thanks, Don

Specifically where should the mainLocalFolder be located? (The code is "static String mainLocalFolder = "files//items//";" Should there be a folder for each IZ?

It looks like the code is searching the 'OLD' folder on the sftp server instead of stopping at items. Should this be removed? It looks like it was created by the scfapp.

2019-11-26 12:58:39,105 Thread-5id INFO SFTPUtil 128: get files from ftp folder:/AlmaRSA/01WRLC_AMU/items/OLD/ local folder : files//items//targz//

"static String mainLocalFolder" is used in both com.exlibris.request.RequestsMain.java and in com.exlibris.items.ItemsMain.java. These string do not provide a path to the locations of the folders.

Perhaps if you add a "mainLocalPath" in the conf.json and concatenated mainLocalPath + mainLocalFolder files would be put in the proper place. Otherwise the app has no way of knowing where the mainLocalFolder should be. Currently we are having the files placed under /opt/local/scf/files.

Please let me know if you have any questions.

thanks,
Kathy

It looks like there might be a problem with either the API call or the api_key. Can you please assist with trouble shooting this?

Below is an excerpt from the application.log file:
16:33:10,253 Thread-5id DEBUG ItemsMain 69: get network system number
2019-11-26 16:33:10,255 Thread-5id INFO ItemsHandler 15: New/Update Item. Barcode: 31194009275128
2019-11-26 16:33:10,256 Thread-5id DEBUG SCFUtil 81: get SCF Item. Barcode : 31194009275128
2019-11-26 16:33:10,257 Thread-5id INFO AlmaRestUtil 11: Starting to handle retrieve Item: 31194009275128X.
2019-11-26 16:33:10,257 Thread-5id INFO AlmaRestUtil 12: Item barcode: 31194009275128X - calling GET
2019-11-26 16:33:10,258 Thread-5id INFO AlmaRestUtil 21: Sending GET request to URL : https://api-na.hosted.exlibrisgroup.com/almaws/v1/items?item_barcode=31194009275128X&apikey=notOnLog
2019-11-26 16:33:11,468 Thread-5id INFO AlmaRestUtil 44: Response Code : 400
2019-11-26 16:33:11,469 Thread-5id INFO AlmaRestUtil 70: message: No items found for barcode 31194009275128X.
2019-11-26 16:33:11,470 Thread-5id DEBUG SCFUtil 88: No items found . Barcode : 31194009275128
2019-11-26 16:33:11,470 Thread-5id DEBUG ItemsHandler 19: The item does not exist in the remote Storage

Please let me know if you need further information.
thanks,
Kathy

I was not able to build app due to mismatch of arguments between SCFUtil and ItemApi.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project AlmaRemoteStorageApp: Compilation failure
[ERROR] /opt/local/scf/AlmaRemoteStorageApp/src/main/java/com/exlibris/util/SCFUtil.java:[368,44] method scanIn in class com.exlibris.restapis.ItemApi cannot be applied to given types;
[ERROR] required: java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String
[ERROR] found: java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String
[ERROR] reason: actual and formal argument lists differ in length

In the example conf.json file there is a list of institutions, the first being 01AAA_RS with lib code RS and location rs_shared, and no *_job_id parameters. This duplicates the example values at the beginning of the file for remote_storage_* parameters, which we are specifying for the WRLC Shared Collections Facility (SCF).

Do we repeat that information for the SCF in the institutions list?

I am starting to install software for the server that will host the remote storage app. I have several questions on versions of software.

What version of Tomcat do you recommend? I notice that your configuration specifies webapp-runner version 8.0.30.2. (See below)

I believe that webapp-runner versions are set to map to the tomcat version that it relies on. Tomcat 8.0.x has been superseded and the tomcat8.0 branch of webapp-runner is deprecated. Is there any specific reason we would need to use Tomcat 8.0?

Also, what version of java have you been using? Java * or a later version? If it is 8 will install an earlier version of maven.

thanks,
Kathy