Review

• Development status: https://cernbox.cern.ch/index.php/apps/files/?dir=/__myprojects/digital-repositories/Services_Projects/CDS/Books&
• Issues: https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%206

Records

• Edit and create operations for Locations, Documents and Items

Migrator kit

• iteration on the rules

Q&A:

• should the keywords on the document be extendable (should we provide autocompletion in record editor, or should it be static list)?
• what happens when the item is to be picked up in the library and the librarian handles the pending request? (ITEM_AT_DESK) (we lose the opportunity to take the book back into the available)
• E-book: _access field, is it just a label? Medium: CD-ROM, etc... does it mean that they cannot circulate?

If decide to restrict records by CCID, the current code for record restrictions is wrong because it tries to match int values to Invenio user id.
See: https://github.com/inveniosoftware/invenio-app-ils/blob/master/invenio_app_ils/records/permissions.py#L81

Since we are deploying on OpenShift, we need solid infrastructure monitoring:

Me: Is there a way to get alerts when a POD is "down" for any reason?

OpenShift: no, nothing out of the box. You would have to take care of this yourself, e.g. pushing the alerts to central monitoring or deploy your own promethus instance

See: https://discourse.web.cern.ch/t/how-to-integrate-openshift-metrics-in-our-own-prometheus-server/375

On the medium term, Openshift 4.0 will come with a completely revamped monitoring system that should make all this a lot easier. We’re expecting this sometime in Q1 2019.

• create Piwik
• Investigate how to have alerts/central monitoring send logs
• Request a new instance to monitoring to collect all application logs, e.g. INFO in indexer: {"action": "before_indexing", "name": "referenced_records_indexer", "origin": {"pid_type": "pitmid", "pid_value": "e1m4q-gnf08"}, "referenced": {"pid_type": "docid", "pid_value": "dhyh5-1qn47"}, "uuid": "0f15f84f-ee86-435f-8359-d6a61b94d867"} all logs from any POD in OpenShift are already sent to central monitoring, we can request access if needed
• Flower for celery
• Rabbit MQ
• HaProxy Stats dashboard

Similar setup to #10 . Differences:

• ES On Demand
• DB On Demand

Based on the requirements documentation prepare a test script (list of test cases) to verify that the final system delivers the agreed functionality.

Possible additional way to use the query builder helper for the homepage:

See gif:

Integrate CERN Piwik.
How does it work with a React SPA?
What about invenio-stats?
Do we need anything more? Ask library

When ready for production, perform a stress test and document (as we did for Videos).
How much max traffic do we target?
How much our deployment config can take?
Does OpenShift scale up nicely when under stress?

Proposed by Library - possible as a feature for future CDS

We need to indicate the delivery method per loan request - to pick up or to be sent by internal post,
the delivery method has to be highlighted when checkout in the backoffice

• Add icon in the loans search page
• Add icon in the loans details page
• Add line and icon in notification when a loan is checked out

https://codimd.web.cern.ch/0p-ZhZDnTAycgWGNULBXBQ#UC-See-%E2%80%9Chelp%E2%80%9D--%E2%80%9Cinteresting-links%E2%80%9D-pages

requirements in here replaced by https://github.com/CERNDocumentServer/cds-books/files/4568026/ILS-homepage.pdf in #101

Here are the remaining actions that you can already do in the current version before the migration, if this is ok for you:

1. Please could you delete all the items of the records listed in tobedeleted.csv

2. Please change the items in the list scanning.csv to “scanning”, the status doesn’t exist yet, but it would be great if you could create it in the current version already.

3. Change all the "untraceable" to “missing”

4. Delete all the request with dates in the past except:
   tmp-14478436
   waiting 27/01/2013 00:00
   26/07/2013 00:00 27/01/2013 10:21

Now, only the following statuses should remain, all the others have been cleaned:

• On shelf
• Missing
• On order
• Not published
• Out of print
• Under review
• Scanning
• In binding

csv.zip

These are the last designs for books. Any feedback is welcomed.
Book_DetailedView.pdf
Book_Homepage.pdf
Book_Search.pdf

Comes from CERNDocumentServer/cds-videos#507

CERNDocumentServer/cds-videos#507 (comment)

In general the new UI looks very nice. Just two tiny comments usability which there may or may be not be a good reason for: 1) I find the two search fields (top bar + facets) confusing in search-2.pdf and 2) The "Chapters in this book" and "I want to purchase a personal copy of this book" I wouldn't expect to find in the lower bottom corner.

CERNDocumentServer/cds-videos#507 (comment)

Thanks for the feedback! I have just updated the designs (for some reason the initial upload contained the first iteration).
Comment 1): a few people complained, we will be removing the search in the facets.
Comment 2): it should already be taken into account in the new designs that I have uploaded.

CERNDocumentServer/cds-videos#507 (comment)

This design is really nice.
A few comments from my side:

1. Book_detailedview:

• Some information need to be displayed together with the title and the authors: Edition, year and publisher. The rest can go under "Information".
• The link "Purchase a copy" where it is located might be misleading for the users. It should be "Purchase a personal copy" and should probably be in a separate box.

2. Book_Search:

• When there is no print copy available or all copy are on loan, it should be made clear for the users that they can request it already at this level. If they do not go on the detailed view, they do not see this option.
• The facet should include other criteria: year, publisher, availability and medium.

3. Book_Homepage:

• We should find somewhere the links we sent you: Check your loans, Access the CERN Library electronic resources from outside CERN, Request a document from another Library, Suggest a book for the Library collection, Buy a document for your personal collection, Library website, Library contact
  (Also for Book_Search)
• We will suggest another text than "Wellcome to the CERN Library" and another image (those are periodicals and not books :-)
• I find the box in the middle with the suggested title a bit big compared to the search box

Investigate if this service is still needed. If yes, investigate if our REST endpoint can be used or if we need a custom implementation.
The current code is in our internal repo, in: aleph/aleph_webinterface.py

Review

• Issues:
  • https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%2011
• Timeline: https://cernbox.cern.ch/index.php/s/ATMZVY65kdYC7R5

UI

• Frontsite: homepage first implementation
• Electronic items: backend implementation
• Deletion of records
• Technical tasks

Migration

• Volumes/Series

Q&A

• We are still trying to understand how to model correctly Volumes.

Review

• Issues:
  • https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%209
• Timeline: https://cernbox.cern.ch/index.php/s/sWuZHrXKHkC7Hfc

UI

• Force checkout for the user

• added query builder helper

• added facets for the documents (document type and language)
• filtering by available items

Migrator kit

• Migration rule for the related links: CERNDocumentServer/cds-dojson#200

Q&A

Request and set-up the ElasticSearch on Demand instances required (prod/qa)

Review

• Development status: https://cernbox.cern.ch/index.php/apps/files/?dir=/__myprojects/digital-repositories/Services_Projects/CDS/Books&
• Issues: https://waffle.io/CERNDocumentServer/cds-books?label=books-sprint4

Loans

• search ui with sample facets, sorting, pagination
• details page: details of the loan, item preview when attached to it, item replace
• create a request of a new loan for an hardcoded user

Items

• search ui with sample facets, sorting, pagination
• details page: details of the item, pending loans, loans history

Documents

• search
• details, by url

Migrator kit

• iteration on the rules

Here more details on how to handle the e-books:
https://its.cern.ch/jira/browse/CDS-1126
( refers to CERNDocumentServer/cds-dojson#178)

Here a relevant SNOW ticket about EZProxy:

• https://cern.service-now.com/nav_to.do?uri=u_request_fulfillment.do?sys_id=96b5b6a04f6e4740b9cc09de0310c76a%26sysparm_view=tabs_new
• http://library.cern/resources/remote

Documentation from EbookCentral: https://support.proquest.com/articledetail?id=kA11W000000XbrxSAC

( requires CERNDocumentServer/cds-dojson#201 )

Entering a query in the homepage searchbar and clicking the search button or pressing enter doesn't redirect to the search page

Since the look and feel is the same for PROD/QA/Dev, user (and us) is not sure in which instance is working on.
Change the logo/highlight in some way that QA/Dev are not prod.

• Dev instance
  • Request the creation of the website. Note the responsible and the name.
  • Fork and work on https://gitlab.cern.ch/invenio/template-openshift.
  • Deploy current repo in the dev instance.

Analyse what needs to be done to implement an importer of books. No UI implementation, focus on the the logic and backend.

• Library input
• Books Project requirements

Draft requirements & questions

Providers

The library would like to know how many loans per department. We need to store a "snapshot" of info in the loan data model because users can change department.

Create a dedicated instance in Sentry and integrate

• A task should be created to import ldap users to our system so can be in sync.
  • Clean up task for checking and removing users that they are not longer in CERN users.
  • Reindex all users to elasticsearch

inveniosoftware/invenio-oauthclient#178

Don't forget them

Slightly updated requirements regarding facets coming from @agentilb and Tullio:

The list of facet should indeed include:

• medium (e-books/ hard copy)
• type (book / preprint/…)
• language
• authors
• immediate availability (books available online and or not on loan)
• year of publication
• library (for the books available in paper): so people would be able to search for books in Central Library only.

For “topic”, can we leave it for the time being, it requires a huge cleaning on our side, is it something we could add at a later stage?

For the sorting, we see:

• year of publication
• title
• circulation information: which is actually popularity, and beside information of circulation we could add usage statistics of ebooks, that we can provide at a regular basis. (Note that since there will be some books without paper or electronic copies, so with no “popularity”, but I guess there is no big deal, they’ll appear at the bottom of the list.
• relevance: this is a tricky one… here we see that the results should be sorted according the closeness between the query and title/author fields. But I guess this is something to be discussed together with the search strategy.

To Be Discussed when time comes.

Item
Add document_pid and internal_location_pid as mandatory.

Internal Location
Add location_pid as mandatory.

Fix breaking code. :)

By default, in OpenShift we will have a domain with the .web certificate.
See instructions here on how to get rid of it.
https://discourse.web.cern.ch/t/remove-subdomain-web-from-mysite-web-cern-ch-to-have-mysite-cern-ch/528

The domain should be: catalogue.library.cern

• Use EzProxy in the detail view to allow access from outside CERN

It looks like that to be able to use Amazon covers we need to comply with Amazon TOS, which in a few points (to be reviewed!) means:

• cannot download images
• cannot cache images
• cannot manipulate the URL
• can cache/save the URL to an image for max 24 hours
• we can store the ASIN (Amazon identifier) which is useful to fetch the book with Amazon API

Reference: (check point VIII):
FR: https://partenaires.amazon.fr/help/operating/license
IT: https://programma-affiliazione.amazon.it/help/operating/license
Same page in .com is blank: https://affiliate-program.amazon.com/help/operating/license but reading around, it has the same policy.

How fetching works now in CDS

1. Search using API by ISBN: http://webservices.amazon.fr/onca/xml?SearchIndex=Books&Power=ISBN%3A9780486688954...
2. Normally there is only 1 results, parse it and fetch the images urls

Amazon API documentation

Extra

It might be wise to create a dedicated microservice for this.

A tool to fetch book cover from multiple sources (as inspiration):
https://github.com/fredericd/coce

When restricting a document, it would be nice to propose autocomplete suggestions with e-groups or users found in LDAP.
We should ask if there is a set of e-groups that should be pre-filled when restricting a record, as done in Videos.

relates to inveniosoftware/invenio-app-ils#242

The library would like to use their own schema for the book items instead of MARC21 as it is too complex and at the same time it doesn’t cover fully their needs.

20170127-holdings-JSON.xlsx

Before doing the migration we should clean a bit the current data , #2

Comes from CERNDocumentServer/cds-videos#405

Analyse what needs to be done to implement an importer of books.

https://codimd.web.cern.ch/0p-ZhZDnTAycgWGNULBXBQ#UC

Things to do:

• have a CLI command to delete/anonymize a given user (right to be forgotten)
• ensure logs are kept for the already decided amount of time
• if easy, from the profile page, allow users to download their own data (using the same queries implemented for EDH)

Decide which style we want to follow for the shake of consistency, if you cannot spot the difference, ask @kprzerwa

Review

• Timeline: https://cernbox.cern.ch/index.php/s/k7b2tOghzz8xC9y
• Issues: https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%205

Dashboard

Completed

Documents

• Search
• Details

Locations

See list of locations

Editor

Close to the end

Q&A

• DASHBOARD

  1. the entry List of items with more than one request makes more sense in form List of items with number of loan requests greater than number of available items. implemented

  2. the entry List of most loaned books on a specific period is a statistic dashboard functionality, not operational. The operational dashboard (home-page of the backoffice) will contain a hub to perform actions not to read statistics. to clarify that this is more a query/stats and not part of an operational dashboard

• How do you create multiple loans for the same patron in the current system?

• ITEM STATUS

  • we would like to change the name of the status of the item when it is not missing/on-binding etc. Do you agree to use the name USABLE? We have already discussed that available, on shelf or loanable are very misleading.

• ITEMS/E-BOOKS

  • we would like to create a new specific record for them, completely independent of items, and at the Document level have a reference to the each ebook information. (update CERNDocumentServer/cds-dojson#193)

https://codimd.web.cern.ch/0p-ZhZDnTAycgWGNULBXBQ#UC-Librarian-can-see-all-current-loans-requests-and-expired-loans-entire-loans%E2%80%99-history

• Locations: block creation and deletions
• Circulation transitions: remove "In transit"

To be completed.

SEO

• all links should have valid href
• each page should have the correct title and description: see React doc, they suggest to use React Helmet
• add opening hours to be visible in the card in the test results

Optimisations

• What happens on slow connection when reaching the homepage and while downloading the React Js? Test UI with no cache and throttling and optimize the index.html that is rendered before JS
• test with Google Mobile Friendly Test
• test with Google Search Console
• check performance with Lighthouse

Google Rich Results, interesting for us?

Accessibility

• run a test
• is color contrast good enough?

Localization

• the meta attributes should indicate that the website's language is english
• pages should be displayed in the same way regardless of the browser's language setting. In particular verify that dates are displayed consistently

Review

• Issues: https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%207

UI

• Human readable errors
• Backoffice:
  • documents facets and search bar fixes
  • multiple loans checkout for a user
• Login redirection when user is logout

Migrator kit

Q&A:

• is it possible that we have document with 2 items, one book and other medium, and the user wants to request one of the specific item?

Review

• Issues:
  • https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%2010

UI

• Frontsite: search, details page first implementation
• Technical tasks

Migrator kit

• Reviewed data chunks to find not addressed in rules for fields - lot of fields found
• fixed rules for collections and document_types

Q&A

Review

• Issues:
  • https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%207
  • https://waffle.io/CERNDocumentServer/cds-books?milestone=Sprint%208
• Timeline: https://cernbox.cern.ch/index.php/s/2MA9evs8JgxNVUA

UI

• Multiple loans checkout for a user
• Human readable error handling and notifications
• Keywords: document details page, filter

Migrator kit

• Implemented rule for field 695: CERNDocumentServer/cds-dojson#198
• Partially implemented rule 8564: CERNDocumentServer/cds-dojson#200

Q&A

• Validate feature: Librarian creates loan request on behalf of an user
  • find the document first (search by title, etc)
  • document details page: click on a button "Request a loan for an user"
  • Find/Select the user
  • Fill start - end date, delivery method

When the website will be on maintenance (or due to an error), we should display a backup maintenance page as it happens for videos.cern.ch, see here.

• integrate load balancer
• integrate backup website
• move reverse proxy config from nginx to load balancer

[14W + 5W(Invenio)] CDS/Invenio-ILS

[63d] Backend

• [2d] handle _access on documents/items
• [15d][not core] Calendar functionality
• [10d][not core] Statistics - page views
• [2d][later] Integrate final data model
• [15d][core] Files storage -> add, update, remove files
• [5d][not core] Email notifications: reminders and updates
• [5d] CDS Overlay bootstrap and ILS integration
• [2d][core] CDS Overlay CERN Oauth for authentication
• [1d] CDS Overlay E-groups for permissions
• [10d][not core] CDS Overlay Amazon harvester
• [4d][not core] CDS Overlay EDH termination sheet

[75d] UI

• [25d] Back office UI
  • Librarian can create a Loan for user
  • Librarian can manage Loan requests
  • Librarian performs loan return or extend
  • Librarian can see Loan information (individual loan, loan history, etc.)
  • Librarian can search with facets
• [5d] Search with facets
• [10d] Better homepage
• [10d] Better details page: request loan, available copies, e-books
• [15d] Authenticated views: user/loans pages, user cancels loan, history
• [3d] Better webpack build
• [20d] Invenio Search UI
• [5d] CDS Overlay logo/about/faq/help pages
• [5d] CDS Overlay sharing/basic SEO

## [1W] Circulation

• Enhance search class methods by addressing these issues
  • inveniosoftware/invenio-circulation#18
  • inveniosoftware/invenio-circulation#19
  • inveniosoftware/invenio-circulation#20

[25d] Acquisition

• The backend should be implemented and should be plugged in with the ils backend and ui
• migrate data

[25d] ILL

• The backend should be implemented and should be plugged in with the ils backend and ui
• migrate data

[40d] Migration

[20d] Migration of records

• cleaning schemas and managing different document types within one big schema
• implement new rules according to the needs of Library (fields to be decided about how to migrate, some discoveries during cleaning the data)
• implementing the actual migration (db configuration, move and reorganise the migration code - it is written in cds-dojson)
• reorganizing the rest of the code in cds-dojson and cds-migrator-kit repositories

[5d] Migration of circulation data

• migrate loans
• migrate users

[15d] Migration of items

• migrate the items
• migrate the volumes

[10d] Invenio Record Editor

• Pass custom configuration from invenio instance to the angular editor.
• Display problems if any when edited record is not valid.
• Test it with an actual invenio instance i.e. ILS
• handle references

[5d] Infrastructure

• OpenShift
• ESoD vs ES (fix branch in invenio-search)

[0] Testing

• List test scenarios, ensuring all requirements are taken into account

Nice to have

• way to copy data between prod and pre-prod

Verify that UI looks good on mobile and when using Internet Explorer

Review

• Issues:
  • https://github.com/orgs/inveniosoftware/projects/19?card_filter_query=milestone%3A%22sprint+12%22
• Timeline: https://cernbox.cern.ch/index.php/s/HaQ69Tkgvw8oThf

UI

• Backoffice: Volume/Series
• Cancel loan with reason
• Document access restrictions UI

Migration

• Data model review for Volumes/Series
• Working on migration of Volumes/Series, Keywords, Locations

Q&A

• License/Copyright fields