curationexperts / goldenseal Goto Github PK

Text from the SOW:

Provide support for automatic lease termination and notification of embargo expiration. This functionality is largely an outcome of the base application development and is listed here as a checkpoint to measure project progress.

Since Goldenseal has lots of "under construction" errors and issues, create a bare rails app we can deploy to demonstrate the systems work.

Display metadata field on the show page for an Image.

Customize the layout and metadata fields shown for images. Configure the OpenSeadragon based, IIIF-compliant image viewer to WU specific requirements.

Display the metadata fields on the show page for an Audio object.

In /opt/sample-data/DCE_WU_content_model/DC-fromDLXScolls/*.xml

Add text-specific metadata fields to search results listing.

From the SOW:

Customize the layout and metadata fields shown on individual item views. Provide links to raw MODS, VRA, or other XML based metadata attachments.

ActionView::Template::Error (undefined local variable or method `new_user_session_path' for #<#<Class:0x007f7c9d8a9b90>:0x00000005469da8>):
    2:   <%= render 'shared/add_content' %>
    3:   <%= render 'shared/my_actions' %>
    4: <% else %>
    5:   <%= link_to 'Log In', new_user_session_path, class: 'btn btn-primary login', role: 'menuitem' %>
    6: <% end %>
  /opt/goldenseal/shared/bundle/ruby/2.2.0/bundler/gems/curation_concerns-b326ca12bb8a/app/views/shared/_site_actions.html.erb:5:in `__opt_goldenseal_shared_bundle_ruby_______bundler_gems_curation_concerns_b___ca__bb_a_app_views_shared__site_actions_html_erb__1420409845893598810_70086594714280'

When not signed in an viewing a show page.

Versions should match #1

From the SOW:

Provide user-friendly browser based content upload for single and multiple files. Metadata is applied to all files within a single upload. This functionality is largely an outcome of the base application development and is listed here as a checkpoint to measure project progress.

/opt/sample-data/DCE_WU_content_model/gass-FOXML-MODS-textimage

Is this out of scope?

Part of SOW E-2:

Configure a book-reader like interface for WU TEI.

Tasks:

• Gather requirements for the book viewer

Questions:

• Is there an existing book reader that they are interested in using?

Steps to reproduce:

1. Log in as non-admin user
2. Visit http://localhost:3000/dashboard
3. Click "View Works"
4. Select some works in the list
5. Click "create collection"
6. Fill in the title and submit the form

The error:

NoMethodError in CollectionsController#show
undefined method `from_field=' for CollectionSearchBuilder:Class

/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/sufia-core-6ed042ae7086/app/search_builders/collection_search_builder.rb:5:in <class:CollectionSearchBuilder>' /Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/sufia-core-6ed042ae7086/app/search_builders/collection_search_builder.rb:2:in<top (required)>'
activesupport (4.2.3) lib/active_support/dependencies.rb:457:in load' activesupport (4.2.3) lib/active_support/dependencies.rb:457:inblock in load_file'
activesupport (4.2.3) lib/active_support/dependencies.rb:647:in new_constants_in' activesupport (4.2.3) lib/active_support/dependencies.rb:456:inload_file'
activesupport (4.2.3) lib/active_support/dependencies.rb:354:in require_or_load' activesupport (4.2.3) lib/active_support/dependencies.rb:494:inload_missing_constant'
activesupport (4.2.3) lib/active_support/dependencies.rb:184:in const_missing' /Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/sufia-core-6ed042ae7086/app/controllers/concerns/sufia/collections_controller_behavior.rb:46:incollection_member_search_builder_class'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/hydra-collections-486ed3b756e8/app/controllers/concerns/hydra/collections_controller_behavior.rb:159:in collection_member_search_builder' /Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/hydra-collections-486ed3b756e8/app/controllers/concerns/hydra/collections_controller_behavior.rb:149:inquery_collection_members'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/sufia-core-6ed042ae7086/app/controllers/concerns/sufia/collections_controller_behavior.rb:61:in query_collection_members' /Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/hydra-collections-486ed3b756e8/app/controllers/concerns/hydra/collections_controller_behavior.rb:45:inshow'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/sufia-core-6ed042ae7086/app/controllers/concerns/sufia/collections_controller_behavior.rb:31:in `show'

Not sure if this is a failure of transcoding or only of playback.
I uploaded a short video. The system created a Video work attached the file, and added a viewer, but there's no derivative available for viewing: http://wustl.curationexperts.com/concern/videos/hm50tr73g
Gist of production log: https://gist.github.com/acozine/6a4d8d3f3d79cbb6dceb - lines 24 and 54 seem to point to the issue.

Update the importer to import the <dc:rights> fields when importing the sample files from /opt/sample-data/DC-fromDLXScolls/*.xml

Since we're currently working on the Text importer, I'm updating the scope of the story to import the dc:rights for Text objects only.

Add role & task, test.

This is part of SOW C-2.

We'll need a iiif backend server. Maybe use riiif.

From the SOW:

Implement DLXS-like collection groupings where an item can only be contained by a single administrative collection. Provide ability to create and manage administrative collections, including ability to manage collection membership. Administrative collections will be required to implement DLXS-like collection level customizations.

Gather requirements for Admin collections.

Questions:

• Where do self-deposits go? Can the user choose which admin collection their work will be in? Or is there a limited set of choices? Or a single self-deposit collection?
• What does it mean to implement DLXS-like collection customizations?

From the SOW:

Provide support for automatic lease termination and notification of embargo expiration. This functionality is largely an outcome of the base application development and is listed here as a checkpoint to measure project progress.

NOTES:
The system should the basic lease & embargo functionality previously available in Worthwhile.

Further customize the metadata provided in search result listing for images. E.G. dimensions, Color profiles, original format, etc.

Question:

• What fields should be shown in the search results listing for an image?

Somehow my collection got indexed twice and it shows up twice in the drop-down menu for adding items to the collection.

<div class="form-group collection_id">
      <div class="controls">
          <select name="id" id="id" class="form-control" aria-labelledby="qn59q396v-modal-title">
<option value="">Make a Selection</option>
<option value="hx11xf24j">Collection AAA</option>
<option value="ht24wj40p">Collection AAA</option>
          </select>
      </div>
</div>

• install x-sendfile on wustl.curationexperts.com
• create ansible role for installing and configure xsendfile on centos
• apply ansible role to wulib.wustl.edu servers

Send the default derivative specs for

• audio
• video
• images

1. Click the "+" menu
2. Select "Add a Collection"
3. Notice the warning message in the flash box: "Select something first"

We should remove that message since we don't have any checkboxes that a user can select on the page that displays their list of works.

Add metadata fields to search results listing for Audio and Video objects.

On the search results page, I can correctly filter by "Works" or "Collections", but if I select the "All" filter, there are zero results.

From the SOW:

DCE will implement a core profile of Dublin Core metadata terms to support repository-wide search, browse, and administrative activities. Core terms will be determined based on TEI, MODS, VRA, and other metadata cross-walks provided by WU. The system will provide RDF based metadata to support future linked open data (LOD) interfaces (including DPLA harvesting). The system will support supplemental schemas via attached metadata files.

The behavior of the importer should be similar to the TEI and Transcript importer.

• The importer should take the path to a directory containing one or more TEI transcripts,
• The importer should take an optional parameter to set the access level [open, institution, private]
• The importer should take an optional parameter to set the Administrative Collection
• The importer should search for associated image files in the directory where the transcript is and it's children

Import the following fields

• title /vra/collection/work/image/titleSet/title
• contributor /vra/collection/sourceSet/source/name
• description /vra/collection/work/image/relationSet/display
• content license /vra/collection/rightsSet/rights/@type
  "publicDomain" maps to "Public Domain Mark 1.0", all others map to "All Rights Reserved"
• publisher /vra/collection/locationSet/location/name/@type=other
• date_created /vra/collection/work/image/dateSet
• extent /vra/collection/work/image/measurementsSet

The name of the corresponding image file is at

• /vra/collection/work/image/titleSet/title/@href

Sample data is located in /opt/sample-data/DCE_WU_content_model/schiele-VRA-imageonly

NOTE: Image importer is a prototype,time-bounded feature to help us scope how much additional work would be required to build a complete metadata parser and/or how metadata would need to be modified to be more easily machine readable.

Part of SOW D-2:

Implement a basic media player based on WU specific requirements

• OS and version Red Hat (CentOS) 7.0
• Ruby version ruby 2.1.5
• Fedora version Fedora 4.3.0
• Java version Sun Java JDK 1.8
• Java server Tomcat 7.0.59
• Solr version Solr 4.10.3
• Database MySQL 5.6.23
• Web server Apache 2.2.15-39
• App Server Passenger 4.0.59

Display the metadata fields on the show page for a Video object.

Worthwhile currently only supports Fedora 3 based repositories. DCE will make the necessary updates to support Fedora 4. These updates will be made available to the broader Hydra community under Apache 2.0 open source licensing terms.

NOTE: at this point we will use sufia-core or whatever the successor gem to worthwhile is based on the recent PCDM refactoring.

Steps to reproduce the error:

1. Log in as a non-admin user
2. Visit http://localhost:3000/dashboard
3. Click the "Create Work" button
4. Fill in the form and submit it

You will get an error:

NameError in CurationConcerns::GenericWorksController#create
uninitialized constant Hydra::PCDM::AddObjectToObject

/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/hydra-works-4e0e27fa4d5f/lib/hydra/works/services/generic_work/add_generic_file.rb:15:in `call'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_file_actor.rb:50:in `create_metadata'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:70:in `attach_file'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:33:in `block in attach_files'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:32:in `each'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:32:in `all?'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:32:in `attach_files'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/curation_concerns-models/app/actors/curation_concerns/generic_work_actor.rb:9:in `create'
/Users/valerie/.rvm/gems/ruby-2.2.2@goldenseal/bundler/gems/curation_concerns-3959d2079498/app/controllers/concerns/curation_concerns/curation_concern_controller.rb:33:in `create'

in /opt/sample-data/DCE_WU_content_model/great-depression-TEI-textvideo

Versions should match #1

From the SOW:

Configure derivative generators for WU specific standards and practices for image, audio, and video access proxies.

• Audio
• Video
• Office Docs
• PDF
• Images
• TEI (?)

From SOW:

DCE and WU will collaborate to identify sample content and metadata for the initial collection to be migrated. In addition, we will identify up to 3 additional content types or collections for migration analysis, especially high-risk or high-complexity collections such as the Red Brush collection. This story does not have specific time allocated to it, but is listed as a placeholder to ensure these activities take place early on in the development timeline.

• Identify the primary collection for testing
• Representative samples from primary collection
• Media and associated transcript samples
• Image samples
• TEI samples

From the SOW:

DCE will implement the core object types supported by Worthwhile: PDFs; Microsoft Office, OpenOffice, & LibreOffice Documents; Tiff & JPEG Images; MP3 & Wave Audio; and open source Video formats. This functionality is largely an outcome of the base application development and is listed here as a checkpoint to measure project progress.

• Can ingest and download a Word Document
• Can ingest and download an Excel Spreadsheet
• Can ingest and download a PowerPoint Presentation
• Can ingest and display a PDF
• Can ingest and display an Image (TIFF)
• Can ingest and display an Image (JPEG)
• Can ingest and display an Image (JP2)
• Can ingest and play MP3 audio
• Can ingest and play WAV audio
• Can ingest and play MPEG-1 video
• Can ingest and play MPEG-2 video
• Can ingest and play MPEG-4 video
• Can ingest and play WEBM video
• Can ingest and play AVI video
• Can ingest and play MOV video

From the SOW:

Integrate the repository with WU’s authentication and authorization infrastructure. The core access policies supported by Worthwhile are: public access, restricted institutional access (via login), and private to the depositor and administrators (via login). This functionality is largely an outcome of the base application development and is listed here as a checkpoint to measure integration with the WU authentication infrastructure.

Questions:

• What is the authentication infrastructure? (NTLM or Kerberos)
• What is the authorization technology?
• Can we get URLs & credentials if needed?
• Are you able to manage groups/roles with your campus authorization? Or do we need the app to manage groups locally?

The DC parser reads XML files and extracts DC metadata for the records.
In the sample data, they have a dc:date field. We need to import that field.

The xpath mapping is located in:
dc_xml_parser.rb

From the SOW:

Implement the customizations necessary to associate images into multi-image works including multi-sided and accordion type images. (Note that book-like objects with associated transcripts are covered under E-1)

• Metadata can be associated an image-class work
• One or more images can be attached to an image-class work
• Images attached to a work can be ordered (eg. 2-sided and accordion images)

NOTE:
This story covers the model only, UI features are covered in #11 (upload), #31 (show), & #27 (search)

DCE provides Ansible scripting to support automated configuration management of core infrastructure components including Fedora, SQL database server, Web application server, Solr and related components.

• #1 DCE Sandbox
• #2 WU Sandbox
• #3 WU Production

From the SOW:

Default access levels are “Public”, “Intitution-only”, and “Private”. DCE will implement the necessary enhancements to support additional levels of access granularity

Questions:

• What access categories do you need? Are the defaults enough?
• What sorts of things need IP/location protection? Is this just on-campus IPs?
• How many IP ranges will there be?
• Do we need to differentiate between "read" access and "discover" access?

By default when you create a work, if you don't explicitly set the visibility, it will be set to the most restricted level. The current behavior of the importer is to not set visibility, so all bulk imported works will have the default visibility. It would be labor-intensive to edit each new record to change the visibility, so we'll probably need a way to set the visibility for multiple records at a time.

Some possible options:

• Add a command line switch to the importer script so that the person who is running the import can set the visibility for all objects in that import.
• Some kind of multi-select batch edit in the UI.
• Ability to change the visibility at the collection level and have the new visibility apply to the members of the collection.
• A command line script or rake task to edit visibility for multiple records.

jpeg2000 and TEI here:
/opt/sample-data/DCE_WU_content_model/circuit-court-records-TEI-textimage