nulib / avalon Goto Github PK

Description

Update Avalon's default search styling to NU branding. Verify default search functionality works.

Done Looks Like

• NU's search styles applied
• Default search functionality works as expected

Description

This is our first crack at this. We will have lots of room for improvement. The goal is to develop a list of action items to take after/ during the pilot to:

• improve performance
• Reduce cost
• increase redundancy (if necessary)

Description

Based on data from the pilot, come up with a cost breakdown. In addition, we should verify that there are no more cost-saving measures we should be taking (reserved instances/ glacier or reduced redundancy storage for certain files / smaller instances based on load_

Done looks like

• analysis document is written and published
• action items written for cost savings measures

Description

It seems like we have several very large derivatives (derivatives that are larger than masters). We should evaluate and retranscode as necessary. This ideally should take place before cutover.

Done looks like

@mbklein sets up a meeting with RDC/Devops to go over infrastructure, published cloud formation scripts, and next steps needed (represented in other tickets)

The goal of the meeting is for RDC/Devops to finalize the plan for the pilot and create any new issues/ carve out time for training if necessary.

Topics to be covered:

• Infrastructure
• Cloudformation
• Needs from monitoring

@carrickr can you add a link or drop in the thoughts you wrote up directly here?

Description

Before go-live we should load-test avalon on AWS.

Done looks like

• a realistic load is discussed
• scripts are written for load testing (do we have a tool for this?)
• crank is turned

Overview of timeline

< May 11 (prep work, systems ready)

During this cycle, the rdc team was largely heads down investing, drafting the AWS pilot proposal (https://github.com/nulib/repodev_planning_and_docs/wiki/Avalon-AWS-Pilot-%28Summer-2017%29), and ensuring we could pull this off.

• Set up another meeting how/why/theory and maps #20
• Dig into AWS Cloud watch / discuss strategy with Devops team #21
• Devops to go over aws cloud formation scripts / Read avalon one/MBK to assist #22
• @mbk creates initial pass of system as part of grant work
• Finalize how auth will work with AWS #23
• Plans for migrating data to AWS #15

May 11-21

Discussion with outside folks after a call was made that we could hit our summer session deadline.

• skinning first pass @adamjarling May 11-May 21.
• System Stable @mbklein May 15
• Migration path decision collections @carrickr May 15 #15
• Inform Molly, Alex, and Cecile (Features, redundancy) (MMLC) @Nic4Images
• Work with IT Infra to make sure FTP clients (Cyberduck) are available on library machines (Molly and Alex and ours). Email sent from CC to Bob and Marek May 11. @ccaizzi ### May 28-30 (stabilize systems, migrate)
• How we are handling Monitoring/ How we are going to do it #21
• Plans for disaster (e.g. system outages) #24

May 22 - May 31

During this period we will be moving data and swill start testing/ train users.

• Move derivatives to Avalon AWS #52
• Migration Frist run @carrickr @mbklein (May 30) #60 #61
• Migrate test data, start testing @carrickr #60 #41 #61
• Set storage to Reduced Redundancy for derivatives #51
• Molly, etc, will need to use FTP for batches. Ensure it's loaded on system
• text needed About, Help (from current footer and https://wiki.dlib.indiana.edu/display/VarVideo/Avalon+User+Guide), and Contact Us @Nic4Images #62 #63

June 1 - June 5

Testing and final migration (if necessary)

• Metadata testing ( @jenyoung ) June 1 -5 #75
• Final migration of all data and derivatives #61 #30
• training of Cecile and RDC (train the trainers) June 1 #76
• wire up canvas #65
• Procedures written on how to deal with emergencies @carrickr @mbklein @d-venckus @Toputnal #79

June 5-9

NOTE: DS will be in Italy, Nicole is at a conference. @jenyoung is in charge.

• Team does full regression testing on production if necessary. #73 (first pass). Testing should be completed after data migration (notice from @carrickr) and before June 8.
• Molly and Cecile test June 8 (on production) @jenyoung and @ccaizzi to triage.
• Freeze of adding content to AVR until Monday June 12.

June 12-16

Intercession. During this week we will do final production testing, ensuring that canvas is working, etc. In addition, we will ensure that all of our users that need access have it.

• Production Testing (Spot check permissions, etc) Erin will be conducting June 12 PM along with Course Reserves and Media and Design (Cecile) #73 @jenyoung
• New material is only being put in Avalon
• Final sign off (DS/CC/JY)
• Switch DNS / Golive #77 #102
• Firedrill for Avalon (Carrick to act as chaos monkey)

June 19 Go live.

• attach outlying master files #142
• Final, final migration of any new data #144 #143
• verify about text is working #62
• Final splits of HLS #123

Post Go-Live

• Evaluate very large derivatives #53
• Cost analysis for 12 months/ Dig into ways to save money/ After kickoff #26
• Lifecycle management of digital objects
  (e.g. pushing transcoded objects into glacier
• Monitoring for improvements in speed / bulletproofness. Continue to offer suggestions
• Load tests / how do we get a realistic figure #27
• Plans for syncing master files from AWS #25
• improvements #28
• Celebratory Donuts/Beer/ and fame and glory

Risks -- any of these can put us off time

• Migration goes bad/ off
• Training doesn't go well (users are confused, things don't work)
• Canvas integration fails
• Authentication stops working
• Devops has issues with security, etc.

Hi,
Can I have access to libwowza1p.library.northwestern.edu? I'd also like to know what you actually it when you SSH into it as I can't even get a response from it when on the SSL VPN. I don't need any special power perks but I do need to be able to cat logs, etc.

Thanks!

I went thru my initial assessment again comparing what had been removed after the script was made to work again on Alps. There are still files that should be removed on Alps. I do not know if they are in the farchive.

alps_list_2016.xlsx

• Contact Ruby Rebolledo and find out how to configure OAuth
• Configure OAuth

Description

Customize application header and footer to use NU's branding header and footer templates.

Done Looks Like

• Include NU template header and footer markup
• Apply header and footer styling
• Update text and links

Description

Move production Avalon Derivates to AWS bucket in preparation for cut over. We will need to mark time that this occurs as well, so we can grab any diffs production or course reserves works on in the meantime.

Done looks like

• Derivates are moved
• Date is noted for final diff move

Description

Master files in Avalon should sync to local storage during the pilot phase. We have not made a decision about long term storage of files, and it is out of scope for this project. For now we want to have a copy of renamed master files (not derivatives) synced to local storage.

Done looks like

• master files are stored locally
• Decision is made whether to keep files on S3

Michael found 1800+ audio files that did not get moved to farchive during his audit. He will generate a report with file name and collection for DCM team to determine if there are records in AVR (spot checking makes sense) or if they were part of a failed batch and re-ingested, or if they were test/old files and not supposed to be moved. related to issue #12 now closed

Description

Provisioning the Avalon in AWS environment is handled via cloud formation scripts. It is critical the Devops team have time to understand how the scripts work and what they do. MBK will give an overview, and point to any training.

Done looks like

• the teams feel comfortable with cloud formation

Description

We may switch from LDAP to NUIT's OATH for our sign in on Avalon. Per @mbklein "I'd rather be dealing with tokens than firewalls"... The end goal is that authentication into Avalon works as expected for an end users : I can log in to Avalon using my netID and pass.

Done looks like

• NUIT is called for help, if necessary.
• Documentation is written about authentication patter
• Code is written, if necessary
• Deployed Avalon instance can accept netID/ Pass.

So my thoughts are we need three main components: Player, ActiveEncode (for derivatives), and ActiveStreaming. We'd also have enriched metadata as a need, but it's not minimum out the door. We can add that in as we go given our current Avalon instance doesn't have particularly rich metadata.

This would look something like:

ActiveEncode

• Ties into HydraDerivatives (integrated into Hyrax)
• Can use ffmpeg (for developer machines)
• Can use Amazon Encode (for production)
• Derivatives are not stored in Fedora / can be accessed with zero Fedora calls

Player

• HTML 5 and HLS with fallback option
• IIIF-AV compliant
• Captions support, but entirely 508c compliant yet
• Integrated into Hyrax, probably in the form of being a Flipflop feature (off is use Hyrax's current lightweight player)

ActiveStreaming

• Cloudfront capable for production
• nginx for developers
• Anyone who says use Red5 gets shot with a nerf dart
• Integrated into Hyrax, probably as a Flipflop feature again (off defaults to just play from disk)
• Can stream with zero calls to Fedora for authorization, etc.

Anything else NU needs? We can probably then evolve these into larger tickets on the main Avalon board. On our end I'm seeing a workflow of:

• Create the gem / service / whatever
• Get it in Hyrax using Flipflop
• Get a point release of Hyrax cut
• Pull said point release in for our NextGen work
• iterate if it turns out we missed something NU leads for the MVP

Description

As we move forward with the Avalon on AWS pilot, we need a plan for monitoring. If systems fail, people will need to be notified and a call tree will need to be set up to bring the system back up. OPSgenie should get notifications.

Opsgenie integration with AWS Cloudwatch.

https://www.opsgenie.com/docs/integrations/aws-cloudwatch-integration

Done looks like

• Team feels comfortable with cloud watch
• initial monitoring is setup and documented.
• monitoring should be analogous to current monitoring
• errors on ops genie should ping MBK as on call for course pilot

Systems needing monitoring:

• Fedora
• Avalon Web
• Avalon Workers
• RDS / Do we need to monitor this?
• SOLR

Severity

Is the production site running?

• yes
• no

Are staff blocked from performing their work?

• yes
• no

Descriptive summary

After a successful batch ingest, media files are not being automatically removed from the folder on alps.

Expected behavior

Media files removed from folder upon successful batch ingest.

Actual behavior

Media files remain in alps folder after successful batch ingest

Folders with this issue:

• Dave Eanet
• Imagination without borders
• J Rothfuss

I’m loading the episode 2 file from my local drive to that Alps dropbox now – it should take about 45 minutes to finish loading, then I’ll try uploading to our Avalon item again. It does now appear when I Open Dropbox in the Avalon item, so looks like this is going to work – yay! Thank you, Michael!
Molly

From: Michael B. Klein
Sent: Friday, January 06, 2017 1:10 PM
To: Main Library Reserves [email protected]; Debs Cane [email protected]; Nicole Gabrielle Finzer [email protected]
Cc: Alex Barontini [email protected]; Kurt I Munson [email protected]
Subject: Re: adding 2 files as sections to an existing “Course Media” item in Avalon – for a Monday 1/9/17 Course Reserves request

Try the Alps dropbox directory called “Course_Media,_Northwestern_University_Libraries”

mbk

From: Main Library Reserves [email protected]
Date: Friday, January 6, 2017 at 11:48 AM
To: Debs Cane [email protected], Nicole Gabrielle Finzer [email protected], "Michael B. Klein" [email protected]
Cc: Alex Barontini [email protected], Kurt I Munson [email protected]
Subject: adding 2 files as sections to an existing “Course Media” item in Avalon – for a Monday 1/9/17 Course Reserves request

Hi, Debs, Nicole, and Michael,

Prof. Michael Kramer has asked to have episodes 2 & 3 of Jazz added to one of his Winter Quarter Canvas sites by this Monday, the 9th. I’ve captured both and have placed the files in Alps, Course Video. They are:

21448275530002441_jazz_e2_gift.mov
21448275530002441_jazz_e3_ourlang.mov

Ideally, we’d add these as sections to our already existing Avalon item for the Jazz series – see https://avalonweb2p.library.northwestern.edu/media_objects/numedia:17994. However, because episodes 1 and 4 through 9 were added to Avalon as part of the Course Media collection, I’m unable to add episodes 2 and 3 to the same item in Avalon. I tried adding the episode 2 file to Course Media in Alps to see if that might work. I was able to place it there, but then it doesn’t appear when I Open Dropbox in our Avalon item. Can this be fixed so that these 2 episodes can be in the same Avalon item as the other episodes? If so, could it be done sometime today to meet our Monday deadline? Please let us know as soon as possible.

Thanks,

Molly

Molly Zolnay
Library Assistant
Northwestern University Libraries
Northwestern University
www.library.northwestern.edu
(847)467-1985

DONE

Give a report to CC about files moved, data in .

It seems for the last quarter we only increased our storage by ~2tb. Which seems considerably lower than it should be. @mbklein to investigate.

@d-venckus can you do a report on our isilon storage and attach to this ticket?

• David dsc712
• jim jre700
• David V. drv460
• Carrick Rogers
• Chris Syverson
• karen d
• brendan q.

My understanding (correct me if I'm wrong @mbklein) is that right now you need a VPN to connect into AWS machines. To load the data into the Fedora up on AWS, I assume we don't want to install a VPN on avalon-prod, rather we want to either set up some kind of keypair or else temporarily expose our Amazon Fedora to avalon-prod while we push data.

Description

We need to either create a new GH repo or update the current 4.x gh repo/fork (this one here!). This should be done ASAP so Adam can start to work on customizing avalon.

We need ssh (and root) access to avalonweb2p, thanks!

Description

Update Avalon sidebar to reflect NU sidebar branding.

Done Looks Like

• Avalon sidebar visually looks like NU template sidebar

2.4 tb is currently being eaten up by matterhorn files which can be cleaned up per conversation in #4 .

Please run investigate and run

https://wiki.dlib.indiana.edu/display/VarVideo/Maintaining+Matterhorn+Disk+Space

per MBK.

Description

When things fail we need to know who to call when. This is somewhat related to #21.

Done looks like

• documents written describing outage/ system down plans

Publish out the network diagram for Avalon in AWS. Just attach to this issue. I'll put it in the wiki.

Severity

Is the production site running?

• yes
• no

Are staff blocked from performing their work?

• yes
• no

Descriptive summary

Erin cannot access alps.library.northwestern.edu/dropbox

Description

Apply Northwestern branding styles to NU's Avalon 6 application

Done Looks Like

• #34 Apply NU Branding Sass/CSS to Rails app
• #35 Header & footer
• #36 Search
• #37 Favicon
• #38 Page layout
• #39 Side navigation styling
• #40 Homepage (re-create current Browse functionality)

Description

Use Northwestern favicon instead of Avalon favicon

Done Looks Like

• Northwestern favicon appears in browsers

Description

Update NU Avalon homepage to reflect NU branding, and to incorporate 'Browse' functionality.

Done Looks Like

• Come up with idea to incorporate Avalon's default 'Browse' functionality

TIMELINE

• skinning @adamjarling May 11-May 21.
• System Stable @mbklein May 15
• Migration path/ test 2-3 collections @carrickr May 15
• Migration Frist run @carrickr @mbklein May 30
• Metadata testing ( @jenyoung ) June 1 -7
• Testing (Everyone) June 5-12 (June 8 Cecile and Molly will be testing)
• Tell Molly/ etc that new material should be loaded in new system only June 5?
• Production Testing June 12

Nicole and David out June 5-9. Nicole out June 12 week as well.

INTERCESSION June 12-16

• Final Migration (if necessary) June 14
• Go Live June 15

Classes Start June 19

Critical Messaging and tasks

• Demo new system and workflow to Molly, Cecile, etc. May 29-June 2 (@Nic4Images to create meeting to demo and show them at their desks with @davidschober)
• Migrate test data, start testing @carrickr #60 #41 #61
• Molly, etc, will need to use FTP for batches. But there are no limits on web uploads
• text needed About, Help (from current footer and https://wiki.dlib.indiana.edu/display/VarVideo/Avalon+User+Guide), and Contact Us @Nic4Images #62 #63

Risks -- any of these can put us off time

• Migration goes bad/ off
• Training doesn't go well (users are confused, things don't work)
• Canvas integration fails
• Authentication stops working
• Devops has issues with security, etc.

This is pre work for #15 and #30

25-100 items should be migrated for testing purposes.

• ID test set of items
• move derivatives
• create HLS streams
• move metadata/reindex in whatever way we deem so.

DONE

New collection added to the HP of media.library.northwestern.edu

Email from Laura

As I mentioned yesterday to Michael and Chris, I have been working on an audio collection of interviews with African authors from the Herskovitz Library. It’s nearly ready to be imported to AVR, and I was wondering if it would be possible to have a link to it on the homepage like the Marcellus and Football collections. This collection is significant, and I think it would score a lot of points with Esmeralda to have be quite visible. Michael and Chris told me that it’s possible to adjust that page however we want – I’m hoping that it’s an easy task and that there’s time for one of the developers to do it.

Let me know if it’s feasible, and what the time frame would be. I am just waiting for Jen or Nicole to sign off on the metadata to import the collection.

Thanks!
Laura

Description

Create a Northwestern look & feel to NU's Avalon application by using NU template Sass files.

Done Looks Like

• NU's Sass files included in the application
• NU base level styling applied
• NU styles and Bootstrap styles are imported in the correct order
• Declare Sass variables representing NU colors, sizes, etc used in Bootstrap's compilation

Description

Train Cecile and Molly on features of the new system. Of note, playlists and markers, fast uploads and batch ingestion.

@mbklein @adamjarling has started to work on customizing Avalon 6.x for NU. We need a repo for him to push to (and you to pull from ) for http://avalon.repo.rdc.library.northwestern.edu.

Can you take this and run with it?

Hi Nicole,

I need to add Sharaku https://avalonweb2p.library.northwestern.edu/media_objects/numedia:47291 to Avalon for a class, and I’m getting the below errors when I Import the metadata (ALMA # 21678441120002441). Could this be fixed as soon as possible? I can’t seem to add Access Control for the class until these Resourse Description errors disappear. Please let me know.

Thanks!

Molly

Description

Currently we're unclear on how security patches will work in an AWS cloud environment (burn instance and bring it back up, patch hot, etc). In particular do we have a plan for SOLR ( @mbklein ), etc. If we can burn things down and bring it back up, it should be OK.

Notes

We need a new testing script for Avalon 6.0 - below is the one used for the Avalon project. We need to alter for local testing.

DONE LOOKS LIKE

• script below is altered for NUL testing

SCRIPT

Testing

Using the spreadsheet below, please complete assigned sections and kick off bugs.

create copy of the testing template

And save it to Avalon Test Scripts/completed_tests using YYYYMMMDD_lastname as a filename.

Bugs found

For each bug found, add a new github issue to avalon and label it "bug".

Bugs will be triaged and dealt with as necessary.

Completed tests

Please put completed test script in completed tests

Test on

FILL IN SERVER NAME

Example Comment when you're done with a task

Hi all, I section 1 script and notes in: https://docs.google.com/spreadsheets/d/1v-K4NWEPyERabHHCMPADI79P-d1FAu2_fqy_dgfngdI/edit#gid=1730858809

FYI I found bugs #1, #1, #1

Tasks and assignees

delete as necessary if not doing full testing. .

• 1. Create private collection
• 2. Web Ingest an item
• 3. LTI
• 4. Embed
• 5. Batch ingest
• 6. Structural Metadata
• 7. Create group
• 8. Bulk actions
• 9. Search/playback
• 10. Accessibility
• 11. Playback - sticky quality
• 12. Captions
• 13. IP based access control
• 14. Start/end dates for access (Lease)
• 15. Playlists and Items
• 16. Zoom Scrubber

When a user goes to log in, the user is returned to the home screen not logged in, and has to log in once more in order to be logged in.

Description

Use NU template page layout types to display content.

Done Looks Like

• Update markup in app to use NU template markup
• Adjust CSS / styling to ensure display is consistent with NU branding

Description

In order to hold down costs, we should set the derivatives storage to reduced redundancy. These are easily regrettable files, and even at this level are fairly durable.

Done looks like

• derivatives bucket set to RR

Set up a local instance for development.

Description

Make the NU Avalon responsive initially without major structural updates to the Avalon templating.

Note: There is only so much which is possible give Avalon's default markup. The scope of this ticket is to fix/update any UI issues which are major after applying NU brand assets.

Prof. Kaveh Askari is currently teaching 2 classes in Qatar (2017SP_MIT_351-0_SEC70 and 2017SP_MIT_398-0_SEC71) and has encountered the following issue with the playback of Avalon video titles for his classes…

If a video has been paused for about 10 to 15 minutes, he is unable to resume playback and the browser becomes unresponsive. This happens with both Chrome and Safari.

Kaveh reports that this has been the case with all his Avalon video titles so far. They are…

For 2017SP_MIT_351-0_SEC70:
· Alexandria Again and Forever
· Cairo Station
· Dry Summer
· Egyptian Story
· Rana’s Wedding
· Wadjda
· Waltz with Bashir

For 2017SP_MIT_398-0_SEC71:
· Battle of Algiers
· Black Girl
· District 9
· The Good, the Bad, the Weird
· Hour of the Furnaces
· The Music Room
· Night Cries
· Welcome to Nollywood
· The World

I’ve asked him to give Firefox a try, and suggested refreshing the page, etc. (See below correspondence for more details).

Might there be some way of preventing this playback issue from occurring? Would you please investigate and get back to Kaveh?

Many thanks, and all best,

Molly