[Yale ArchivesSpace] Update on ArchivesSpace

[Wisner, Melissa]

Hello,

There are several important updates to share with colleagues regarding ArchivesSpace work.

Library IT Index Work on Prod ASpace

*         Last Wednesday 9/30, a partial reindex was conducted on production ASpace. This happened several times over the past 3 months, however, this time Library IT adjusted a configuration to temporarily disable solr backups. This was to test the theory that indexing and logging failed due to lack of disk space during routine backups. Additionally, a script written by Hudson Molonglo was put in production to capture minute by minute disk capacity. This captured space fluctuations, and the details of disk usage over several days.

*         Prior to disabling  backups, production ASpace regularly reached 100% capacity, leaving as little free space as 12MB. Following the change last week, indexing has remained up to date, the application continues logging, and we have not fallen below 12GB of free space. We want to continue checking with ASpace users to confirm a sustained improvement. While the early news is promising, I'd like another solid week of logging before I cross this off our list.

*         To assuage any concerns regarding suspending backups, I will tell you that if anything catastrophic happens, we can rebuild off the last copy very quickly. ASpace is designed to work that way. This is also not permanent practice. We will work on several recommendations in the coming weeks.

Hudson Molonglo Work

*         HM is completing their analysis of the ongoing index issues with production ASpace. They posit the root cause is a shortage of disk capacity during routine solr index backups. They provided a set of recommendations to facilitate ongoing backups, storage of backups, and disk monitoring. We are pleased the solution is low-cost and straight forward in terms of system support. Library IT is working on implementing these recommendations in the current production instance (as a reminder hydratest is a temporary home for production ASpace). We will ensure these standards are configured in the permanent location of ASpace, whether new local hardware is used, or if a decision is made to host in the cloud with Lyrasis.

*         As noted earlier, the initial results are promising, but my preference is to pop the cork a week or two later.

*         With the indexing analysis wrapping up, HM is now able to begin work on plugin updates for the 1.3 release of ASpace. Our new estimate for this work is the end of next week (10/16).

Upgrading to 1.3 release

*         Last week Library IT started the process of upgrading the "dev" instance of ASpace to release 1.3. This included a refresh of the database and index from production, and enabling all the plugins already compliant with 1.3. We are waiting for the additional plugins from HM, and as mentioned, we hope this is sometime next week.

*         This work is a critical task to start the hosting evaluation project with Lyrasis. A prerequisite is a common code version of ASpace and compliant plugins.

*         Once all plugins are delivered, Library IT plans to upgrade the local test and production instances. Staff can benefit from the improvements the release will bring prior to the final decision regarding permanent location of ASpace. This is an important step to provide good customer service, even if in a temporary configuration.

Hosted ArchivesSpace Evaluation

*         This afternoon there was another conference call with Yale Library and Lyrasis. We are anticipating a final Statement of Work from Lyrasis by the end of next week, detailing the scope of the hosting evaluation. The project will run during the next two months, and will involve Library IT, Archivists, Lyrasis, Yale's Procurement staff, Yale's Information Security department, and University ITS.

*         Timeframes and commitments have been reviewed with all participants and we will be sharing regular updates on our progress.

Please let me know if you have any questions, comments, or considerations.

Sincerely,
Melissa Wisner

Yale Library IT



From: Wisner, Melissa
Sent: Wednesday, September 23, 2015 1:15 PM
To: 'yulaspace at mailman.yale.edu'
Cc: libraryit at yale.edu
Subject: Update on ArchivesSpace

Hello Everyone,

There are several updates to share on ArchivesSpace efforts.

Hosted ArchivesSpace Evaluation

*         Ray, Maureen and I participated in a discussion with Lyrasis last week about the proposed project to evaluate their hosting service for ASpace. We are outlining a project proposal and timeline. Ideally the evaluation period will begin October 1, and it will involve testing several environment configurations that may meet or exceed the current production hardware at YUL.

*         We also talked to the University of Tennessee at Chattanooga and the University of Southern California about their experience as hosted AS customers. Both sites are small in comparison to YUL, but the calls were helpful steps in gathering data about this service and whether it will work for YUL. USC does have a custom plugin for Aeon in use with ASpace and indicated Lyrasis has continued to maintain it in the hosted environment. This area is of particular importance to YUL due to our many customized plugins.

*         We are also discussing terms and conditions we would need in documents such as a Service Level Agreement, and written policies on data security and data retention.

*          A critical stipulation I would like to negotiate is for Lyrasis to support YUL at least one core release earlier than their current general release (e.g. support us on 1.3 even if 1.4 is the general release). Typically in a hosted arrangement, Lyrasis alerts customers 2 weeks prior to a planned upgrade in production. I want YUL to have more management of that activity. This will allow YUL flexibility in scheduling upgrades to ASpace, and will provide additional time if needed to modify plugins.

HM Development

*         HM is working on analysis of our production ASpace indexing issues. Work is taking place during North American evenings, due to the time difference with Australia. This is an effort to hold more real time conversations during the troubleshooting period.

*         Approval has been received for HM to update Yale plugins for compliance with version 1.3 of ArchivesSpace. HM will begin this work, estimated at 30 hours, once the indexing analysis is complete.

*         We are hoping both the index analysis and plugin work completes by the end of next week, so those changes can be promoted to production ASpace.

Preparing for ASpace ver 1.3

*         Work is starting in Library IT to prepare the development instance of ASpace for upgrading to 1.3

*         This is a critical step to support the two previously mentioned updates. Readying a local instance for 1.3 gives us a head start for testing the plugins when they are delivered by HM. It also is a requirement for starting the Lyrasis hosting evaluation. Lyrasis needs our database to be a core code release and confirmation that all plugins are compliant with 1.3, before they bring it into a hosted environment.


An important point of negotiation, both philosophical and practical (my favorite kind!), is that ASpace is an open source product. Any development therefore begins with customization. Lyrasis states they need to host customers with a core code release. I understand why from a support pov, however reality is ASpace is meant to be customized. We did hear from USC that Lyrasis has supported the custom Aeon plugin they use with ASpace, but YUL has more than one. Even if we never added others, we will always be in a state of customization, modifying the current set as required to maintain compliance with ASpace core code changes. As we evaluate the hosting service, we need to have an open conversation with Lyrasis about the ongoing needs of a site the size of YUL, our commitment to support the development of ASpace as a modern collection management system, and what we would expect in compromise or exchange.

Please let us know if you have any questions about these areas of work, or other aspects of ArchivesSpace performance efforts.

Melissa Wisner
Library IT
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.yale.edu/pipermail/yulaspace/attachments/20151006/2b829102/attachment-0001.html