[Yale ArchivesSpace] Further error occurrences in ArchivesSpace following last weekend's upgrade

[Gorham, Matthew]

Hi Everyone,

This is just quick a follow-up to let you know that the resequencing of the ArchivesSpace PROD database was completed successfully over the weekend, and initial tests indicate that it has corrected the problem that was causing widespread occurrence of errors in resource records in the MSSA, BRBL, and Medical Historical repositories. Melissa Wisner has indicated that the error we saw so much of last week may occur again from time to time in ArchivesSpace 1.5, but all signs indicate that it is no longer being triggered anytime a user attempts to make multiple edits within a single resource record.

As always, please report any problems or errors you do encounter in ArchivesSpace to Library IT via ServiceNow (http://web.library.yale.edu/form/report-problem). Thank you again for your patience last week while we ironed out this issue.

All best,
Matthew (on behalf of the Yale Archival Management Systems Committee)

Matthew Gorham
Assistant Head of the Manuscript Unit for Processing
Beinecke Rare Book & Manuscript Library, Yale University
P.O. Box 208330
New Haven, CT 06520-8330
203.432.7364
matthew.gorham at yale.edu<mailto:matthew.gorham at yale.edu>
www.library.yale.edu/beinecke<http://www.library.yale.edu/beinecke>

From: Gorham, Matthew
Sent: Wednesday, April 12, 2017 12:09 PM
To: yulaspace at mailman.yale.edu
Subject: Further error occurrences in ArchivesSpace following last weekend's upgrade
Importance: High

Hi Everyone,

During the day yesterday (Tuesday) we saw a big uptick in the occurrence of one of the errors that we encountered in Monday's QC testing of the ArchivesSpace software upgrade - namely, the one related to the ongoing resequencing issue that prevented users from editing and saving archival objects within resource records. The user who first reported the error yesterday said that it kept occurring in any resource record in which she attempted to make multiple edits, and after some further testing by other users, we confirmed that attempting to perform multiple edits within the same resource record is what triggers the error message and locks up the record for any further editing. After more testing throughout the morning and early afternoon, we determined that this error only seems to occur in resource records in the MSSA, BRBL, and Medical Historical Library ArchivesSpace repositories, and that it can still be fixed on a case-by-case basis using the Resync plug-in that I detailed in my message to this list yesterday (that message, which includes instructions on how to use the resync plug-in, is pasted below for reference). So, while this issue makes work in ArchivesSpace very inconvenient for many of our users for the time being, we've determined that there is no reason for work in the application to cease. I want to also assure everyone that your data in ArchivesSpace is safe, and has not been compromised in any way by this issue.

Melissa Wisner was in contact with our reps at Lyrasis throughout the day yesterday, and after doing their own round of testing, they are recommending a full resequencing of the PROD database to correct the problem in a more comprehensive way. Performing the database resequence requires that ArchivesSpace be taken completely offline for a period of 48 hours, and since, fortuitously, we have a three-day weekend coming up, we've asked Lyrasis to perform the resequence starting at 6:00 p.m. on Thursday evening so that it is completed well before the start of work on Monday morning. Please be aware that ArchivesSpace will be unavailable for staff use beginning on Thursday evening at 6 pm and it should be back online sometime on Sunday. During this time, the Aeon-ArchivesSpace add-on will be redirected to the TEST instance of ArchivesSpace for Aeon request data. It will be redirected back to PROD first thing on Monday morning.

As for why we are seeing such an uptick in this error's occurrence - we think it has to do with the incorporation of Yale's container management plug-in into the core code of 1.5, and the fact that we are in a unique position as one of the only ArchivesSpace institutions that had been using that plug-in in production prior to the upgrade. As for why we didn't identify this problem prior to the upgrade - simply put, we just weren't able to see how pervasive it was until 1.5 was put into heavy production use, which ultimately revealed the specific combination of workflows (i.e. performing multiple edits in a single resource) that triggers it. Certainly, this is something we will test thoroughly prior to the next upgrade, though as I mentioned in Monday's message, Lyrasis has indicated that changes to the overall application architecture in version 2.0 of ArchivesSpace should permanently fix the root cause of this problem.

I apologize for the inconvenience that this issue is already causing for many of you, as well as any inconvenience caused by this weekend's downtime in ArchivesSpace. If you have questions about any of the above, please don't hesitate to contact me.

All the best,
Matthew (on behalf of the Yale Archival Management Systems Committee)

Matthew Gorham
Assistant Head of the Manuscript Unit for Processing
Beinecke Rare Book & Manuscript Library, Yale University
P.O. Box 208330
New Haven, CT 06520-8330
203.432.7364
matthew.gorham at yale.edu<mailto:matthew.gorham at yale.edu>
www.library.yale.edu/beinecke<http://www.library.yale.edu/beinecke>

Previous Message:

Hi Everyone,

Please feel free to log back in to ArchivesSpace PROD and resume your regular work. Thanks again for your patience, and apologies for the delay in completing the QC process. As I mentioned in my previous message, everything looks good overall after the upgrade, but we did encounter three errors during our testing of the application this morning that I want to make you aware of. The good news is that two of these errors have easy fixes, and the other one does not seem to be part of any system-wide issue.


1.       When a few of our testers tried to conduct searches in the Manage Top Containers interface, the search failed and they all received the same error message. The cause of this error turned out to be a Web browser cache issue, and after these folks cleared their browser caches, their searches were successful. So, as a precaution, it would be a good idea for everyone to clear their browser caches before they resume work in PROD.

2.       Two of our testers tried to edit and save archival objects within a specific resource record in the MSSA repository, and both received an error message and were not able to save their changes. This error is related to a resequencing issue in ArchivesSpace that has been known for a while, but will not be fully fixed until the production release of ArchivesSpace 2.0, because the fix involves a change in the overall architecture of the application. The good news is that we haven't encountered this error very often locally (Melissa Wisner confirmed that similar errors haven't officially been reported to Library IT in nearly a year), and we have a fix available for when it occurs on a case-by-case basis. There is a plug-in installed in PROD, only available to Repository Managers, that enables resyncing of individual resource records when this type of error occurs. To find this plug-in, click on the gear-shaped icon next to your repository's name from any page in ArchivesSpace, hover over Plug-ins in the drop-down menu, and choose Resync from the list of plug-ins. On the resulting page, enter the name of the resource record that you wish to resync into the typeahead, and click Queue Job. Again, this feature is only available to Repository Managers, so if you are a regular ArchivesSpace user, please consult with your Repository Manager if you need to use it. Melissa Wisner performed a resync of the record in which this error occurred this morning, and it solved the problem. Hopefully, given the infrequency of this error's occurrence, you won't even need to worry about this.

3.       One of our testers tried to move an archival object within a resource record in the MSSA repository, received an error message, and was not able to move the object. The folks at Lyrasis checked this out and determined that it is not a system-wide problem, and that it was likely just a one-off error with this particular record.

As always, please report any problems or errors you do encounter in ArchivesSpace to Library IT via ServiceNow (http://web.library.yale.edu/form/report-problem). Lastly, I want to thank my YAMS colleagues and Christy Tomecek from MSSA for their efforts today, our Lyrasis reps (who got up extra early on the West Coast to be on call for us), and especially Melissa Wisner, who worked tirelessly over the past few months to make sure that this upgrade would go smoothly.

All my best,
Matthew (on behalf of the Yale Archival Management Systems Committee)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.yale.edu/pipermail/yulaspace/attachments/20170417/d6261ccd/attachment-0001.html