Simple Archive Format Scripts

A set of scripts used to perform different tasks related to DSpace and involving SAF.

A script designed to find and remove duplicate documents specified in an SAF Batch import directory. This performs checksum operations on the files (md5sum) and removes files whose checksums match another file. This ensures that only one copy of a given file exists.

Example Usage: saf-dspace-deduplicate.sh source_directory.

A script designed to find incorrectly generated PDF files produced by SAFCreator. The invalid PDF files generally occur because remove websites provide invalid data (such as setting a mimetype to 'application/pdf', returning HTTP 200, and setting the contents of the provided PDF to 404 Not Found). This also helps identify duplicate PDFs.

Example Usage: saf-dspace-find_problematic_pdfs.sh source_directory.

A script for renaming the directories generated by SAFCreator into a name specified by a specific column in some mapping file. SAFCreator currently does not support naming directories by some column name, so this script bridges the gap.

Example Usage: saf-dspace-rename_directory.sh -m mapping.csv source_directory.

A script for removing duplicte PDFs identified by the saf-dspace-find_problematic_pdfs.sh script. The duplicates directory should be the checksums directory found in the write directory produced from the saf-dspace-find_problematic_pdfs.sh script.

Example Usage: saf-dspace-remove_problematic_pdfs.sh source_directory problems_directory.

• DSpace
• SAFCreator