User Tools


Creating a S3 record set

In order to proceed with this step, all the files must be uploaded in DragonDisk (S3).

A S3 record set is just a 2-column .txt file (tab delimited) that contains the unique identifier of the book in one column and the S3 file path in the other column. The unique identifier is that number you see in the filename, and it's also in the book's MARC record in the 037$a field. The S3 record set allows us to connect the files in S3 with their records in Drupal by using the unique identifier.

  1. Open DragonDisk
  2. Navigate to the folder where the files are stored. They will be in the download.nnels.ca directory in the CA > NNELS folder.
  3. Once you're inside the folder where all the files are, select all files (Command+A on a Mac) and then right-click and select Copy URL to Clipboard
  4. Paste the content into a .txt file (i.e. into TextEdit on Mac, or Notepad on Windows)
  5. You'll need to modify the .txt file. The best way to do this is to open the .txt file in a Text Editor that lets you use regex or regular expressions. We recommend Sublime Text or TextWrangler. Carry out the 2 Find & Replace operations below:

First, replace part of the file path URL:

Find: http://download.nnels.ca.s3.libraries.coop
Replace: s3:/

Second, extract the unique identifier from the file path and move it into the left column:

Find: (s3://CA/NNELS/)([A-z].*_)(\d{1,13})(.*)
Replace: \3\t\1\2\3\4

Note that you made need to alter the regex to fit the file path pattern that you're working with.

You should now have 2 columns: on the left there should be the unique identifier and on the right there should be the file path (separated by a tab).

The S3 record set is ready to upload to Drupal. Move on to Uploading a S3 record set.

public/nnels/publishing/create-s3.txt · Last modified: 2018/08/16 22:06 (external edit)