Page History

The Archives Transfer Tool is currently a Dropbox based app for donors to transfer digital material (often for MIT community groups and members).

The system is a small collection of platforms/tools:

• a Dropbox Shared Folder into which files can saved by end users

• a folder on the NAS (Network Attached Storage) for preparation work on the files before ingest into preservation

• a Python script that moves files and metadata from Dropbox to the NAS

...

This step is done synchronously with the collections team. Once they create a submission agreement with the group, take the follow following steps:

1. Create If this is a group's first transfer: create a subfolder in the Dropbox shared folder linked above that will receive content and proceed to step 2. If the group has done at least one transfer already, proceed to step 9.
   

2. Add (including adding the authorized userwho will be doing a transfer as an editor of this subfolder)

3. Download the sample_default_metadata.json file and rename it to default_metadata.json 

4. Use a text editor to update the Create a default metadata file with information from the submission agreement that will apply to all the files that will be uploaded to the subfolder.
   

   1. You can edit or delete any field except the following that will be overwritten by the tool as a part of the transfer process:
      1. "Transfer Date"
      2. "Beginning Year"
      3. "Ending Year"
      4. "Description"
      5. "Dropbox SHA256"
5. If a field is not used (for instance, "campus-address"), delete the whole line
6. Save the json file and upload it to the Dropbox subfolder for the transfer
7. Download the sample_csv.csv file and upload it to the Dropbox subfolder for the transfer. This will be used by the person transferring the files to submit metadata.

8. On the NAS, create Create a subfolder within cdps-processing/ATT to receive the content copied by the Python script from Dropbox. Name it the same as the Dropbox subfolder.
   

9. Direct a user to follow the these steps outlined here to transfer the files.

10. When DDC has been notified that an authorized group transferred a set of files, confirm all have been uploaded to the associated folder in Dropbox

11. Files will now need to be downloaded from Dropbox to the NAS using a Python script before accessioning or processing can begin. The next steps below are where the process is handed over fully to the DDC processing team.
    

Getting situated with the ATT python app

1. Make sure that you are logged Log in to Dropbox in your default web browser
   

2. Download the latest version of the the ATT repository (which also means you need to have Python 3.x installed on your computer) to your workstation and followed follow the instructions in the README.

3. Mount the cdps-processing share on your workstation

Moving the files

1. Open Terminal (Mac/Linux) / PowerShell (PC) and navigate to the root of the downloaded copy of the interim-att folder.

2. Make sure that your environment is properly configured:

   1. you should have a .env file in the folder (see the README for details)

   2. you should have run make install (or make update) to get the rest of the environment configured

3. Run the “check” command (pipenv run att check)  to verify that the script can connect to Dropbox and connect to the NAS

   Code Block
   language bash linenumbers true collapse true
   interim-att % pipenv run att check Loading .env environment variables... 1. Go to: https://www.dropbox.com/oauth2/authorize?response_type=code&client_id=6pvw4x2047kxu4d&token_access_type=offline&code_challenge=<challenge_code>&code_challenge_method=S256 2. Click "Allow" (you might have to log in first). 3. Copy the authorization code. Enter the authorization code here: <auth_code_from_browser> 2025-06-06 12:04:46,793 INFO att.cli.check(): Successful Dropbox OAuth via PKCE 2025-06-06 12:04:47,775 INFO att.cli.check(): SUCCESS: Connected to MIT Dropbox 2025-06-06 12:04:47,777 INFO att.cli.check(): SUCCESS: NAS Folder is connected.

4. Run the “bulk-file-copy” command (pipenv run att bulk-file-copy), passing in the name of the CSV file (--remote-csv "DropBoxFolderName/csv_name.csv") as the required argument. You will need to include the Dropbox subfolder you are transferring from in the path to the CSV. For instance, AMITA/sample_csv.csv 

   Code Block
   language bash linenumbers true collapse true
   interim-att % pipenv run att bulk-file-copy --remote-csv "AMITA/sample_csv.csv" initial-application Loading .env environment variables...

5. Navigate to the ATT folder in the cdps-processing share to verify that the files were copied and that the metadata files and checksum manifest files were generated in the correct subfolder.

...

The downloaded folder is not organized in a way that will allow for processing in Archivematica. To do this, you will need to use another tool for logical transfer packaging such as SIPCreator or DART. You can follow the steps on those pages with specifics to content from ATT:

1. When packaging the files, select the content files only
2. Once the content has been packaged, add the original metadata files and manifest files to the submissionDocumentation folder as outlined in the section of the tool you used (SIPCreator or DART)
3. When done packaging the filesOnce complete, follow the next steps outlined in those documents.the respective tool's documentation.
   
4. Once fully complete, You should go back and clean out the subfolder in Dropbox. Make sure that when you are cleaning up the subfolder that you leave behind the default_metadata.json and sample_csv.csv files!

can be found in the "erst" folder of the submission drive. These instructions explain how to confirm the transfer and get it ready for processing.

If you have received a transfer from the archives transfer tool, it will already be in the erst folder in the submission drive within a folder by group or individual name.

1. Within the folder, each transfer will have a random unique ID and a sequential number within that. For example, a path in the erst folder might be: "Department of Urban Studies and Planning\21-10-2019.1571687161334.zBHEEP\21" where "21" is the folder with the actual content.
2. Review the contents of the transfer to ensure the material was what the donor intended to send.
3. If not correct, contact the donor to do another transfer or confer with the donor that extra material is alright to delete.
4. In order for Archivematica to be able to parse the package, you must move and edit some of the metadata files.
   1. Create a submissionDocumentation folder as outlined in the adding submission documentation to the bag section of the Exactly documentation.
      
   2. Move the “att-metadata.txt” file into the submissionDocumentation folder.
   3. Rename the “att-manifest-md5.txt” to checksum.md5 and move it into the metadata folder (the folder one level up from submissionDocumentation).
5. Change the name of the sequential number folder that contains the files and the metadata folder (for instance the "21" folder mentioned in the example of step 1) to the standard transfer package name related to its accession, in other words: accession number appended with acc, for example: 2020_034acc or if transferring multiple packages from the same accession, 2020_034_001, etc.
   
6. Proceed to the Next steps guidance at the start of this section.