MIT SAP Dropbox

What is the Dropbox?

The dropbox is the standard method for data providers to automatically deliver files created outside of SAP as feeds into MITs SAP R/3 environments. Data providers must be registered with the dropbox. These providers FTP data files to the dropbox. Based on a combination of file naming conventions and registered dropbox userids, the files are automatically delivered to the appropriate MIT SAP R/3 environment.

Dropbox Files

For each feed, two files are delivered to the ftp drop box, the data file and the control file. The data file contains the data to be processed into MIT's SAP R/3 implementation. The control file consists of a single-record file whose purpose is to signal that the data file has been successfully and completely delivered. This control file is intended to detect situations where the data file may have been only partially transmitted for some reason (say, a communications failure). Therefore, the data file should always be transmitted first, and the control transmitted only after you know that the data has been sussessfully (and completely) transmitted.

The control file is a single-record file which contains control information in the standard format described below.

In situations where it would be extremely difficult to fulfill this preferred control file convention, alternative arrangements can be negotiated. In such cases delivery of control totals by an alternative mechanism would be imperative and encryption of the data file (see below) would become even more significant.

Control File Format

Standard Control record fields

The values in each of these six fields should be right-aligned numeric digits with leading zeros. No dollar signs, commas, or decimal points should be inserted. If a credit total is used, it should be in control field 1. If a debit total is used, it should be in control field 2. When the control totals are totals of monetary amounts, they should include cents (but without an explicit decimal point). The control file is required to be a minimum of 73 characters long (that is, a control file must contain at least the two count fields, the first two control fields, and a new-line character).

Encrypting Files

The dropbox expects data files to be encrypted by PGP.

Normally, public-key cryptography is used, but arrangements can be made to use conventional cryptography instead, if required. PGP's "ASCII-ARMORED" mechanism is strongly preferred, so that the file can be FTPed using FTP's text mode. However, if necessary, it might be possible to make arrangements to FTP PGP binary encrypted files. The data file should be encrypted with a MIT-supplied public key and signed with the data provider's private key. The data file should be encrypted as a text file and encoded with PGP's "ASCII-ARMORED" mechanism. The control file must not be encrypted. Files which decrypt properly will have been validated for completeness and accuracy as well as having a confirmation that they were provided by the organization they claim to have come from.

Please note that when the data file is encrypted, the byte count (stated in the control file) is of the encrypted (ciphertext) data file as it appears on MIT's SAP-DROPBOX system (a Unix system that uses a single end-of-line character to separate lines) and the record count (stated in the control file) is of the decrypted (plaintext) data file. If you transmit an ASCII file from a non-Unix system, you must use FTP's text mode. In this case, if your system doesn't use any character to separate lines, or uses a two or more character sequence to separate lines, then you will have to adjust the byte count from the byte count of the encrypted data file on your system. If your system also uses a single end-of-line character (for example, a UNIX or a Macintosh system), then the byte count should not need any adjustment. If your system uses a pair of characters for end-of-line (for example, a Windows system), then you would adjust the byte count by taking the byte count of the encrypted file on your system and subtracting the number of lines in the encrypted file. If your system doesn't use any end-of-line characters (for example, VM/CMS), then you would adjust the byte count by taking the byte count of the encrypted file on your system and adding the number of lines in the encrypted file.

File Naming Conventions

File names on the SAP-DROPBOX machine should be lower case, if possible. The names of the two files (the data file and the control file) must be identical except for the first character, which distinguishes between the two files. Note that the one-record control file is never encrypted, but since its file name must be identical to the file name of the data file (except that the first letter is a "c" instead of a "d"), it must end with ".pgp" or ".asc" extension if the data file has such an extention.

If the data file is sent as a PGP binary encrypted file, the extension ".pgp" should be added to the file names after the date&time stamp and if the data file is sent as a PGP ASCII-armored encrypted file, the extension ".asc" should be added to the file names after the date&time stamp.

Both the data and control file must carry the same date&time stamp. The date&time stamp may represent the time that the file was prepared or the time it was FTPed to the SAP-DROPBOX, but we strongly prefer that it represent the period to which the contents of the data file applies. If there is no relevance to the time of day, it may be "000000".

File Name Components

Sequence number

Each pair of (data + control) files must have a sequence number exactly one greater than the sequence number of the immediately previous pair of (data + control) files of the same feed type from the same data provider.

The sequence number is a three-digit right-aligned number with leading zeros. The value immediately following "999" is "000". There is a separate series of sequence numbers for each combination of data provider and feed.

Examples of Provider Codes

Provider codes will generally be suggested by the organization they represent, provided that the requested code is available. The following are provided as possible examples.

Examples of Feed Codes

Feed Codes will generally be specified by the MIT feed implementors. The following are provided as possible examples.

FTPing to the Dropbox

Ftp Site

The host name to ftp to is sap-dropbox.mit.edu .

File Destination

The user name & password to use for the ftp session depends on the identity of the data provider and the purpose and destination of the feed.

Getting Set Up

Contact the MIT Financial System Services Technical Services Team at Tech-Services

When (or shortly after) the Bridges Team sets up a set of Dropbox Accounts, they will need some information from you. Please be prepared to provide:

• How long will you want us to retain the input files on the Dropbox and SAP machines?
• Contact Information: Who should they contact in case of a technical difficulty with your incoming file or in case of a scheduled outage of the Dropbox function?
  • Name
  • Telephone number
  • E-Mail address

Dropbox File Retention

As you may know, the dropbox is the mechanism used to move files from non-SAP systems into (and out of) SAP. The method of doing this usually includes ftp'ing a file to the drop box machine, which in turn, moves that file to the target system (either PS1, SF2, or SF5). Since these files are archived after they are copied to the correct environment, as well as archived on the environment itself, we are using a lot of disk space. By default, the retention of the files will be as follows:

Dropbox:

Files received on the drop box for prod (PS1) will be retained for 7 days in the archive directory.

Files received on the drop box for tst1 (SF5) will be retained for 7 days in the archive directory.

Files received on the drop box for dev2 (SF2) will be retained for 7 days in the archive directory.

SAP:

Files in the archive directory will be retained for 60 days.

Files in the sapin, sapout, and saptmp directories will be retained for 60 days.

These files should be considered unrecoverable after this time. This file purging will be implemented for ALL systems that use the dropbox, even the ones that are either manually cleaned up or cleanup themselves (i.e. use the same filename, or delete old files), unless other arrangements are made.

If you know of a reason to keep these files around longer (or shorter) than the above, please let fss-infra@mit.edu know and we can set up retention cycles individually for that system.

Note: If you would like to keep files to meet general auditing requirements, we can copy the files into SAP Archive Server. Please answer the following questions and let fss-infra@mit.edu know.

1. Who (MIT Kerberos ID) can access the SAP Archive Serve? .
2. Which files to be moved to SAP Archive Server - Control file, Encryped file or Decryped file (any combinations)?
   

Retention for testing:

There are 2 ways that this can be accomplished. First the file can be retained and maintained outside of the SAP/Dropbox environment. By doing this the entire end to end process can be tested each time, by ftp'ing the file to the appropriate testing drop box. I think this is the best way, but if you do not want to maintain the file yourself, the other option is to provide me with the test file(s) and I will put the files in a holding area (one directory level up from the dropbox), and these files can be moved into the dropbox directory with a mail message to fss-infra.

If there are files that absolutely need to be kept around in the SAP environment (i.e. sapin or saptmp), then please send me the file names and I will be sure that the files are excluded from the automatic purge process.