Data Transfer via AWS S3

Teena Blaydes -

Note: Only student files may be transferred via AWS s3. We also will *not* process any files through s3 until your team has completed testing and informed us that it's okay to enable production; This will allow your files to automatically process.

 

Why do you use Amazon S3?

AWS S3 is an industry standard for data storage and transfer. In order to provide the best security of your sensitive data, we use a rigorous access control policy and encryption in transit and at rest. Data can only be uploaded not read from AWS s3 and will only sit there for a short time before it is ingested into Handshake by an automated process.

What setup do I have do on my end?

You do not have to setup an s3 bucket or interact with amazon in any way other then using one of the many tools available to upload to Handshake’s s3 bucket and folder for your team. We will provide you with the folder, access id, secret key, and required bucket and prefix for uploading your file.

What if I can't install python to use the aws cli?

The AWS cli is just interacting with the AWS s3 rest api-- If you'd like, you can use any tool that can talk to a RESTful service such as curl. This isn't recommended because it opens up more opportunities for errors. See here for an example: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash

What sort of encryption is used with the data?

We use server side 256 bit AES. “Amazon S3 Server Side Encryption employs strong multi-factor encryption. Amazon S3 encrypts each object with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it regularly rotates. Amazon S3 Server Side Encryption uses one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.” All data sent in transit is transferred over TLS.

 

How do I transfer the data to my secure folder?

The easiest way to transfer data is with the AWS CLI (http://aws.amazon.com/cli/)

  • Install the AWS CLI
  • You should have received an ‘Access Key ID’ and a ‘Secret Access Key’ from the Handshake team.
  • Type “aws configure” in your command line prompt and follow the prompts to enter your Key ID and Secret Key mentioned above.
    • These should be entered precisely as received
  • Unless otherwise instructed by the Handshake team, you should use “us-east-1” as your region.
  • To help alleviate issues with sending your file to us, leave the output field as is
  • For more information see this article.

Once you have the CLI set up, uploading your data is as simple as issuing the following command:

$aws s3 cp [/path/your_local_file] s3://handshake-importer-uploads/[your folder]/[[yyyymmdd]_users.csv]

Handshake has one aws bucket ‘handshake-importer-uploads’ and every school has a unique folder which comes after the bucket which they only have write access to upload files.

Example upload:

aws s3 cp 20140410_users.csv s3://handshake-importer-uploads/importer-production-hudson_university/20140410_users.csv


Example response:

upload: to s3://handshake-importer-uploads/importer-production-hudson_university/20140410_users.csv


If you wish to do a test run, simply send a file and our data team will verify that it is received and ready for production.

 

Can I whitelist s3 though my firewall?

There are a lot of IPs for s3 but they are updated here on a regular basis: https://ip-ranges.amazonaws.com/ip-ranges.json (we only use the us-east-1 region)

If you do want to whitelist by domain:

https://s3.amazonaws.com/*

 

How can I update my previous aws cli setup if new keys are provided?

Just type "aws configure" in your command line prompt, then proceed with the steps as you did during the initial setup.

 

How will I know my file was successfully received by Handshake?

 

Upon request, we can enable your email for receipts of s3 transfers. This just confirms that we've received it and it let's you know if there are any issues (Example: There's a required header that's missing: username). 

 

Can I use s3 to upload other file types? (like majors or contacts)

This type of transfer is only intended for student syncs. If you'd like to upload another file type, please use the Importer App. If you do not have access to the Importer Application, please reach out to your Account Manager.

 

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk