Redshift Jobs

Need to setup amazon environment to run existing code so we can have a development environment ready to use. We have 3 repository of code scattered and need to trace and setup in amazon cli way to run. There are csv data files that are ftpped and there are scripts that check if a file is fully loaded then runs the ETL scripts that cleans it and then put into redshift. After in redshift need to...

So I need to have several processes run after it senses that the ftp upload is complete. 1. Check a directory to see if the files are finished uploading. (new) 2. After upload, check filesize and md5 3. Run some cleaning script. (existing scripts already, need to modify) 4. Run script to load to S3 (existing scripts already, need to modify) 5. Run scripts to load from S3 to redshift (existing scr...

Need to add/modify 4 scripts: 1. check ftp upload, file size, md5 checksum 2. run loading data into s3 buckets on aws 3. modify existing scripts that run data cleaning 4. modify existing scripts to load data into redshift. When bidding for this please tell me what does s3 stand for in the first sentence, if you don't then I know you are a robot and a copy/paste bidder and will no...

Need 3 process scripts and 1 modifications to existing python code to load data into redshift. - a process to pick up data that has been uploaded, once the upload is terminated (we could ask them to upload a dummy file at the end of the process, typically a manifest with all the expected file size, or even a md5sum) - a process to check the uploaded data, and clean it, then move it to S3 (what w...