We need a Linux comandline program, which combines several S3 objects into one.
## Deliverables
# BlobMerger
## Technologies used
It is a Linux shell application, which works with TCP sockets and HTTP protocol.
Application has just one thread. **Multiplexing** socket handle mode is expected.
## Description
Application receives merge tasks through STDIN. Task will contain *signed* urls on the file parts, and *signed* POST request parameters, required to create new S3 file.
File combining should be done **without** file buffer in the local filesystem.
In-memory buffer should be used, with size of 4 Kb or more. The exact buffer size to use is set in the config file.
Combining is a simultaneous reading one file part and writing the result file.
## Specification
Application reads data from STDIN in the following format:
**ACTION <session_id|QUIT>** (message **ACTION QUIT** shuts down the application)
**DESTURL**
**DESTPARAMCOUNT**
(sequence of exact *DESTPARAMCOUNT* params)
**DESTPRAM <name=value>**
**DESTPRAM <name=value>**
**DESTPRAM <name=value>**
**…**
**SOURCECOUNT**
(sequence of exact *SOURCECOUNT* SOURCE entities)
**SOURCE <????? ? s3>**
**SOURCE <????? ? s3>**
**SOURCE <????? ? s3>**
**…**
Once all message is recieved, application writes to the STDOUT message
**WAIT**
Message handling is started.
Once task is completed, application writes to the STDOUT message
**DONE **
After that application is waiting for the next task.
Or
**ERROR** ??" if error happened and application is going to exit with error. Error description should contain all required information and be human-readable.
## Implementation
* Application should use select() with two filehandles : GETHANDLE and PUTHANDLE ??" one for reading, another one for writing.
* Application should write into the logfile all commands, that were recieved on STDIN and wrote on STDOUT, replacing value of Signature with asterixes everywhere.
* In case of connection for read file part is failed, application should retry three more times, if still no success, then application should exit with error.
* In case od connection for write result file is failed, application should exit with error.
* While interactiong with S3, application should check HTTP Statuses of the S3 responses and exit with error, once status is unexpected.
* Exiting with error: Application should write error to the logfile first. If the last command written to STDOUT was WAIT, than error message should be written to the STDOUT.
* Log file entries should have the following format:
**YYYY-MM-DD HH:MM PID sessionId message**
* Application should write two log files simultaneously: [login to view URL] and blob_merger_session_id.log.
If application finishes its work on current session successfully and there were no errors during the session, than [login to view URL] should be deleted.
* Application should read config text file, where buffer size and log files directory path are defined. Config file should be at ~/.blob_merger