Blobs:BlobMerger

We need a Linux comandline program, which combines several S3 objects into one. ## Deliverables # BlobMerger ## Technologies used It is a Linux shell application, which works with TCP sockets and HTTP protocol. Application has just one thread. **Multiplexing** socket handle mode is expected. ## Description Application receives merge tasks through STDIN. Task will contain *signed* urls on the file parts, and *signed* POST request parameters, required to create new S3 file. File combining should be done **without** file buffer in the local filesystem. In-memory buffer should be used, with size of 4 Kb or more. The exact buffer size to use is set in the config file. Combining is a simultaneous reading one file part and writing the result file. ## Specification Application reads data from STDIN in the following format: **ACTION <session_id|QUIT>** (message **ACTION QUIT** shuts down the application) **DESTURL** **DESTPARAMCOUNT** (sequence of exact *DESTPARAMCOUNT* params) **DESTPRAM <name=value>** **DESTPRAM <name=value>** **DESTPRAM <name=value>** **…** **SOURCECOUNT** (sequence of exact *SOURCECOUNT* SOURCE entities) **SOURCE <????? ? s3>** **SOURCE <????? ? s3>** **SOURCE <????? ? s3>** **…** Once all message is recieved, application writes to the STDOUT message **WAIT** Message handling is started. Once task is completed, application writes to the STDOUT message **DONE ** After that application is waiting for the next task. Or **ERROR** ??" if error happened and application is going to exit with error. Error description should contain all required information and be human-readable. ## Implementation * Application should use select() with two filehandles : GETHANDLE and PUTHANDLE ??" one for reading, another one for writing. * Application should write into the logfile all commands, that were recieved on STDIN and wrote on STDOUT, replacing value of Signature with asterixes everywhere. * In case of connection for read file part is failed, application should retry three more times, if still no success, then application should exit with error. * In case od connection for write result file is failed, application should exit with error. * While interactiong with S3, application should check HTTP Statuses of the S3 responses and exit with error, once status is unexpected. * Exiting with error: Application should write error to the logfile first. If the last command written to STDOUT was WAIT, than error message should be written to the STDOUT. * Log file entries should have the following format: **YYYY-MM-DD HH:MM PID sessionId message** * Application should write two log files simultaneously: [login to view URL] and blob_merger_session_id.log. If application finishes its work on current session successfully and there were no errors during the session, than [login to view URL] should be deleted. * Application should read config text file, where buffer size and log files directory path are defined. Config file should be at ~/.blob_merger