Find Jobs
Hire Freelancers

AWS Lambda / Python Guru - Re-Architecture

$750-1500 USD

Terminado
Publicado hace más de 6 años

$750-1500 USD

Pagado a la entrega
Each month we receive a CSV data file that has about 5-6k records. We need that data processed and put into a MySQL data base. Here is the background and workflow: 1) We receive the raw data file as a CSV (via email) 2) We have a script that uploads that CSV to a AWS S3 bucket (called raw_data) 3) A Python Lambda is triggered that iterates over each of the records in the file.....for each record we pull a unique identifier and use that identifier to call an external api and pull in other relevant infoation 4) the results from #3 are inserted into a AWS MySQL RDS instance. 5) Move the raw data file from the raw_data bucket to a processed_data bucket So if it's working what else do we need? We'll the max time a Lambda can run is 5 mins, and the script is timing out before if finished so we need to refactor it. What we would like, is to update it to the following pattern: 1 Master lambda that reads all of the records and fans out the processing via SQS (as 5-6k messages). Then have another lambda that is triggered that will only process one message at a time. The deliverables would include: 1) All source code (the current lambda python script would need to be updated) 2) Cloud Formation Templates 3) Configured security groups such that ports, etc aren't open to the world and the only the applicable resources can talk to one another (S3, SQS, Lambda, RDS, etc) Open to fair estimates for this project
ID del proyecto: 15775786

Información sobre el proyecto

23 propuestas
Proyecto remoto
Activo hace 6 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
Adjudicado a:
Avatar del usuario
Great idea! I just think Redis is faster than SQS since you want to use the queues just from s3. SQS is fine when you want to use the queues from outside of AWS. Relevant Skills and Experience - AWS Lambda, SQS, RDS - Redis, Celery - Ansible Proposed Milestones $950 USD - all
$950 USD en 10 días
4,8 (234 comentarios)
8,3
8,3
23 freelancers están ofertando un promedio de $1.168 USD por este trabajo
Avatar del usuario
Thank you for looking at our bid. Separate Lambada per record that would process (instead of one Lambda processing all records as being done currently), request your response. (guess externalizing api call through an independent Lambda would not make your current master process faster) Relevant Skills and Experience 100+ projects through Freelancer - Firebase, AWS, S3, dynamo, kindly visit our profile on Freelancer to review our past. Proposed Milestones $1500 USD - to be discussed Just need to review few things (1-2 days) & discuss before accepting award. We have an AWS dev account, may be inactive post trial period. We request your response to discuss.
$1.500 USD en 21 días
4,9 (79 comentarios)
7,9
7,9
Avatar del usuario
Hello, I'd be glad to complete this python project for you. Loking forward to chat with you soon for more details. Best regards, Relevant Skills and Experience Python , data processing, automation Proposed Milestones $882 USD - payment after project completion
$750 USD en 7 días
5,0 (180 comentarios)
7,4
7,4
Avatar del usuario
Hello sir I am a python developer with 7 years of experience. I am very familiar with aws s3, lambda, sqs and etc. I am confident with this job and can start the work right now. Best, Zhang Relevant Skills and Experience python, aws Proposed Milestones $888 USD - Re-Architecture
$888 USD en 10 días
4,9 (127 comentarios)
7,7
7,7
Avatar del usuario
I have read your project details. I have to ask a few questions. Can you please message me via chat so we can discuss all the details to elicit all the requirements and hence start the development? Relevant Skills and Experience I will showcase all the skills and my experience for this project over the chat. Please message me so we can resume this discussion. Stay tuned, I'm still working on this proposal.
$1.250 USD en 20 días
5,0 (42 comentarios)
7,6
7,6
Avatar del usuario
Hello, I always provide fastest results with quality,I have done more than 300 projects here..I have an experience of more than 10 years in web development and maintenance. Relevant Skills and Experience I have in-depth knowledge of php, mysql, javascript , jquery, paypal integrations, API's, css, html, html5, wordpress. I will look forward to discuss with you. Thank you. Proposed Milestones $988 USD - milestone
$988 USD en 20 días
4,6 (125 comentarios)
7,7
7,7
Avatar del usuario
Hi I am interested in this project. I am high skilled Python developer and know well about Amazon Lambda. I understood your requirements and ready to discuss. Let us discuss in detail Relevant Skills and Experience Python, Amazon Lambda Proposed Milestones $1411 USD - disucss
$1.411 USD en 20 días
5,0 (42 comentarios)
7,0
7,0
Avatar del usuario
Hi, .I specialise in web development. I live in Oxford. Your project looks exciting to me and I’ve got the skills required Relevant Skills and Experience Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture Proposed Milestones $1250 USD - project milestone I've done someting similar recently, will be happy to share.
$1.250 USD en 20 días
5,0 (18 comentarios)
6,1
6,1
Avatar del usuario
This bid is to modify an existing AWS lambda function by leveraging the SQS service and a sub-lambda to process the file incrementally in order to avoid timing out. Relevant Skills and Experience Python, AWS, S3, Data Processing, MySQL Proposed Milestones $1000 USD - Modify / test the existing Lambda function to overcome timeout issue.
$1.000 USD en 10 días
5,0 (44 comentarios)
6,3
6,3
Avatar del usuario
Thanks for sharing the details. I can help you with programming and infrastructure both. I have been dealing with these services since 14+ years. Relevant Skills and Experience PHP MySQL JAVAScript Ajax CSS HTML5 IOS Android Proposed Milestones $1250 USD - To be decided after discussion
$1.250 USD en 30 días
5,0 (51 comentarios)
5,9
5,9
Avatar del usuario
We love challenging opportunities. Looks like AWS configuration/services is a major issue here. We are clear about the requirements and our certified AWS solution architect can certainly solve it. Relevant Skills and Experience AWS Certified Developer Proposed Milestones $1323 USD - AWS Configuration + Python Script review Are you open to changes in AWS services being used?
$1.323 USD en 12 días
4,3 (51 comentarios)
6,8
6,8
Avatar del usuario
Amazon AWS Expert Team We are a competent team for AWS and proud member of Amazon AWS global partner network (APN) and a certified AWS service provider. Skills and Expertise We help our clients at every step of their Cloud journey including - Planning, Procurement, Deployment, Migration, Optimization, Support and security. Our managed service helps optimizing Cloud efficiently for Cost, Speed, Scale, Security and Automation. In order to understand more about your project requirements. Kindly initiate chat. Thanks! aastha
$750 USD en 7 días
4,9 (46 comentarios)
6,0
6,0
Avatar del usuario
Hello AWS Lambda / Python Guru - Re-Architecture Relevant Skills and Experience Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture Proposed Milestones $1250 USD - when i did it I have some questons for your project now thx
$1.250 USD en 20 días
4,9 (26 comentarios)
5,7
5,7
Avatar del usuario
Dear sir alredy done this type of job having 8 year exp in php , pls ping me for more detail am waiting your reply ! Relevant Skills and Experience 10 year exp in amazon web services Proposed Milestones $1666 USD - complete Stay tuned, I'm still working on this proposal.
$1.666 USD en 25 días
4,9 (30 comentarios)
5,8
5,8
Avatar del usuario
A cloud engineer with 2.5 years of hands-on experience with AWS clouds. Has delivered high quality, scalable, available, reliable and fault-tolerant solutions using AWS. seeks quality work Relevant Skills and Experience I have expertise in following areas 1. Lambda(Integration with SNS, SQS, S3, dynamodb, API gateway, Kinesis etc) 2. SQS(Decoupled architecture, SAM) 3. S3, EC2, ECS, Kinesis,Dynamodb etc 4. python etc Proposed Milestones $150 USD - Project Award $300 USD - First Milestone(deliverable tbd) $300 USD - Final Milestone(deliverable tbd) Can we discuss further and refine the architecture?
$750 USD en 20 días
5,0 (21 comentarios)
5,2
5,2
Avatar del usuario
I have been in this field for more than 7 years. I have experience in php,perl,wordpress,Html,postgress ,Web Services, Rest Api, Zend ,YIi,Mysql. Relevant Skills and Experience so I am right candidate for your project because I have experience in C#Aws,,Perl,php,node.js,Crypto currency,Wordpress,Ebay,Amazon,Instagram,Mysql,Mongo Db,Api projects. Proposed Milestones $1250 USD - AWS Lambda / Python Guru - Re-Architecture
$1.250 USD en 20 días
5,0 (20 comentarios)
5,4
5,4
Avatar del usuario
Hi, I am Python expert and have rich experiences with AWS. I can complete this project successfully. Hope to work with you. Stay tuned, I'm still working on this proposal.
$1.323 USD en 20 días
5,0 (11 comentarios)
4,8
4,8
Avatar del usuario
Can help you with this. Message me . I will deliver in 10 days. Relevant Skills and Experience more than 5 years experience in full stack development Proposed Milestones $1500 USD - code,deployment
$1.500 USD en 10 días
5,0 (6 comentarios)
4,0
4,0
Avatar del usuario
Hi, I am an experienced programmer and I'm sure I can complete your project asap. Please contact me and tell about your project details. I promise a high quality and punctual work. Thanks. AWS Lambda / Python Guru - Re-Architecture Amazon Web Services, Aws Lambda, MySQL, Python, Software Architecture,
$1.250 USD en 6 días
4,9 (8 comentarios)
4,2
4,2
Avatar del usuario
Lets introduce our company Gropse Technologiesthat has been in business for the last 5 years developing mobile apps and websites. We have an excellent track record for the best customer satisfaction. Stay tuned, I'm still working on this proposal.
$1.250 USD en 20 días
5,0 (7 comentarios)
4,2
4,2
Avatar del usuario
As a python expert, I will like to complete your project. we can discuss details over chat.
$750 USD en 5 días
0,0 (0 comentarios)
0,0
0,0

Sobre este cliente

Bandera de UNITED STATES
BALTIMORE, United States
5,0
97
Forma de pago verificada
Miembro desde jul 18, 2007

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.