#GridDynamics #AWS #cloudPatterns #Lambda #ETL #SERVERLESS

Dynamic Talks Meetup 2019

2019 Saint Petersburg, Russia office Grid Dynamics
Build and automate serverless ETL job orchestration with AWS Lambda
"Serverless patterns and best practices ETL pipeline orchestration"
With the growth of steps in Extract Transform Load (ETL) jobs the complexity of managing data flow orchestration is growing. While serverless computing allow us not to worry about infrastructure and scaling, calculations are performed on the distributed nodes of the pipeline, the number of which can reach thousands in scale. This complicates the control of completion of the data processing steps and the ETL job in general. And in this case automation of ETL plays a critical part in defining and reporting processing status and job completion.
Video in RU
53 minutes. You can watch it on youtube
Slides in EN
36 slides.