Daily Incremental load ETL pipeline for Ecommerce company using AWS Lambda and AWS EMR cluster, Deployed using Apache airflow in a docker container.
dockerdockerfileawspostgreslambdaaws-lambdadocker-composedocker-containeraws-s3lambda-functionspostgresqlaws-emraws-ec2redshiftoltpaws-emr-clustersaws-glueredshift-awsglue-jobslambda-trigger
-
Updated
Mar 17, 2023 - Python