Real-time AWS ecommerce data pipeline using MSK Serverless, MSK Connect, S3, AWS Glue, Delta Lake, SCD Type 2, and Redshift Serverless.
aws apache-spark aws-s3 pyspark data-engineering aws-ec2 apache-kafka aws-glue delta-lake amazon-msk scd-type-2 dimensional-modeling data-lakehouse elt-pipeline medallion-architecture incremental-loading amazon-redshift-serverless
-
Updated
Aug 18, 2026 - Python