data-pipeline

2 articles

2025-05-15 DATA

Big Data on AWS Deep Dive (Part 6): End-to-End Data Pipeline — From Source to Feature Store

Connect all the dots: trace a click event from client SDK through API Gateway, MSK, Firehose, S3, warehouse layers (ODS→DWD→DWS→ADS), to DynamoDB for real-time serving.

2025-05-14 DATA

◆Big Data Deep Dive · Part 5

Big Data on AWS Deep Dive (Part 5): EMR, Glue ETL, Flink, and Pipeline Orchestration

Compare EMR Serverless, Glue ETL, Managed Flink, and choose the right compute engine. Then orchestrate data pipelines with MWAA (Airflow) and Step Functions.