Interactive

Projects

Core Competencies
Python AWS Glue SageMaker Lambda Power BI SQL DAX Bayesian Opt
AWS Pipeline

YouTube Trending Data Pipeline

Fully automated end-to-end data pipeline built on AWS that continuously ingests YouTube trending data via the YouTube Data API, cleans and transforms it through multiple stages, and stores it in structured format.

  • Implemented Medallion Architecture with Bronze, Silver, and Gold layers.
  • Used AWS Lambda for triggers, Glue for distributed ETL, and S3 for partition storage.
  • Orchestrated entire pipeline with Step Functions, achieving 100% automation.
AWS S3 Lambda AWS Glue Step Functions
3
Data Layers
100%
Automated
AWS
Cloud Native