Job Description
Transform data architectures as a Senior Data Engineer in Toronto, ON, emphasizing AWS migration and modernization. Showcase your PySpark proficiency in a hands-on role that drives innovation.
You will spearhead the transformation of existing Databricks pipelines to AWS, focusing on both the lift-and-shift phase and subsequent optimization. This position demands a proactive approach to design scalable, consumable data products and deliver high-quality outputs from complex datasets.
Key Responsibilities:
• Migrate Databricks Spark workloads to AWS EMR
• Enhance and optimize data pipelines for performance
• Utilize Parquet and Iceberg to structure data
• Collaborate with teams to deliver data solutions
• Drive production readiness and deployment processes
Requirements:
• 8-10+ years of experience in Data Engineering
• Expert in Python and PySpark
• Strong experience in Apache Spark environments
• Knowledge of AWS ecosystem tools
• Ability to work au...
You will spearhead the transformation of existing Databricks pipelines to AWS, focusing on both the lift-and-shift phase and subsequent optimization. This position demands a proactive approach to design scalable, consumable data products and deliver high-quality outputs from complex datasets.
Key Responsibilities:
• Migrate Databricks Spark workloads to AWS EMR
• Enhance and optimize data pipelines for performance
• Utilize Parquet and Iceberg to structure data
• Collaborate with teams to deliver data solutions
• Drive production readiness and deployment processes
Requirements:
• 8-10+ years of experience in Data Engineering
• Expert in Python and PySpark
• Strong experience in Apache Spark environments
• Knowledge of AWS ecosystem tools
• Ability to work au...