Job Description
Title: Devops Engineer Databricks+AWS
Duration: Long term
Location: Remote /Canada
Job Description:
About the role
Client is migrating 100+ Databricks pipelines (batch and streaming) from the AWS West environment into AWS East Databricks workspace. Four Data Engineers are handling pipeline migration. This role owns the infrastructure, CI/CD, and deployment layer that underpins all of them.
What you'll own
- Design and maintain Terraform modules using the Databricks Terraform provider to provision and manage jobs, clusters, cluster policies, permissions, secret scopes, and instance profiles must-have
- Build and maintain GitHub Actions CI/CD workflows for automated Databricks job deployment across dev, staging, and production environments must-have
- Configure and manage Databricks Asset Bundles (DABs) as an IaC deployment mechanism alongside Terraf...