Job Description
Job Summary We are seeking a skilled Azure Databricks Engineer with strong experience in Azure Data Factory (ADF) to design, develop, and maintain scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate will have expertise in big data processing, ETL/ELT workflows, and distributed computing using Databricks.
Key Responsibilities
Design and implement scalable data pipelines using
Azure Data Factory (ADF)
Develop and optimize data processing workflows using
Azure Databricks (PySpark/Scala)
Build ETL/ELT processes for ingesting, transforming, and loading large datasets
Integrate data from multiple sources such as APIs, databases, and data lakes
Work with
Azure Data Lake Storage (ADLS)
and
Azure SQL Database
Monitor, troubleshoot, and optimize pipeline performance and data quality
Implement data security, governance, and compliance st...
Key Responsibilities
Design and implement scalable data pipelines using
Azure Data Factory (ADF)
Develop and optimize data processing workflows using
Azure Databricks (PySpark/Scala)
Build ETL/ELT processes for ingesting, transforming, and loading large datasets
Integrate data from multiple sources such as APIs, databases, and data lakes
Work with
Azure Data Lake Storage (ADLS)
and
Azure SQL Database
Monitor, troubleshoot, and optimize pipeline performance and data quality
Implement data security, governance, and compliance st...