Job Description
About The Role
We are seeking an experienced Python/PySpark developer with strong data engineering skills to design and deliver scalable, high‑performing applications that handle large datasets and complex data workflows. The ideal candidate is language‑agnostic, adaptable, and capable of picking up new technologies quickly. This role requires hands‑on development, unit/integration testing, and the ability to work in a self‑managed pod alongside other developers, technical leads, architects and a Product Owner.
Key Responsibilities
- Develop and optimize data pipelines for large‑scale datasets using Data‑Bricks. Ensure data quality, governance, and compliance across pipelines.
- Write clean, reusable and optimized PySpark code adhering to Data‑Bricks best practices.
- Implement unit, integration, and automated tests to ensure reliability. Collaborate with QA engineers to maintain a robust control framework.
- Apply CI/CD pri...