Job Description
Purpose
A Data Engineer is responsible for building and maintaining efficient, reliable, and automated data pipelines and systems to ensure seamless data flow across the IBDF. This role focuses on ensuring data infrastructure availability, performance, and scalability through the use of automation and monitoring tools. A Data Engineer collaborates with cross-functional teams to optimize data operations and troubleshoot incidents.
Accountabilities
- Experience in building and maintaining data pipelines using tools like Apache Airflow, Spark, or BigQuery.
- Strong knowledge of database management systems (SQL, and NoSQL, e.g., PostgreSQL, MySQL, MongoDB).
- Familiarity with automation and monitoring tools to ensure data infrastructure performance (e.g., Cloud Logging, Apache Airflow).
- Knowledge of agile methodologies for project management and delivery.
- Excellent collaboration skills with operational and inf...