Job Description
Become a Senior PySpark Engineer with Insight Global, emphasizing collaborative framework development in a hybrid setting. Transform how teams build scalable data solutions.
We are on the lookout for senior candidates with extensive knowledge of Spark internals. Your role will enhance existing frameworks, foster robust data engineering practices, and ensure clear communication with stakeholders. By bridging gaps in data architecture, you will drive innovative solutions within the team.
Key Responsibilities:
• Strengthen frameworks for team usage
• Design reusable libraries and patterns
• Optimize costs related to Spark SQL processing
• Effectively resolve production requirement ambiguities
• Facilitate operations on shared data platforms
Requirements:
• Strong grasp of PySpark internals
• Databricks hands-on experience required
• Familiarity with Delta Lake and Lakehouse
• Skills in orchestration via Azure Data Factory
• Exceptional communication ...
We are on the lookout for senior candidates with extensive knowledge of Spark internals. Your role will enhance existing frameworks, foster robust data engineering practices, and ensure clear communication with stakeholders. By bridging gaps in data architecture, you will drive innovative solutions within the team.
Key Responsibilities:
• Strengthen frameworks for team usage
• Design reusable libraries and patterns
• Optimize costs related to Spark SQL processing
• Effectively resolve production requirement ambiguities
• Facilitate operations on shared data platforms
Requirements:
• Strong grasp of PySpark internals
• Databricks hands-on experience required
• Familiarity with Delta Lake and Lakehouse
• Skills in orchestration via Azure Data Factory
• Exceptional communication ...