Job Description
Should be able to create and maintain optimal data pipeline architecture. Bee able to work on large datasets and to meet functional and business requirements. Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Data Bricks, MS Azure Data Factory.
Should be able to understand existing implementation and should enhance wherever needed. Create and maintain document and communicate standard methods and tools used. Team player and should work with other data engineers, data ingestion specialists, and experts across the team.
Experience in performing root cause analysis on internal and external data and processes. Experienced using the following software/tools: Big data tools: Spark (Must), Hadoop (Not mandatory) Relational SQL and NoSQL databases, including COSMOS. Data pipeline and workflow management tools: DataBricks (Spark + Python), ADF, Dataflow Understanding and implementation knowle...
Should be able to understand existing implementation and should enhance wherever needed. Create and maintain document and communicate standard methods and tools used. Team player and should work with other data engineers, data ingestion specialists, and experts across the team.
Experience in performing root cause analysis on internal and external data and processes. Experienced using the following software/tools: Big data tools: Spark (Must), Hadoop (Not mandatory) Relational SQL and NoSQL databases, including COSMOS. Data pipeline and workflow management tools: DataBricks (Spark + Python), ADF, Dataflow Understanding and implementation knowle...