BriefCase
BriefCase

Azure (adf+adb+pyspark+sql) Job in Rarr Technologies

Azure (adf+adb+pyspark+sql)

Rarr Technologies 4+ weeks ago
Apply Now
Job Summary
  • Requirement is for an Azure Data Engineer who has hands on Experience in Azure Platform, Azure Data Lake, Azure Data Factory, Data Bricks, CI/CD.
  • The Data Engineer is responsible for identifying relevant data to be gathered and implementing a solution to do so, including knowledge of where to get the data, how to extract it in the right format, and how to augment the data to support data science/analysis requirements. Able to build a robust, fault-tolerant data pipeline that cleans, transforms and aggregates un-organised and messy data into databases or data sources. Able to ensure that data flows smoothly from source to destination so that it can be processed.
  • Job Description:
  • The Data Engineer is an expert in SQL development support to the data and analytics IT experts in database design, data flow and data analysis activities. The Data Engineer plays a key role in the development and deployment of innovative big data platforms for advanced analytics and data processing.
  • The Data Engineer defines and builds the data pipelines that will enable faster, better, data-informed decision-making within the business.
  • Deep experience in developing enterprise data management strategies including operational data stores, data warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery.
  • Work together with the Solution- and Data Architect to design the next generation big data platform to support the digital marketing platform.
  • Design and implement a new big data solution which supports high amounts- and velocity of data, supports future growth, use latest big data technologies/techniques, is cloud supported and in-line with Shell's strategies.
  • Design and implement modern data pipelines to extract, clean and process data in batch and real-time from different data sources.
  • Use the latest development, testing and deployment techniques to quickly deploy new releases to e.g. deploy new data pipelines and add data sources.
  • Help establish best practices around data governance, data security- and privacy (GDPR).
  • Work together with the solution- and data architect to the define the target data strategy.
  • Drives and applies Agile Framework within his/her operating environment. Will take direct reports, when applicable, through Agile Framework and its respective tools.
Experience Required :

5 to 13 Years

Vacancy :

2 - 4 Hires

Similar Jobs for you

See more recommended jobs