Data Engineer- Pyspark Job in Xduce

Title: Data Engineer- Pyspark
Experience: 0-2 Years

Hands on experience in creating pipelines in Azure Data Factory using activities like Move &Transform, Copy, filter, for each, Get Metadata, Lookup, Data bricks etc
Hands on experience working with different file formats like json, csv, Avro, parquet etc.. using Databricks and Data Factory
In-depth knowledge of Spark Architecture including Spark Core, Spark SQL, Data Frames and Spark Streaming
Extensive knowledge and Hands on experience implementing cloud datalakes like Azure DataLake Gen1/Azure DataLake Gen2
Experience working with ETL process

Job Summary:

Data Engineer Data Mapping : The position is intended for Data Mapping of Insurance Business Systems Data (Policy/Claims/Billing) in Big Data/ (Azure) cloud environment to support the needs and objectives of the Organizational Units. The individual will perform mapping for migration of on-prem workloads to the cloud, and facilitate governance and streamlining operations of (Azure) cloud computing resources.
The role requires a self-starter that can be productive with minimal direction.

Key Responsibilities:

Specific KEY duties include but are not limited to:
Drives Cloud First strategy for all initiatives
Analyzes complex legacy systems and come up with cloud migration strategies with full stack solution designs
Collaborate with team to define overall data approach
Perform data mapping and data profiling of Insurance data
Confirm data readiness for feeds and conversion based on target system and business process requirements.
Data Lake/ Data Vault with IBM Insurance data model experience on Azure with Pyspark coding

Other Responsibilities:

Reviews project requirements, evaluating industry-strength XaaS solutions and providing architectural recommendation based on best-fit and/or best-of-breed considerations

Works in onsite/offshore model. Person will work closely with offshore delivery team to drive quality and deliverable.

Education and Experience:

Bachelor s degree (4-year) in Engineering or Computer Science; graduate degree is preferred
Minimum 10 years of experience in IT project delivery; must have at least 2 years hands-on Logical Data Mapping from Relational to Big Data/ Cloud

Nice to have (But not mandatory) Cloudera or AZURE Certification

Skills & Other Requirements:

Must have strong written and oral communications skills
Must have knowledge of IT project execution and delivery methodologies
Must have technical understanding of common applications, server software, and database technologies
Understanding of budget planning/development and cost allocation processes is desirable