Data Engineering Consultant Job in Gaininsights
Data Engineering Consultant
- Chennai, Tamil Nadu
- Not Disclosed
- Full-time
- Permanent
Data Engineering Consultant
Experience: 4+ years
Location:
Roles and Responsibilities
Interaction with Client for requirement gathering
Data Pipeline design and development
Develop and perform tests on all ETL codes
Performance tuning of the ETL jobs, SQL and Automation
Support and Maintenance of the ETL system, Data Warehouse, data marts & datalake
End to End Data Processing which includes Ingestion, Curation, Analytics (BI & ML)
Skills and Experience
Minimum 4 years of experience in designing and delivering complex, large-volume data warehouse/data lake using,
Talend Data Integration / Master data Management / SSIS
Azure Data Factory / AWS Glue / Python
SQL / No-SQL / Un Structured data handling / API Data Retrievals
Proficient in ETL & ELT Processes, SQL, PLSQL / T-SQL
Strong experience in Business Rules Validation, Source Target Mapping Design, Performance Tuning and High-Volume Data Loads
Must have experience in a complex Data Integration projects using heterogeneous source system (Database, SAP, Sales force, Oracle Fusion etc) with Transformation logic, Group, Normalize, Denormalize components, hierarchical execution, memory management etc
Experience in Optimization
Data store performance tuning / Query Optimization / Procedure Tuning
Data Pipeline optimization / Replicate (CDC) / RealTime Streaming Data (Kafka / Kinesis / Stream Analytics)
Working Knowledge / Previous experience on
source-code control using tools
knowledge of Core Java
Various Type of Data modelling
ETL Server installation / Configurations
Pipeline Execution design / Job Scheduling / Monitoring
CI -CD, Dev Ops
Strong knowledge of Software Development Lifecycle (SDLC)
Strong written and oral communication skills are essential
Ability to work independently and multi-task to meet critical deadlines in a rapidly changing environment
High energy, self-motivation, attention to detail, creativity, flexibility, and ability to work under pressure
Willing to learn new skills and implement new technologies
Minimum 4 Years
2 - 4 Hires
