Data Engineer Job in Mindpool Technologies Limited

Data Engineer

Apply Now
Job Summary

This is a remote position.

Description:

Sr. Data Pipeline Engineer



Job Description:

Responsibilities include understanding Data Pipeline & Data Engineering requirements, architecture design, development of Data Pipeline mapping and framework using GCP, SQL, Informatica, Python, Exasol. This is a technical position providing hands-on delivery role, working with the cross-functional teams, while ensuring excellent cross functional relationship.



Job Details:

Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google big data technologies.

Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Work with stakeholders including the BSA, Report developers, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Additional responsibilities include troubleshooting, maintenance, and optimization or enhancement of existing processes.

Partner with engineering leads and architects to define & coordinate technical design.

Design and code reviews to ensure standards and quality level for the build

Performance tuning of Data Pipeline jobs to meet SLA

Prepare technical documentations on the deliverables

Identify, define and implement best practices for process improvements for SDLC management



Experiences:

Platform, product & software experience: You have 6+ years of hands-on experience using SQL, GCP, Informatica, Python and version control (git) with good analytical & debugging skills. Good to have experience in Data flow, Exasol, big data infrastructure, real time streaming, CDC, Java, java script.

Data modeling: Flair for data and strong design skills in data model, PL/SQL, dimensional model/star & snowflake schema, data lake/mart/set and to bring efficiency in data modeling for efficient querying data for analysis, understands criticality TDD and develops data validation techniques.

Cross functional domain experience: Must have experience in cross functional domain and end to end knowledge of business function like CRM (Oppty, Quote, Order, Fulfilment, Asset) and Finance (AP, AR, Revenue).

Project management: You demonstrate excellent project and time management skills, exposure to scrum or other agile practices in JIRA.

Must possess excellent verbal and written communication skills. Must be able to effectively communicate & work with fellow team members and other functional team members to coordinate & meet deliverables.

CS fundamentals: You have earned at least a B.S. / MS in Computer Science, or related degree AND you have a strong ethos of continuous learning.

Must Have :

Software engineering & Architecture :
Python / Java, version control (git), REST API, analytical & debugging skills

Big Data :
Google Cloud Platform, Big Query, Data Flow

Data Modeling :
Data modeling, PL/SQL, star & snowflake schema design, in-memory

Real Time Systems :
Google pub/sub, GCP technologies, Kafka

Good To Have:

Software engineering & Architecture :
PySpark, Apache Beam

Big Data :
AWS/Azure, Hadoop Eco System, HDFS, Spark

Data Modeling :
NoSQL & indexing technologies

Real Time Systems :
AWS/Azure streaming technologies, Storm, Spark Streaming

Viz. Tools :
Tableau, PowerBI

Data Science/ML Ops :
Teraform, Ansible, Kubeflow, Google AI Platform

Leadership Competencies:



Collaborative: Facilitates effective collaboration among coworkers and external partners by encouraging to work together as a team, to share their honest views and reacts in a non-defensive way, involves others as appropriate when making decisions on behalf of the team. Makes sure that both individuals and the team receive credit for contributions and accomplishments.



Cultivates Innovation: Motivates others to find new and better ways to address challenges. Encourages people to incorporate varied perspectives to strengthen the innovation process. Provides support and resources for promising ideas through prototyping and experimentation. Challenges the team/ others to develop breakthrough solutions.



Drives Results: Emphasizes the importance of achieving results; challenges poor outcomes or unproductive behaviors in workgroups. Provides assistance and encouragement to help workgroups/ project groups over obstacles. Takes steps to promote urgency in workgroups to reach goals and meet deadlines. Generally, overcomes obstacles and delivers results through a team.



Other Leadership Competencies:

People

Influencing People: Communicates Effectively, Drives Engagement, Drives Vision and Purpose, Organizational Savvy, Persuades

Optimizing Diverse Talent: Builds Effective Teams, Develops Talent

Results

Taking Initiative: Action Oriented, Resourcefulness

Self

Being Authentic: Instills Trust

Being Flexible and Adaptable: Being Resilient, Manages Ambiguity, Nimble Learning

Being Open: Demonstrates Self-Awareness, Self-Development, speak up & step up.

Thought

Making Complex Decisions: Balances Stakeholders, Decision Quality

Understanding the Business: Business Insight, Customer Focus, Financial Acumen, Tech Savvy

Experience Required :

Fresher

Vacancy :

2 - 4 Hires

Similar Jobs for you

See more recommended jobs