Sr. Data Engineer Job in Infosenseglobal

Number of Vacancy : 1

Applicant must have: 4+ years Experience

Shift Time: 02:00 PM to 11:00 PM

Qualification: Bachelor s or master s degree in computer science.

Mandatory Skills:

• Spark, Python
• SQL/No SQL (SQL DB/Mongo DB)
• Airflow
• Django
• DevOps & CI/CD
• Communication and Leadership Skills

Preferred Skills:

• Programming Language: Python
• SQL & No SQL DB: PostgreSQL & Mongo DB
• Pipeline: Apache Airflow
• Process Framework: Spark
• Rest API: Django
• DevOPS, CI/CD: GitLab, Docker

Candidate Profile:

• Advanced working SQL knowledge and experience working with relational databases and No SQL databases
• Experience building and optimizing data pipelines, data model and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
• A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
• Working knowledge of Docker, Kubernetes and DevOPS
• Experience supporting and working with cross-functional teams in a dynamic environment.

We are looking for a candidate with 4+ years of experience in a Data Engineer role, they should also have experience using the following software/tools:

• Experience with object-oriented/object function programming language: Python.
• Experience with relational SQL and NoSQL databases, including Postgres and MongoDB
• Experience with data pipeline and workflow management tools: Airflow,Mlflow etc.
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience on Docker, Kubernetes
• Should be able to review design, code & provide review comments with justification
• Good experience with automated unit testing script and UAT support
• Zeal to learn new tools/technologies and adoption

Job Description Roles & Responsibilities:

• Create and maintain optimal data pipeline architecture
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Play role of Senior Data Engineer as individual contributor in Projects
• Agile delivery-Person must be able to show design and code on a daily basis
• Eager to learn New Technologies
• Help to cultivate good positive work culture in the team.