Consultant (de_cons) Job in Themathcompany

Job Description

Group Company: TheMathCompany Pvt Ltd

Designation: Consultant (DE_CONS)

Office Location: Residency Road Bangalore

Years of experience: 6 to 8

Salary Range: INR to

Position description: As a data engineer, you ll have an opportunity to work on the universe of data and solve some very interesting problems by creating and maintaining scalable data pipelines dealing with petabytes of data. All our projects entail working on cutting edge technologies, petabyte scale data processing systems, data warehouses and data lakes to help manage the ever-growing information needs of our customers.

Primary Responsibilities:

• Experience in understanding and translating data, analytic requirements and functional needs into technical requirements while working with global customers
• Build and maintain data pipelines to support large scale data management in alignment with data strategy and data processing standards
• Experience in Database programming using multiple flavor of SQL
• Deploy scalable data pipelines for analytical needs
• Experience in Big Data ecosystem - on-prem (Hortonworks/MapR) or Cloud (Dataproc/EMR/HDInsight)
• Worked on query languages/tools such as Hadoop, Pig, SQL, Hive, Sqoop and SparkSQL.
• Experience in any orchestration tool such as Airflow/Oozie for scheduling pipelines
• Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow
• Understand and execute IN memory distributed computing frameworks like Spark (and/or DataBricks) and its parameter tuning, writing optimized queries in Spark
• Hands-on experience in using Spark Streaming, Kafka and Hbase
• Experience in any orchestration tool such as Airflow/Oozie
• Experience working in an Agile/Scrum development process
• 6+ years of experience in building data processing applications using Hadoop, Spark and NoSQL DB and Hadoop streaming
• BE/BS/MTech/MS in computer science or equivalent work experience.

Additional Responsibilities:

• Exposure to latest cloud ETL tools such as Glue/ADF/Dataflow is a plus
• Expertise in data structures, distributed computing, manipulating and analyzing complex high-volume data from variety of internal and external sources
• Experience in building structured and unstructured data pipelines
• Proficient in programming language such as Python/Scala
• Good understanding of data analysis techniques
• Solid hands-on working knowledge of SQL and scripting
• Good understanding of in relational/dimensional modelling and ETL concepts
• Understanding of any reporting tools such as Tableau, Qlikview or PowerBI

Reporting Team

• Reporting Designation:
• Reporting Department: Data Engineering (DE)
  

Educational qualifications preferred

• Category: Bachelor's Degree, Master's Degree
  
• Field specialization: Computer Science
  
• Degree: Bachelor of Engineering - BE, Bachelor of Science - BS, Master of Engineering - MEng, Master of Science - MS
  
• Academic score:
• Institution tier:

Required Certification/s:

Required Training/s:

Required work experience

• Industry:
• Role:
• Years of experience: to

Key Performance Indicators:

Required Competencies:

Required Knowledge:

Required Skills:

Required abilities

• Physical:
• Other:

Work Environment Details:

Specific requirements

• Travel:
• Vehicle:
• Work Permit:

Other details

• Pay Rate:
• Contract Types:
• Time Constraints:
• Compliance Related:
• Union Affiliation: