Senior Software Qa Automation Engineer Big Data Job in Pubmatic

Job Description

PubMatic is one of the leaders in tech stack when it comes to Big data infrastructure and data processing. We at PubMatic process more than 150 Bn ad impressions a day which contribute to around 100 tera bytes of uncompressed data. To maintain and process this raw data we have our own data centres across the globe and do in house ingestion, ETL and aggregation using a mammoth Hadoop infrastructure on top of thousands of BareMetal nodes built from scratch. Having most things built in house, PubMatic is a very early adaptor of new technologies that come along in the big data space.

We are looking for individuals with knowledge and experience of working with distributed environments for becoming a part of our big data test engineering group. The individual will also be responsible for automating various big data flows, performance engineering and fine tuning the big data pipeline and making sure the quality of the above mentioned giant business critical infrastructure is intact. Individual will also get an opportunity to work with the high speed GPU machines used for rapid computing of large data sets.

Requirements:

• Should have minimum 3 years of experience on working BigData technologies
• Good Programming skills.
• Hands on Experience in Automating Backend Applications (e.g. db, REST API's)
• Hands on experience with Automating any backend applications (e.g db , server side).
• Knowledge of relational databases and SQL
• Good debugging skills.
• Strong working experience working in Linux/Unix environment.
• Strong understanding of testing methodologies.
• Hands on experience in working on Big Data technologies like Hadoop, Spark
• Hand on experience in working with ETL Testing
• Hands on experience in QA Automation Framework development & Design & Strong hold on data structures.
• Preferred language Python/Shell Scripting
• Strong Understanding of OS and performance benchmarking
• Quick learner and good team member with positive attitude.
• Good verbal and written communication skills.

Duties and Responsibilities:

• Testing big data ingestion and aggregation flows using spark shell and related queries
• Developing automation framework using programming languages such as python and automate the big data workflows such as ingestion, aggregation, ETL processing etc
• Debugging and troubleshooting issues within the big data ecosystem
• Set up the Big data platform and Hadoop ecosystem for testing
• Define test strategy and write test plan for the data platform enhancements and new features/services built on it.
• Define the operating procedures, service monitors and alerts and work with the NOC team to get them implemented.
• Responsible for system & performance testing of the data platform and applications
• Solve problems and establish plans and provide technical consultation in the design, development and test effort of complex engineering projects
• Review product specifications and write test cases, develop test plans for assigned areas.
• Identifies issues and technical interdependencies and suggest possible solutions.
• Recreate complex customer and production reported issues to determine root cause and verify the fix.