Big Data With Spark And Scala Job in Cognisun
Big Data With Spark And Scala
Cognisun
4+ weeks ago
- Ahmedabad, Gujarat
- Not Disclosed
- Full-time
- Permanent
Job Summary
Qualification : BE/B.tech or ME/MTech
Job Description
We are looking for a Spark developer who knows how to fully exploit the potential of our Spark cluster.
You will clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts.
Responsibilities
- Create Scala/Spark jobs for data transformation and aggregation
- Produce unit tests for Spark transformations and helper methods
- Write Scala doc-style documentation with all code
- Design data processing pipelines
Skills
- Scala (with a focus on the functional programming paradigm)
- Scalatest, JUnit, Mockito, Embedded Cassandra
- Apache Spark 2.x
- Apache Spark RDD API
- Apache Spark SQL Data Frame API
- Apache Spark MLlib API
- Apache Spark GraphX API
- Apache Spark Streaming API
- Spark query tuning and performance optimization
- SQL database integration Microsoft, Oracle, Postgres, and/or MySQL
- Experience working with HDFS, S3, Cassandra, and/or DynamoDB
- Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
- This involves both ad-hoc requests as well as data pipelines that are embedded in our production environment.
Qualification : BE/B.tech or ME/MTech
Experience Required :
Minimum 3 Years
Vacancy :
2 - 4 Hires
Similar Jobs for you
×
Help us improve TheIndiaJobs
Need Help? Contact us