Data Engineer: Big Data-aws Job in Ibm India Pvt Ltd
Data Engineer: Big Data-aws
Ibm India Pvt Ltd
4+ weeks ago
- Cochin, Ernakulam, Kerala
- Not Disclosed
- Full-time
- Permanent
Job Summary
- Developed the Pysprk code for AWS Glue jobs and for EMR.. Worked on scalable distributed data system using Hadoop ecosystem in AWS EMR, MapR distribution..
- Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).
- Developed Hadoop streaming Jobs using python for integrating python API supported applications..
- Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD s were used to apply business transformations and utilized Hive Context objects to perform read/write operations..
- Re write some Hive queries to Spark SQL to reduce the overall batch time
Experience Required :
Fresher
Vacancy :
2 - 4 Hires
Similar Jobs for you
Help us improve TheIndiaJobs
Need Help? Contact us