Data Engineer Job in Aptus Data Labs
Data Engineer
- Bengaluru, Bangalore Urban, Karnataka
- Not Disclosed
- Full-time
- Permanent
Big Data Platform Engineer Proven experience with deploying and tuning Open Source components into enterprise ready production tooling Experience with datacentre (Metal as a Service MAAS) and cloud deployment technologies (AWS or GCP Architect certificates required) Deep understanding of Linux from kernel mechanisms through user space management Experience on CI/CD (Continuous Integrations and Deployment) system solutions (Jenkins). Using Monitoring tools (local and on public cloud platforms) Nagios, Prometheus, Sensu, ELK, Cloud Watch, Splunk, New Relic etc. to trigger instant alerts, reports and dashboards. Work closely with the development and infrastructure teams to analyze and design solutions with four nines (99.99%) up-time, globally distributed, clustered, production and non-production virtualized infrastructure. Wide understanding of IP networking as well as data centre infrastructure Skills Expert with software development tools and source code management, understanding, managing issues, code changes and grouping them into deployment releases in a stable and measurable way to maximize production Must be expert at developing and using ansible roles and configuring deployment templates with jinja2. Solid understanding of data collection tools like Flume, Filebeat , Metricbeat, JMX Exporter agents. Extensive experience operating and tuning the kafka streaming data platform, specifically as a message queue for big data processing Strong understanding and must have experience: Apache spark framework, specifically spark core and spark streaming, Orchestration platforms, mesos and kubernetes, Data storage platforms, elasticstack, carbon, clickhouse, cassandra, ceph, hdfs Core presentation technologies kibana, and grafana. Excellent scripting and programming skills (bash, python, java, go, rust). Must have previous experience with rust in order to support, improve in house developed products Certification Red Hat Certified Architect certificate or equivalent required CCNA certificate required 3-5 years of experience running open source big data platforms
5 to 8 Years
2 - 4 Hires
