BriefCase
BriefCase

Data Engineer Job in Aptus Data Labs

Data Engineer

Aptus Data Labs 4+ weeks ago
Apply Now
Job Summary

Big Data Platform Engineer

Roles & Responsibilities

  • Proven experience with deploying and tuning Open Source components into enterprise ready production tooling Experience with datacentre (Metal as a Service MAAS) and cloud deployment technologies (AWS or GCP Architect certificates required)

  • Deep understanding of Linux from kernel mechanisms through user space management

  • Experience on CI/CD (Continuous Integrations and Deployment) system solutions (Jenkins).

  • Using Monitoring tools (local and on public cloud platforms) Nagios, Prometheus, Sensu, ELK, Cloud Watch, Splunk, New Relic etc. to trigger instant alerts, reports and dashboards. Work closely with the development and infrastructure teams to analyze and design solutions with four nines (99.99%) up-time, globally distributed, clustered, production and non-production virtualized infrastructure.

  • Wide understanding of IP networking as well as data centre infrastructure

    • Skills

  • Expert with software development tools and source code management, understanding, managing issues, code changes and grouping them into deployment releases in a stable and measurable way to maximize production Must be expert at developing and using ansible roles and configuring deployment templates with jinja2.

  • Solid understanding of data collection tools like Flume, Filebeat , Metricbeat, JMX Exporter agents.

  • Extensive experience operating and tuning the kafka streaming data platform, specifically as a message queue for big data processing

  • Strong understanding and must have experience:

  • Apache spark framework, specifically spark core and spark streaming,

  • Orchestration platforms, mesos and kubernetes,

  • Data storage platforms, elasticstack, carbon, clickhouse, cassandra, ceph, hdfs

  • Core presentation technologies kibana, and grafana.

  • Excellent scripting and programming skills (bash, python, java, go, rust). Must have previous experience with rust in order to support, improve in house developed products

    • Certification

    Red Hat Certified Architect certificate or equivalent required CCNA certificate required 3-5 years of experience running open source big data platforms



    Experience Required :

    5 to 8 Years

    Vacancy :

    2 - 4 Hires

    Similar Jobs for you

    See more recommended jobs