Devops Engineer

DevOps Engineer

Location:

About the Role

We are hiring a DevOps & Infrastructure Engineer to lead the deployment, scaling, and maintenance of our on-premises open-source data platform. This is a hybrid role requiring expertise in both infrastructure setup (Linux servers, networking, storage) and DevOps practices (automation, CI/CD, monitoring).

You will be responsible for ensuring high availability, performance, and security of the full data stack—from ingestion tools to the data warehouse—along with supporting pipeline execution and system observability. The ideal candidate should have experience working in data-intensive environments and be hands-on with open-source technologies.

Key Responsibilities

• Deploy, configure, and maintain the core components of the on-prem data platform (e.g., PostgreSQL, Kafka, MinIO, Airflow, columnar warehouses like Apache Doris/ClickHouse)
• Automate infrastructure provisioning using tools like Ansible, Terraform, or shell scripting
• Set up monitoring, alerting, and logging across the data stack (e.g., Prometheus, Grafana, Loki, ELK)
• Manage network, firewall, TLS/SSL, load balancers (e.g., HAProxy, NGINX), and DNS configurations
• Ensure secure access to services using authentication (LDAP/SSO) and role-based access control
• Manage backup, restore, and disaster recovery strategies for critical components (DBs, object storage, etc.)
• Collaborate with Data Engineers to provide runtime environments for ingestion, processing, and analytics workflows
• Optimize system resource usage, schedule maintenance windows, and handle upgrade paths for open-source components

Required Skills & Qualifications

• 3–5 years of hands-on experience in DevOps, infrastructure engineering, or system administration
• Strong Linux system administration skills (Ubuntu/CentOS preferred)
• Experience with provisioning and configuration management tools such as Ansible, Terraform, or similar
• Proficient with Docker and basic container networking and orchestration (e.g., Docker Compose, optional K3s/K8s exposure)
• Knowledge of network fundamentals, firewalls, load balancers, DNS, and secure service communication
• Experience with monitoring stacks (Prometheus, Grafana), log management (Loki, Fluentbit, or ELK)
• Familiarity with open-source data tools such as PostgreSQL, Kafka, MinIO, Airflow, and columnar OLAP engines
• Ability to troubleshoot performance bottlenecks across system, storage, or network layers
• Proficient in scripting languages (Bash, Python preferred) and Git-based workflows

Nice-to-Have Skills

• Exposure to CI/CD pipelines (GitLab CI, Jenkins, or GitHub Actions)
• Experience in managing data warehouse systems like Apache Doris, ClickHouse, or Trino
• Understanding of distributed storage, object storage internals, and snapshot-based backups
• Familiarity with SSL certificate management and service hardening best practices
• Experience setting up identity providers (LDAP, Keycloak, or OpenID-based SSO)