Apache Airflow SME

Apache Airflow Subject Matter Expert (SME),

Skills:

• Optimize and fine-tune existing 

  Apache Airflow

   environments, addressing performance and reliability.
• Design and develop scalable, modular, and reusable 

  Airflow DAGs

   for complex data workflows.
• Integrate Airflow with cloud-native services such as 

  data factories, compute platforms, storage, and analytics

  .
• Develop and maintain 

  CI/CD pipelines

   for DAG deployment, testing, and release automation.
• Implement 

  monitoring, alerting, and logging

   standards to ensure operational excellence.
• Provide 

  architectural guidance

   and hands-on support for new data pipeline development.
• Document Airflow configurations, deployment processes, and operational procedures.
• Mentor engineers and lead 

  knowledge-sharing

   on orchestration best practices.
• Expertise in 

  Airflow internals

  , including schedulers, executors (Celery, Kubernetes), and plugins.
• Experience with 

  autoscaling solutions (KEDA)

   and 

  Celery

   for distributed task execution.
• Strong hands-on skills in 

  Python programming

   and 

  modular code development

  .
• Proficiency with 

  cloud services

   (Azure, AWS, or GCP), including data pipelines, compute, and storage.
• Solid experience with 

  CI/CD tools

   such as Azure DevOps, Jenkins, or GitHub Actions.
• Familiarity with Docker, Kubernetes, and related deployment technologies.
• Strong background in monitoring tools (Prometheus, Grafana) and log aggregation (ELK, Log Analytics).
• Excellent problem-solving, communication, and collaboration skills.

Interested? Please send your updated CV to  jobs.india@pixelcodetech.com and a member of our resource team will be in touch.