Job
Description
Primary Skills: Airflow, Autosys, Python Good to have: Finance domain exp JD: 1. Design, implement, and maintain Airflow DAGs (Directed Acyclic Graphs) to automate ETL (Extract, Transform, Load) processes and other data workflows. 2. Configure and manage task dependencies, scheduling, retries, and error handling in Airflow. 3. Integrate Apache Airflow with different data sources, processing systems, and storage solutions (e.g., databases, cloud services, data lakes). Responsibility: 7+ years of experience in managing workflow automation and job scheduling, with at least 3 year focused on Apache Airflow. Strong background in Autosys (preferably 3+ years), including job scheduling, dependency management, and error handling. Experience migrating jobs from Autosys to Airflow and automating complex workflows. Familiarity with Airflow's components, such as operators, sensors, and hooks. Strong knowledge of Python for writing Airflow tasks and custom operators. Experience working with cloud environments (AWS, GCP, Azure). Solid understanding of SQL, databases, and data engineering concepts. Familiarity with containerization technologies (Docker, Kubernetes) is a plus. Experience with monitoring, alerting, and logging systems (e.g., Prometheus, Grafana).
