Data Engineer

We are seeking an experienced Python ETL Developer to create, enhance, and optimize data pipelines. As the ideal candidate, you should possess in-depth knowledge of Python, PySpark, Airflow, and data processing frameworks. You should be proficient in working independently and have excellent communication skills in English. Your responsibilities will include developing and managing ETL pipelines using Python, NumPy, Pandas, PySpark, and Apache Airflow. You will be working on large-scale data processing and transformation workflows, focusing on optimizing performance and scalability. Collaboration with data engineers and business teams is essential to ensure smooth data flow. Troubleshooting and resolving ETL-related issues will be part of your routine to maintain data integrity and reliability. To qualify for this role, you should have over 8 years of Python experience, with at least 5 years dedicated to Python ETL development. Proficiency in PySpark, Apache Airflow, NumPy, and Pandas is required. Familiarity with SQLAlchemy and FastAPI is advantageous. Strong problem-solving abilities and the capacity to work autonomously are crucial. Effective English communication skills are necessary for seamless collaboration with global teams. Preferred qualifications include experience with cloud-based ETL solutions such as AWS, GCP, or Azure. Knowledge of big data technologies like Hadoop, Spark, or Kafka will be an added advantage. Please note that this job description is referenced from hirist.tech.,