Job Description

Responsibilities: Bachelor's degree in Computer Science, Engineering, or a related field. 5+ years of experience in data engineering, ELT development, and data modeling. Proficiency in using Apache Airflow, Dolphin Scheduler, and Spark for data transformation, integration, and management. Experience in implementing workflow orchestration using tools like Dolphin Scheduler, Apache Airflow, SSIS, or similar platforms. Demonstrated experience in developing custom connectors for data ingestion from various sources. Strong understanding of SQL and database concepts, with the ability to write efficient queries and optimize performance. Experience implementing DataOps principles and practices, including data CI/CD pipelines. Excellent problem-solving and troubleshooting skills, with a strong attention to detail. Knowledge of data streaming and real-time data processing technologies (e.g., Apache Kafka). Strong understanding of software development principles and practices, including version control (e.g., Git) and code review processes. Experience with Agile development methodologies and working in cross-functional Agile teams. Ability to adapt quickly to changing priorities and work effectively in a fast-paced environment. Excellent analytical and problem-solving skills, with a keen attention to detail. Strong written and verbal communication skills, with the ability to effectively communicate complex technical concepts to both technical and non-technical stakeholders.