Senior Data Engineer

As a Data Engineer with over 6 years of experience, you will be responsible for designing, building, and managing scalable and reliable data systems. Your proficiency with big data and stream-processing technologies such as Spark and Kafka will be crucial in ensuring the efficiency of data operations. Your hands-on experience with cloud platforms, particularly AWS services like EC2 and RDS, will enable you to leverage the full potential of cloud resources for data management. Key Responsibilities: - Designing and building data pipelines using tools like Airflow to orchestrate data flow efficiently. - Utilizing Databricks for scalable data processing and advanced analytics, enhancing data insights and decision-making processes. - Demonstrating strong knowledge of SQLMesh for modern data workflow management, ensuring streamlined data operations. - Integrating and working with external data sources via REST APIs, GraphQL endpoints, and SFTP servers to enrich the data ecosystem. - Demonstrating strong communication skills and leadership capabilities to effectively collaborate with cross-functional teams. Qualifications Required: - 6+ years of experience in a Data Engineering role, showcasing a deep understanding of data systems. - Proficiency in big data and stream-processing technologies such as Spark and Kafka. - Hands-on experience with cloud platforms, particularly AWS services like EC2 and RDS. - Skilled in building and orchestrating data pipelines using tools like Airflow. - Experience with Databricks for scalable data processing and advanced analytics. - Strong knowledge of SQLMesh for modern data workflow management. - Extensive experience integrating and working with external data sources via REST APIs, GraphQL endpoints, and SFTP servers. - Strong communication skills and leadership capabilities are required to excel in this role.,