Lead Data Engineer

Role Overview: As the Team Lead of Data Engineering, you will be responsible for driving the design, development, and optimization of data engineering pipelines and services with a strong software engineering mindset. Your role will also involve fostering growth and collaboration within the team, mentoring junior colleagues, and refining processes to ensure excellence. If you are excited about making a lasting impact both technically and as a team leader, we are eager to meet you. Key Responsibilities: - Write maintainable, scalable code primarily in Python, with experience in Rust or C# considered a plus. - Process and manipulate data using Polars and Pandas, scaled via Apache Ray. - Utilize various open-source data storage technologies like S3, Apache Iceberg, DuckDB, Clickhouse, PostgreSQL, and others efficiently. - Implement containerization and orchestration tools such as Docker and Kubernetes for automated deployments in cloud environments (AWS, Azure, GCP). - Orchestrate complex workflows using open-source tools like Apache Airflow. - Work with big data and distributed processing technologies including Hadoop, Spark, and Kafka. Qualifications Required: - Strong educational background in Mathematics, Computer Science, Engineering, Physics, or a related field. - Minimum of 8 years of proven experience in Data Engineering, including data warehousing, ETL, automation, cloud technologies, or Software Engineering in data-related areas. - Experience in leading and mentoring junior team members to foster skill development, collaboration, and process improvements. - Ability to collaborate cross-functionally with software engineers, data scientists, and ML engineers to deliver high-impact solutions. Role Overview: As the Team Lead of Data Engineering, you will be responsible for driving the design, development, and optimization of data engineering pipelines and services with a strong software engineering mindset. Your role will also involve fostering growth and collaboration within the team, mentoring junior colleagues, and refining processes to ensure excellence. If you are excited about making a lasting impact both technically and as a team leader, we are eager to meet you. Key Responsibilities: - Write maintainable, scalable code primarily in Python, with experience in Rust or C# considered a plus. - Process and manipulate data using Polars and Pandas, scaled via Apache Ray. - Utilize various open-source data storage technologies like S3, Apache Iceberg, DuckDB, Clickhouse, PostgreSQL, and others efficiently. - Implement containerization and orchestration tools such as Docker and Kubernetes for automated deployments in cloud environments (AWS, Azure, GCP). - Orchestrate complex workflows using open-source tools like Apache Airflow. - Work with big data and distributed processing technologies including Hadoop, Spark, and Kafka. Qualifications Required: - Strong educational background in Mathematics, Computer Science, Engineering, Physics, or a related field. - Minimum of 8 years of proven experience in Data Engineering, including data warehousing, ETL, automation, cloud technologies, or Software Engineering in data-related areas. - Experience in leading and mentoring junior team members to foster skill development, collaboration, and process improvements. - Ability to collaborate cross-functionally with software engineers, data scientists, and ML engineers to deliver high-impact solutions.