Lead Data Engineer

Lead Data Engineer

Exp: 8-12yrs

Location: Hyderabad

The Maps team is developing tools to analyze, visualize, process, manage and curate data on a large scale. Our team combines disparate signals such as data analytics, community engagement, and user feedback to improve the Maps Platform. On any given day you may be asked to analyze large data sets to identify errors in the map, design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into the data processing pipeline. We are seeking a highly experienced and technically adept Lead Software Engineer to guide our team in building low-latency, highly scalable, distributed systems that process large-scale data to power our products. The ideal candidate will be a hands-on leader, skilled at driving results through others, a mentor to engineers, and a key contributor to our technical strategy, with deep expertise in Scala, Python, Kubernetes, and the AWS ecosystem, and familiarity with Apache Spark.

What’s the role all about?

· Lead the design, development, and optimization of robust, high-performance distributed systems. · Drive the delivery of key initiatives by coordinating and guiding the work of other engineers on the team.

• Architect efficient schemas and algorithms for large-scale storage and retrieval of complex data.
• Make architectural decisions and define a long-term technology roadmap. Collaborate with product managers, engineering leads, and cross-functional partners to translate business requirements into technical solutions.
• Conduct code reviews and ensure the highest standards of code quality, performance, and maintainability.

· Troubleshoot and resolve complex production issues in distributed environments. · Stay current with emerging technologies and industry best practices, recommending adoption where appropriate.

What do we look for in you?

• 8+ years of experience in software engineering, with a focus on distributed, high-performance systems.
• Extensive, hands-on experience with Scala and a strong command of functional programming concepts.
• Proficiency in Python for backend development and data processing workflows
• Proven expertise with Kubernetes and AWS services (EKS, EMR, S3).
• Familiarity with Apache Spark for large-scale data processing.
• Solid understanding of distributed systems architecture, concurrency, and data modeling.

· Demonstrated ability to lead teams, coordinate work across multiple engineers, and deliver results. · Proficiency with version control (Git) and CI/CD pipelines.

Good to Have:

• Experience with geospatial data processing and indexing techniques (e.g., R-trees, H3).
• Exposure to computer vision pipelines and concepts in PyTorch or GenAI.

· Familiarity with event streaming platforms (Kafka) and distributed storage formats (Parquet, ORC). · Strong communication skills with the ability to articulate complex technical concepts to both technical and non-technical audiences.