Lead Big Data Engineer

Role & responsibilities

• Sound knowledge in Spark architecture and distributed computing and Spark streaming.
• Proficient in Spark including RDD and Data frames core functions, troubleshooting and performance tuning.
• Good understanding in object-oriented concepts and hands on experience on Kotlin/Scala/Java with excellent
• programming logic and technique.

• Good in functional programming and OOPS concept on Kotlin/Scala/Java
• Good experience in SQL
• Managing the team of Associates and Senior Associates and ensuring the utilization is maintained across the project.
• Able to mentor new members for onboarding to the project.
• Understand the client requirement and able to design, develop from scratch and deliver.
• AWS cloud experience would be preferable.
• Experience in analyzing, re-architecting, and re-platforming on-premises data warehouses to data platforms on cloud
• (AWS is preferred)
• Leading the client calls to flag off any delays, blockers, escalations and collate all the requirements.
• Managing project timing, client expectations and meeting deadlines.
• Should have played project and team management roles.
• Facilitate meetings within the team on regular basis.
• Understand business requirement and analyze different approaches and plan deliverables and milestones for the project.
• Optimization, maintenance, and support of pipelines.
• Strong analytical and logical skills.
• Ability to comfortably tackling new challenges and learn

EXPERTISE AND QUALIFICATIONS

Experience: 5 to 9 years

Must have Skills:

• Kotlin/Scala/Java
• Spark
• SQL
• Spark Streaming
• Any cloud (AWS preferable)
• Kafka /Kinesis/Any streaming services
• Object-Oriented Programming
• Hive, ETL/ELT design experience
• CICD experience (ETL pipeline deployment)
• Data Modeling experience

Good to Have Skills:

• Git/similar version control tool
• Knowledge in CI/CD, Microservices