Lead Data Engineer

We are looking for a Lead Data Engineer to join our growing engineering team. As a member of the team,

• You will work on enterprise data platforms, architect and implement data lakes both on-prem and in the cloud.
• You will be responsible for evolving technical architecture, design and implementation of data solutions using a variety of big data technologies. You will work extensively on all major public cloud platforms - AWS, Azure and GCP.
• You will work with senior technical architects on our client side to evolve an effective technology architecture and development strategy to implement the solution.
• You will work with extremely talented peers and follow modern engineering practices using agile methodologies.
• You will coach, mentor and lead other engineers and provide guidance to ensure the quality of and consistency of the solution.

Must-have skills and attitudes:

• Passion for data engineering, in-depth knowledge of some of the following technologies SQL (expert level), Python (expert level), Spark (intermediate level), Big data stack of one of AWS/Azure/GCP.
• Hands-on experience in data wrangling, data munging and ETL. Should be able to source data from anywhere and transform data to any shape using SQL, Python or Spark.
• Hands-on experience working with variable data structures like XML/JSON/AVRO etc
• Ability to create data models and architect data warehouse components
• Experience with Version control (GIT/BIT BUCKET etc)
• Strong understanding of Agile, experience with CI/CD pipelines and processes
• Ability to communicate with technical as well as non-technical audience
• Collaborating with various stakeholders
• Have led scrum teams, participated in Sprint grooming and planning sessions, work/effort sizing and estimation

Desired Skills & Experience:

• At least 8 years of industry experience.
• Working knowledge of any of the following - AWS Big Data Stack (S3, Redshift, Athena, Glue, etc.), GCP Big Data Stack (Cloud Storage, Workflow, Dataflow, Cloud Functions, Big Query, Pub Sub, etc.), Azure Big Data Stack (Azure Synapse, ADF, Azure functions, etc)
• Working knowledge of traditional enterprise data warehouse architectures and migrating them to the Cloud.
• Experience with Data Visualization tool (Tableau / Power BI etc)
• Experience with JIRA / Azure DevOps etc.