Lead Data Engineer

As an accomplished Lead Data Engineer with expertise in Databricks, you will play a crucial role in enhancing data engineering capabilities while working with advanced technologies like Generative AI. Key Responsibilities: - Lead the design, development, and optimization of data solutions using Databricks, ensuring scalability, efficiency, and security. - Collaborate with cross-functional teams to gather and analyze data requirements, translating them into robust data architectures and solutions. - Develop and maintain ETL pipelines, leveraging Databricks and integrating with Azure Data Factory as needed. - Implement machine learning models and advanced analytics solutions, incorporating Generative AI to drive innovation. - Ensure adherence to data quality, governance, and security practices, maintaining the integrity and reliability of data solutions. - Provide technical leadership and mentorship to junior engineers, fostering an environment of learning and growth. - Stay updated on the latest trends and advancements in data engineering, Databricks, Generative AI, and Azure Data Factory to continually enhance team capabilities. Required Skills & Qualifications: - Bachelor's or master's degree in computer science, Information Technology, or a related field. - 7+ to 10 years of experience in data engineering, focusing on Databricks. - Proven expertise in building and optimizing data solutions using Databricks and integrating with Azure Data Factory/AWS Glue. - Proficiency in SQL and programming languages such as Python or Scala. - Strong understanding of data modeling, ETL processes, and Data Warehousing/Data Lakehouse concepts. - Familiarity with cloud platforms, particularly Azure, and containerization technologies like Docker. - Excellent analytical, problem-solving, and communication skills. - Demonstrated leadership ability with experience mentoring and guiding junior team members. Preferred Qualifications: - Experience with Generative AI technologies and their applications. - Familiarity with other cloud platforms, such as AWS or GCP. - Knowledge of data governance frameworks and tools. In this role, you can enjoy perks such as flexible timings, a 5-day workweek, a healthy environment, celebrations, opportunities to learn and grow, build the community, and medical insurance benefits.,