Technical Lead - Data platform Azure/AWS Databricks

Location : Bangalore,Pune

Experience : 8 - 13 years ( 8 yrs relevant)

Notice period : Immediate to 15 days

Work Mode : Hybrid ( In office mandatory)

Interested candidates share resume to aishahw@hexaware.com.

JD: -

ob Summary: The Technical Lead will be responsible for managing day-to-day operations of the Data Platform in Azure/AWS Databricks environments. This role includes designing and implementing data ingestion pipelines, ensuring data quality and availability, optimizing performance, and maintaining system stability. The candidate will lead architectural decisions, mentor team members, and ensure compliance with security and governance standards. II.

Roles & Responsibilities:

• Design and implement data ingestion pipelines from multiple sources using Azure Databricks. • Ensure smooth and efficient execution of data pipelines. • Develop scalable and reusable frameworks for ingesting datasets. • Integrate end-to-end data pipelines from source systems to target repositories while maintaining data quality and consistency. • Work with event-based/streaming technologies for data ingestion and processing. • Collaborate with project teams to deliver additional components (API interfaces, Search). • Evaluate tools and technologies against customer requirements. • Provide technical advice and resolve issues related to Cloud and Databricks. • Offer on-call and after-hours/weekend support as needed. • Fulfill service requests related to the Data Analytics platform. • Lead optimization and continuous improvement initiatives. • Conduct technical reviews as part of release management. • Ensure adherence to data security standards and controls. • Lead design, development, and deployment of advanced data pipelines and analytical workflows on Databricks Lakehouse. • Collaborate with data scientists, engineers, and business stakeholders to build scalable solutions. • Own architectural decisions and ensure compliance with governance and security requirements. • Mentor data engineers and provide technical guidance. • Implement CI/CD practices for data pipelines using Azure DevOps, GitHub Actions, or Jenkins. III. Years of Experience: • 9+ years of strong hands-on experience in Data Engineering. IV. Educational Qualification & Certifications (Optional): • Bachelors degree in IT, Computer Science, or Software Engineering. • Relevant certifications in Azure, AWS, Cloud Engineering, or Databricks. V. Skill Set Required: Must Have: • Strong experience with Azure/AWS Databricks. • Expertise in Python, Scala, Spark, PySpark. • Proficiency in SQL and RDBMS. • Hands-on experience with data pipeline design and optimization. • Knowledge of structured/unstructured data modeling and storage techniques. • Experience with CI/CD tools (Azure DevOps, Jenkins, GitHub). • Strong troubleshooting and analytical skills. Nice to Have: • Experience with Hive, HBase, Impala, Parquet. • Familiarity with Kafka, Flume, Sqoop. • Knowledge of Airflow for workflow orchestration. • Exposure to Big Data Management tools (Cloudera, Data Lake).  

Regards

Aisha

7618717626