Technical Lead-Databricks

As a Databricks Engineer-Lead, you will be responsible for designing and developing ETL pipelines using Azure Data Factory for data ingestion and transformation. You will collaborate with various Azure stack modules such as Data Lakes and SQL Data Warehouse to create robust data solutions. Your role will involve writing efficient SQL, Python, and PySpark code for data processing and transformation. It is essential to understand and translate business requirements into technical designs, develop mapping documents, and adhere to transformation rules as per the project scope. Effective communication with stakeholders to ensure smooth project execution is a crucial aspect of this role. To excel in this position, you should possess 7-10 years of experience in data ingestion, data processing, and analytical pipelines involving big data and relational databases. Hands-on experience with Azure services like Azure Data Lake Storage, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, and Azure SQL Database is required. Proficiency in SQL, Python, and PySpark for data manipulation is essential. Familiarity with DevOps practices and CI/CD deployments is a plus. Strong communication skills and attention to detail, especially in high-pressure situations, are highly valued in this role. Previous experience in the insurance or financial industry is preferred. This role is based in Hyderabad and requires the selected candidate to work from the office. If you are passionate about leveraging Databricks, PySpark, SQL, and other Azure technologies to drive innovative data solutions, this position offers an exciting opportunity to lead and contribute to impactful projects.,