Databricks Engineer

You are a skilled Data Engineer with over 4 years of experience, seeking a role in Trivandrum (Hybrid) for a contract duration of 6+ months in IST shift. Your main responsibility will be to design and develop data warehouse solutions utilizing Azure Synapse Analytics, ADLS, ADF, Databricks, and Power BI. You will be involved in building and optimizing data pipelines, working with large datasets, and implementing automation using DevOps/CI/CD frameworks. Your expertise should lie in Azure, AWS, Terraform, ETL, Python, and data lifecycle management. You will collaborate on architecture frameworks and best practices while working on data warehouse design and implementation. In this role, you will be expected to design and develop data warehouse solutions using Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services. This will involve developing and optimizing SQL queries, working with SSIS for ETL processes, and handling challenging scenarios effectively in an onshore-offshore model. You will build and optimize data pipelines for ETL workloads, work with large datasets, and implement data transformation processes. You will utilize DevOps/CI/CD frameworks for automation, leveraging Infrastructure as Code (IaC) with Terraform and configuration management tools. Participation in architecture framework discussions, best practices, and the implementation and maintenance of Azure Data Factory pipelines for ETL projects will also be part of your responsibilities. Ensuring effective data ingestion, transformation, loading, validation, and performance tuning are key aspects of the role. Your skills should include expertise in Azure Synapse Analytics, ADLS, ADF, Databricks, Power BI, and Azure Analysis Services. Strong experience in SQL, SSIS, and Query Optimization is required. Hands-on experience with ETL pipelines, Data Warehousing, and Analytics, as well as proficiency in Azure, AWS, and DevOps/CI/CD frameworks are essential. Experience with Terraform, Jenkins, Infrastructure as Code (IaC), and strong knowledge of Python for data processing and transformation are also necessary. The ability to work in an ambiguous environment and translate vague requirements into concrete deliverables is crucial for this role.,