Sr Data Engineer

As a skilled data engineering professional with over 5 years of experience, you will be responsible for designing and implementing end-to-end ETL solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure Databricks, and SSIS. Your strong proficiency in SQL, REST API integration, and automation through CI/CD pipelines using Azure DevOps and Git will be crucial for this role. You must have a solid understanding of maintaining and optimizing data pipelines, warehouses, and reporting within the Microsoft SQL stack. Key Responsibilities: - Design and implement comprehensive data solutions using Microsoft Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS), Azure Databricks, and SSIS. - Develop complex transformation logic using SQL Server, SSIS, and ADF, and create ETL Jobs/Pipelines to execute those mappings concurrently. - Maintain and enhance existing ETL Pipelines, Warehouses, and Reporting leveraging traditional MS SQL Stack. - Create ADF pipelines to handle HTTP requests for APIs and implement best practices for development and deployment of SSIS packages, SQL jobs, and ADF pipelines. - Implement and manage source control practices using GIT within Azure DevOps to ensure code integrity and facilitate collaboration. - Participate in the development and maintenance of CI/CD pipelines for automated testing and deployment of BI solutions. Qualifications Required: - BS/MS in Computer Science or other engineering/technical degree. Preferred Skills: - Understanding of the Azure environment and experience in developing Azure Logic Apps and Azure Function Apps. - Knowledge of code deployment, GIT, CI/CD, and deployment of developed ETL code (SSIS, ADF) would be advantageous. (Location: Ahmedabad/Pune),