Senior Consultant - Azure Data Engineer

Job Description

Role & responsibilities Hands-on experience in Azure Data factory, Azure Databricks and SQL Server Database. Experience in Agile working model. Hands-on experience in programming languages like Python/Pyspark. Hands-on experience and knowledge in Azure DevOps processes for CICD. Build robust and scalable data pipelines using ADF, Databricks for data ingestion, transformation, and loading. Optimize performance and cost efficiency for Databricks clusters, data pipelines, and storage systems. Monitor and manage cloud resources to ensure high availability, performance and scalability. Excellent communication and teamwork skills. Work with stakeholders to gather requirements and translate them into technical solutions. Prepare architecture diagrams, technical documentation for the deployed solutions. Ensure data quality and integrity throughout the data lifecycle. Work across all phases of SDLC and use Software Engineering principles to build scaled solutions. Good to have experience / Knowledge in Migrate SSIS Packages to Azure. Good to have experience / Knowledge in Extract, Transform & Load (ETL) development using SQL Server Integration Services (SSIS), SQL Server Reporting Service (SSRS). Having Knowledge on Data Warehouse, concepts like Star Schema, Snowflake, Dimension and Fact Tables. Experience in programming Tasks-Stored Procedures, Triggers, Sub queries, Joins using SQL Server 2008/2012/2016/2019 with T-SQL. Preferred candidate profile 6+ years of enterprise experience in Azure Data factory and Azure Databricks with: Hands-on experience in programming languages like Python/Pyspark and T-SQL Hands-on experience and knowledge in Azure DevOps processes for CICD Good Data warehouse skills Good dimensional modeling skills Design Source-Target Mapping definitions and associated transformations, jobs and sequences Good to have experience / Knowledge in SSIS, SSRS and SSAS skills Strong SQL programming and stored procedure development skills Strong source system analysis and data analysis skills Good to have experience / Knowledge in C#.NET functions as part of custom transformations Prepare architecture diagrams, technical documentation for the deployed solutions. Ability to effectively work independently and as part of a team providing direction and mentorship to others as needed. Demonstrated project discipline and experience. Must be organized, focused, and driven toward established deliverable dates. Must be able to design solutions and form/drive related plans. Excellent verbal and written English communication skills. Demonstrated proficiency in technical writing is required.