ADF Data Engineer

Overview:

We are seeking an experienced Azure Data Engineer to join our team in a hybrid Developer. This role focuses on enhancing and supporting existing Data & Analytics solutions by leveraging Azure Data Engineering technologies. The engineer will work on developing, maintaining, and deploying IT products and solutions that serve various business users, with a strong emphasis on performance, scalability, and reliability.

Must-Have Skills:

• ADF

• Databricks

Key Responsibilities:

• Incident classification and prioritization
  
• Log analysis and trend identification
  
• Coordination with Subject Matter Experts (SMEs)
  
• Escalation of unresolved or complex issues
  
• Root cause analysis and permanent resolution implementation
  
• Stakeholder communication and status updates
  
• Resolution of complex and major incidents
  
• Code reviews (Per week 2 per individual) to ensure adherence to standards and optimize performance
  
• Bug fixing of recurring or critical issues identified during operations
  
• Gold layer tasks, including enhancements and performance tuning.
  
• Design, develop, and support data pipelines and solutions using Azure data engineering services.
  
• Implement data flow and ETL techniques leveraging Azure Data Factory, Databricks, and Synapse.
  
• Cleanse, transform, and enrich datasets using Databricks notebooks and PySpark.
  
• Orchestrate and automate workflows across services and systems.
  
• Collaborate with business and technical teams to deliver robust and scalable data solutions.
  
• Work in a support role to resolve incidents, handle change/service requests, and monitor performance.
  
• Contribute to CI/CD pipeline implementation using Azure DevOps.
  

Technical Requirements:

• 6 to 8 years of experience in IT and Azure data engineering technologies.
  
• Strong experience in Azure Databricks, Azure Synapse, and ADLS Gen2.
  
• Proficient in Python, PySpark, and SQL.
  
• Experience with file formats such as JSON and Parquet.
  
• Working knowledge of database systems, with a preference for Teradata and Snowflake.
  
• Hands-on experience with Azure DevOps and CI/CD pipeline deployments.
  
• Understanding of Data Warehousing concepts and data modeling best practices.
  
• Familiarity with SNOW (ServiceNow) for incident and change management.
  

Non-Technical Requirements:

• Ability to work independently and collaboratively in virtual teams across geographies.
  
• Strong analytical and problem-solving skills.
  
• Experience in Agile development practices, including estimation, testing, and deployment.
  
• Effective task and time management with the ability to prioritize under pressure.
  
• Clear communication and documentation skills for project updates and technical processes.
  

Technologies:

• Azure Data Factory
  
• Azure Databricks
  
• Azure Synapse Analytics
  
• Unity Catalog
  
• PySpark / SQL
  
• Azure Data Lake Storage (ADLS), Blob Storage
  
• Azure DevOps (CI/CD pipelines)
  

Nice-to-Have:

• Experience with Business Intelligence tools, preferably Power BI
  

• DP-203

  certification (Azure Data Engineer Associate)