Azure Data Engineer

As an Azure Data Engineer at our company, you will be responsible for designing, developing, and deploying scalable data pipelines using Azure Data Factory, Databricks, and PySpark. Your role will involve working with large datasets (500 GB+) to optimize data processing workflows. Collaborating with cross-functional teams to prioritize project requirements and ensuring data quality, security, and compliance with organizational standards will be essential tasks. Troubleshooting data pipeline issues, optimizing performance, and effectively communicating technical solutions to non-technical stakeholders are key aspects of the role. To excel in this position, you should have at least 4 years of experience in data engineering and hold an Azure certification. Proficiency in Azure, Databricks, PySpark, SQL, data modeling, data integration, and workflow orchestration is crucial. Strong communication and interpersonal skills are necessary, along with the ability to work in a 2nd shift. Experience in product engineering would be a valuable asset. Additionally, having a minimum of 2-3 projects working with large datasets is preferred. Key Responsibilities: - Design, develop, and deploy scalable data pipelines using Azure Data Factory, Databricks, and PySpark - Work with large datasets (500 GB+) to develop and optimize data processing workflows - Collaborate with cross-functional teams to identify and prioritize project requirements - Develop and maintain data models, data integration, and workflow orchestration - Ensure data quality, security, and compliance with organizational standards - Troubleshoot data pipeline issues and optimize performance - Communicate technical solutions to non-technical stakeholders Requirements: - 4+ years of experience in data engineering - Azure certification is mandatory - Strong proficiency in Azure, Databricks, PySpark, SQL, data modeling, data integration, and workflow orchestration - Experience working with large datasets (500 GB+) - Strong communication and interpersonal skills - Ability to work in a 2nd shift - Experience in product engineering is a plus - Minimum 2-3 projects with experience working in large datasets Skills: azure, sql, integration, orchestration, communication skills, data security, databricks, workflow, modeling, data quality, data integration, data modeling, advanced SQL, workflow orchestration, data compliance, PySpark,