Azure Data Engineer

Job ID:

Location:

Job Summary:

Responsibilities

• Solution Design and Architecture:
• Collaborate with stakeholders to understand business requirements and design scalable data solutions on the cloud platforms.
• Create and maintain data architecture diagrams and documentation.
• Data Ingestion and ETL:
• Develop and implement ETL processes using Databricks, and other relevant tools.
• Design and optimize data pipelines to ensure efficient data processing and transformation.
• Data Storage and Management:
• Implement and manage Relational Databases, Snowflake and other data sources.
• Ensure data integrity, security, and compliance with industry best practices and organizational standards.
• Data Transformation and Processing:
• Utilize Databricks and other tools for data transformation and processing.
• Optimize queries and data processing workflows for performance and efficiency.
• Monitoring and Optimization:
• Implement monitoring solutions to track the performance and health of data solutions.
• Identify and resolve performance bottlenecks and optimize data processing workflows.
• Collaboration and Communication:
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders.
• Communicate effectively with technical and non-technical audiences, providing insights and recommendations.
• Automation and Scripting:
• Develop scripts and automation processes to streamline data engineering tasks.
  

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer with Azure/AWS cloud technologies.
• In-depth knowledge of Databricks, and other relevant Azure services.
• Strong proficiency in SQL, Python, and/or other scripting languages.
• Experience with data modeling, schema design, and database performance tuning.
• Familiarity with data warehousing concepts and best practices.
• Excellent problem-solving and analytical skills.
• Effective communication and collaboration skills.