Azure Data Bricks Engineer - Python/SQL

Key Responsibilities:

• Data Pipeline Development

  : Design and develop robust data processing pipelines and analytics solutions using Databricks.

• Architecture & Modeling

  : Architect scalable and efficient data models and storage solutions tailored for the Databricks environment.

• Migration & Integration

  : Collaborate with architecture and engineering teams to migrate existing solutions to Databricks.

• Performance Optimization

  : Tune and optimize Databricks clusters and job performance to meet SLAs and evolving business needs.

• Governance & Compliance

  : Implement best practices for data governance, security, and compliance on the Databricks platform.

• Mentorship

  : Guide and mentor junior engineers, offering technical support and best practice recommendations.

• Innovation & Learning

  : Stay informed about emerging data engineering trends and new features in Databricks and related technologies.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related technical discipline.
• 5 to 8 years of overall experience in data engineering, with a minimum of 2 years focused on Databricks.
• Hands-on proficiency in Python, Scala, or SQL.
• Solid understanding of distributed computing principles and big data frameworks like Apache Spark.
• Experience with major cloud platforms (AWS, Azure, or GCP) and their respective data services.
• Demonstrated ability to build scalable and reliable data solutions in dynamic environments.
• Strong analytical and problem-solving skills with high attention to detail.
• Effective communication and collaboration skills within cross-functional teams.

Preferred Qualifications:

• Familiarity with containerization and orchestration tools such as Docker and Kubernetes.
• Understanding of DevOps practices for automating deployments and monitoring data pipelines.