Azure Machine Learning / Databricks

Azure Machine Learning / Databricks Engineer

Roles and Responsibilities:

• ML Solution Development:

  Design, develop, and implement end-to-end machine learning solutions, including data ingestion, feature engineering, model training, evaluation, and deployment, primarily using

  Azure Machine Learning

  and

  Azure Databricks

  .

• Data Pipeline Engineering:

  Build and optimize robust data pipelines for machine learning workloads, leveraging PySpark/Spark SQL within Azure Databricks for large-scale data processing and transformation.

• Azure ML Services:

  Work extensively with Azure Machine Learning services for model lifecycle management, including experiment tracking, model registry, and deploying models as web services or to Azure Kubernetes Service (AKS).

• Databricks Platform:

  Utilize Azure Databricks notebooks, clusters, and Delta Lake for collaborative data science and engineering, ensuring efficient and scalable execution of ML workloads.

• Model Optimization & Monitoring:

  Implement techniques for optimizing model performance and efficiency. Set up monitoring for deployed models to detect drift, bias, and performance degradation.

• Code Quality & MLOps:

  Write clean, maintainable, and production-ready Python/PySpark code. Contribute to MLOps practices for automating ML workflows (CI/CD for ML models).

• Troubleshooting:

  Perform in-depth troubleshooting, debugging, and issue resolution for ML models, data pipelines, and platform-related problems within Azure ML and Databricks environments.

• Collaboration:

  Work closely with data scientists, data engineers, software developers, and business stakeholders to translate machine learning concepts into deployable and impactful solutions.

Preferred Candidate Profile:

• Azure ML Expertise:

  Strong hands-on experience with

  Microsoft Azure Machine Learning Studio

  and its associated services.

• Azure Databricks Proficiency:

  Proven experience in developing and optimizing data and ML solutions on

  Azure Databricks

  using PySpark/Spark SQL.

• Python Programming:

  Excellent proficiency in

  Python

  for data manipulation, machine learning, and scripting.

• Machine Learning Fundamentals:

  Solid understanding of core machine learning concepts, algorithms, and model evaluation metrics.

• Cloud Data Services:

  Familiarity with other Azure data services (e.g., Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory) is a plus.

• SQL Knowledge:

  Good proficiency in SQL for data querying and manipulation.

• Problem-Solving:

  Exceptional analytical and problem-solving skills with a methodical approach to complex data science and engineering challenges.

• Communication:

  Strong verbal and written communication skills to articulate technical solutions and collaborate effectively within a team.

• Education:

  Bachelor's degree in Computer Science, Data Science, Statistics, or a related technical field. Azure certifications related to Data Science or AI are a strong plus.