MLops

Technical Environment / Tools:

• GCP (Google Cloud Platform)
• Vertex AI

Core Skills / Tools & Expected Proficiency

Data Drift & Model Drift Monitoring Advanced

• Apply statistical drift-detection techniques to identify shifts in input data.
• Detect drift in incoming image-based datasets and quantify shifts in model confidence and accuracy.
• Recommend retraining actions based on drift severity and impact.

Model Performance Tracking Advanced

• Compute key ML performance metrics: confidence, accuracy, precision, recall, and F1-score.
• Implement continuous evaluation methods to assess production model performance.
• Track long-term performance trends and detect degradation or instability.
• Build automated ML metric dashboards (e.g., Looker Studio or equivalent) to monitor underperforming, new, or emerging classes.

Observability & Traceability Intermediate to Advanced

• Maintain full traceability of inference workflows to ensure reproducibility.
• Understand and monitor end-to-end image classification pipelines.

Model Behavior & Fairness Assessment Intermediate

• Use Explainable AI techniques to generate interpretable insights into model decisions.
• Apply bias-detection frameworks to identify and report emerging fairness issues.

Alerts, Reporting & Retraining Automation Intermediate to Advanced

• Implement ML monitoring rules such as threshold-based alerts and anomaly-detection triggers.
• Build automated alerting systems to notify stakeholders when model performance deteriorates.
• Summarize and report model performance for auditability and retraining pipelines.