Data Scientist

Key Responsibilities:

Work closely with Data Engineers and ML Engineers to ensure data readiness and model deployment.

Present findings and insights to stakeholders using visualizations and dashboards.

Productionization & Monitoring:

Assist in transitioning models from development to production.

Monitor model drift and performance over time.

Data Exploration & Querying:

• Use Databricks SQL for querying and analysing data.
• Document data lineage and ensure reproducibility

Exploratory Data Analysis (EDA):

• Use Databricks Notebooks (Python, R) to explore structured and unstructured data.
• Perform hypothesis testing and statistical profiling.

Feature Engineering:

• Create and manage features using Databricks Feature Store.
• Transform raw data into meaningful inputs for models.
• Model Development & Training: for fraud claims identification, loss prediction modelling .

Build and train models using MLlib, scikit-learn, TensorFlow, or PyTorch.

• Utilize Databricks Runtime for ML for optimized performance.
• Experiment Tracking & Model Evaluation:

Track experiments and model metrics using MLflow

• Evaluate model performance using cross-validation and business KPIs.
• Collaboration & Communication:

Required Skills:

• Strong programming in Python or R
• Proficiency in machine learning algorithms and statistical methods
• Experience with big data processing using Spark
• Familiarity with cloud platforms (AWS, Azure, GCP)
• Excellent communication and storytelling skills

Tools & Technologies Used:

• Databricks Notebooks
• Apache Spark (via PySpark or SparkR)
• MLflow (Tracking, Registry, Deployment)
• Feature Store
• Delta Lake
• Databricks SQL
• Visualization Tools (e.g., matplotlib, seaborn, Power BI)