Data Engineer

• The ideal candidate will have hands-on expertise in processing large-scale

  IoT sensor data

  from manufacturing equipment, ensuring data quality, integrity, and high performance across multi-plant operations.

Key Responsibilities:

• Design and implement

  ETL/ELT pipelines

  using

  Databricks

  and

  PySpark

  .
• Process and transform

  real-time IoT sensor data

  from multiple manufacturing plants.
• Manage

  data storage, versioning, and optimization

  using

  Delta Lake

  .
• Integrate data from diverse sources (SQL Server, JSON, CSV, APIs, etc.).
• Write and optimize complex

  SQL queries

  and stored procedures for performance.
• Implement

  data quality monitoring

  , testing, and

  CI/CD pipelines

  using

  Azure DevOps

  .
• Collaborate closely with manufacturing and analytics teams to deliver actionable IoT data insights.

• Top 4 Mandatory Skills:

• Databricks & Apache Spark (PySpark)

  Minimum 3 years of hands-on experience

• Azure Cloud Services

  Experience with ADF, Blob Storage, and DevOps

• Python

  Advanced proficiency with OOP concepts, pandas, and robust error handling

• Delta Lake

  Strong knowledge of data versioning, time travel, and optimization

• Good-to-Have Skills:

• Advanced

  SQL

  and stored procedure development
• Experience in

  JSON data processing

  and schema handling

• Git-based version control

  and CI/CD integration using

  Azure DevOps

• Exposure to

  manufacturing or industrial IoT data patterns

• Preferred Domain Knowledge:

• Industrial IoT

  and time-series data management

• Machine data

  and equipment performance monitoring

• Multi-plant data integration

  and central data architecture

• Data quality frameworks

  , downtime, and defect analytics

Qualifications:

• Bachelor s degree in

  Computer Science

  ,

  Engineering

  , or a related field
• 4-6 years of proven experience in Data Engineering
• 2+ years of strong exposure to

  Databricks/Spark ecosystem

• Azure Data Engineer Associate

  or

  Databricks certification

  is preferred

• Soft Skills:

• Excellent communication skills in English (written and verbal)
• Strong analytical and problem-solving mindset
• Collaborative and proactive approach with cross-functional teams
• High attention to detail and commitment to data accuracy