Data Engineer

Job Summary

Data Engineer

Years of experience needed –

5+ years of experience in data engineering or a related field.

Technical Skills:

• Design and implement robust, scalable, and high-performance data pipelines using

  PySpark

  and

  Python

  .
• Develop ETL/ELT workflows to ingest, transform, and load data from various sources (e.g., APIs, databases, cloud storage).
• Optimize data processing jobs for performance and cost-efficiency in distributed environments.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Ensure data quality, integrity, and governance across all pipelines.
• Monitor and troubleshoot production data workflows and resolve issues proactively.
• Leverage

  Azure

  for data storage, processing, and orchestration
• Strong proficiency in Python for data manipulation and scripting.
• Hands-on experience with PySpark and distributed data processing.
• Solid understanding of SQL and relational databases.
• Familiarity with data modeling, data warehousing, and performance tuning.
• Experience with version control (e.g., Git) and CI/CD pipeline