Data Engineer

Job Summary:

Data Engineer

Key Responsibilities:

• Design, develop, and optimize large-scale data processing pipelines using

  PySpark

  on distributed systems (e.g., Hadoop, Databricks, or EMR).
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver appropriate data solutions.
  
• Perform data extraction, transformation, and loading (ETL/ELT) processes from diverse data sources.
  
• Ensure data quality, integrity, and reliability through robust data validation and monitoring frameworks.
  
• Write optimized Spark code for performance and scalability across big data ecosystems.
  
• Participate in code reviews and contribute to best practices for data engineering.
  
• Work with structured and unstructured data, and integrate real-time and batch data pipelines.
  
• Support and maintain existing data pipelines, troubleshoot issues, and enhance them as required.
  

Required Skills & Experience:

• 5 to 15 years

  of experience in Data Engineering or related fields.
• Strong hands-on experience in

  PySpark

  and distributed data processing.
• Solid understanding of data warehousing concepts, data modeling, and big data architecture.
  
• Proficiency with big data tools and platforms (e.g., Hadoop, Hive, Spark, HDFS).
  
• Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
  
• Proficient in writing efficient SQL queries and working with large datasets.
  
• Strong knowledge of programming in

  Python

  .
• Experience with version control systems like Git.
  

Nice to Have:

• Experience working in the

  telecom domain

  , with understanding of telecom data and KPIs.
• Exposure to workflow orchestration tools such as

  Airflow

  ,

  Oozie

  , or similar.
• Familiarity with data governance, data cataloging, and data security principles.
  

Educational Qualifications:

• Bachelors or Masters degree in Computer Science, Engineering, Information Technology, or a related field.
  

Why Join Us:

• Work on cutting-edge big data projects in a dynamic and collaborative environment.
  
• Opportunity to influence data strategy and build scalable data infrastructure.
  
• Flexible working hours and competitive compensation.