Data Engineer

Job Overview:

The Data Engineer will work closely with data analysts, data scientists, and software engineers to ensure reliable data flow, storage, and accessibility. You will be responsible for building and optimizing data pipelines, implementing ETL processes, and maintaining large-scale data infrastructure.

Key Responsibilities:

• Design, develop, and maintain scalable and efficient data pipelines and architectures.
  
• Implement ETL/ELT processes to collect, transform, and load data from multiple sources.
  
• Ensure data quality, consistency, and integrity across systems.
  
• Collaborate with data scientists, analysts, and software engineers to meet business data requirements.
  
• Optimize data storage and retrieval for performance and scalability.
  
• Work with cloud-based platforms (AWS, Azure, GCP) and big data tools (Hadoop, Spark, Kafka).
  
• Monitor and troubleshoot data pipelines and resolve issues promptly.
  
• Document data workflows, schemas, and processes for internal teams.
  
• Stay up-to-date with emerging data engineering technologies and best practices.

Preferred Qualifications:

• Experience with streaming data tools like Apache Kafka or Kinesis.
  
• Knowledge of data modeling and schema design.
  
• Familiarity with BI tools such as Tableau, Power BI, or Looker.
  
• Experience with containerization (Docker) and orchestration tools (Kubernetes).
  
• Knowledge of Agile/Scrum development methodologies.