Senior Data Engineer

Responsibilities:

• Data Pipeline Development & Maintenance: Design, build, and maintain end-to-end data pipelines for ingesting, processing, and transforming large datasets from various sources (e.g., relational databases, APIs, flat files, streaming data).
  
• ETL/ELT Process Optimization: Develop and optimize ETL/ELT processes using industry-standard tools and techniques, ensuring data accuracy, efficiency, and scalability.
  
• Database Design & Management: Design, implement, and manage relational databases, including schema design, indexing, performance tuning, and data governance.
  
• Data Warehousing: Design and implement data warehousing solutions to support business intelligence and reporting needs. Experience with star schema, snowflake schema, and other data modelling techniques is essential.
  
• Database Administration: Perform database administration tasks, including performance monitoring, capacity planning, backup and recovery, and security management.
  
• Data Quality & Governance: Implement data quality checks, validation rules, and data governance policies to ensure data accuracy and consistency.
  
• Cloud Platform Expertise: Leverage cloud platforms (e.g., AWS, Azure, GCP) for data storage, processing, and management.
  
• Collaboration & Communication: Collaborate with cross-functional teams (e.g., data scientists, business analysts, software engineers) to understand data requirements and deliver effective data solutions. Clearly communicate technical concepts to both technical and non-technical audiences.
  
• Mentoring & Knowledge Sharing: Mentor junior engineers and share knowledge and best practices within the team.
  
• Automation & Scripting: Automate data engineering tasks using scripting languages (e.g., Python, Bash).
  
• Stay Up to Date: Continuously research and evaluate new data technologies and techniques to improve our data infrastructure.
  
• Production Support: Monitoring the batch/jobs daily, no matter itweekdays or weekend.
  
• Production Release: Actively participation in release process.