Data Engineer

The Data Engineer plays a crucial role in designing, developing, and maintaining robust data solutions that support business intelligence, analytics, and operational efficiency. This position is integral to enabling seamless data generation, collection, and processing, ensuring high-quality, reliable, and accessible data for stakeholders. The role is especially significant in organisations leveraging advanced data warehousing platforms such as Snowflake and adopting cloud-based architectures.

Key Responsibilities:

• Design and Develop Data Solutions: Create scalable and efficient data architectures tailored to business requirements.
• Data Pipeline Creation: Build and manage automated data pipelines for data ingestion, transformation, and delivery.
• Data Quality Assurance: Implement processes and tools to ensure data accuracy, consistency, and reliability across systems.
• ETL Process Implementation: Develop and optimise Extract, Transform, Load (ETL) workflows using industry-standard tools and techniques.
• Data Migration and Deployment: Lead the migration of data assets across systems and oversee deployment in cloud and on-premises environments.
• Collaboration: Work closely with data scientists, analysts, and business teams to understand data needs and deliver solutions.
• Monitoring and Optimisation: Continuously monitor data pipelines and optimise for performance, cost, and reliability.

Required Skills:

Must-Have:

• Expertise in Snowflake Data Warehouse for data storage, querying, and management.
• Hands-on experience with Data Building Tool (DBT) for data transformation and workflow management.
• Proficiency in designing and implementing ETL processes.
• Solid understanding of data integration, data modelling, and data governance principles.
• Experience with cloud platforms (e.g., AWS, Azure, GCP) for data solution deployment.
• Programming skills in SQL and at least one scripting language (Python preferred).

Good-to-Have:

Knowledge of additional data warehousing solutions (e.g., Redshift, BigQuery).

Familiarity with data visualisation tools and dashboarding.

Experience in real-time data streaming and processing frameworks.

Exposure to DevOps practices and CI/CD pipelines for data deployments.

Roles & Responsibilities:

• Work independently and as part of a team to deliver end-to-end data solutions.
• Act as a subject matter expert (SME) for data engineering best practices and Snowflake platform usage.
• Participate in cross-functional projects, collaborating with IT, analytics, and business teams.
• Proactively identify and resolve data issues, ensuring high data quality and availability.
• Monitor, troubleshoot, and optimise data pipelines for efficiency and scalability.
• Document data architectures, processes, and workflows clearly for technical and non-technical stakeholders.

Professional & Technical Skills:

• Proven experience in ETL development and data pipeline management.
• Strong background in data integration and transformation using modern tools.
• Demonstrated knowledge of data governance, privacy, and security standards.
• Hands-on experience with cloud-based data engineering solutions and services.
• Ability to analyse complex data requirements and propose efficient solutions.
• Excellent communication skills for cross-team collaboration and stakeholder engagement.