Data Engineer

Role Overview:

Experience: 6+years

As a Data Engineer, you will play a key role in designing, developing, and optimizing data pipelines and storage solutions in a complex, enterprise-scale data warehouse environment. You will contribute to full life-cycle software development projects, leveraging modern technologies and best practices to deliver high-quality, actionable data solutions.

Key Responsibilities:

• Participate in the full software development lifecycle for enterprise data projects, from requirements gathering to deployment and support.
• Design, develop, and maintain robust ETL processes and data pipelines using Snowflake, Hadoop, Databricks, and other modern data platforms.
• Work with a variety of databases: SQL (MySQL, Postgres SQL, Vertica), NoSQL (MongoDB, Cassandra, Azure Cosmos DB), and distributed/big data solutions (Apache Spark, Cloudera).
• Write advanced SQL queries and perform complex data analysis for business insights and operational reporting.
• Develop scripts in Python and shell for data manipulation, automation, and orchestration.
• Perform data modelling, analysis, and preparation to support business intelligence and analytics solutions.
• Maintain and optimize Unix/Linux file systems and shell scripts.
• Collaborate with cross-functional teams to translate business requirements into scalable data solutions.
• Present analytical results and recommendations to technical and non-technical stakeholders, supporting data-driven decision making.
• Troubleshoot, diagnose, and resolve complex technical issues across the data stack.
• Stay current with industry trends, tools, and best practices to continuously improve data engineering processes.

Required Skills and Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field (or equivalent experience).
• Demonstrated full life-cycle experience in enterprise software/data engineering projects.
• Hands-on experience with Snowflake and Hadoop platforms.
• Proficient in SQL, Postgres SQL, Vertica, and data analysis techniques.
• Experience with at least one SQL database (MySQL, Postgres SQL) and one NoSQL database (MongoDB, Cassandra, Azure Cosmos DB).
• Experience with distributed/big data platforms such as Apache Spark, Cloudera, Vertica, Databricks, or Snowflake.
• Extensive experience in ETL, shell or Python scripting, data modelling, analysis, and data preparation.
• Proficient in Unix/Linux systems, file systems, and shell scripting.
• Strong problem-solving and analytical skills.
• Ability to work independently and collaboratively as part of a team; proactive in driving business decisions and taking ownership of deliverables.
• Excellent communication skills with experience in presentation design, development, and delivery to communicate technical insights and recommendations effectively.

Preferred/Desirable Skills:

• Industry certifications in Snowflake, Databricks, or Azure Hyperscale are a strong plus.
• Experience with cloud platforms such as AWS, Azure, or Snowflake.
• Familiarity with BI reporting tools like Power BI or Tableau.
• Proficient in using Git for branching, merging, rebasing, and resolving conflicts in both individual and team-based projects.
• Familiar with GitHub Copilot to accelerate code writing, refactoring, and documentation tasks.
• Knowledge of industry best practices and emerging technologies in data engineering and analytics.