Data Engineer

Job Description

As a Data Engineer, you will be an integral part of our team, working on data pipelines, data warehousing, and data integration for various analytics and AI use cases. You will collaborate closely with Delivery Managers, ML Engineers and other stakeholders to ensure seamless data flow and accessibility. Your expertise will be crucial in enabling data-driven decision-making for our clients. To thrive in this role, you need to be a quick learner, get excited about innovation and be on the constant lookout to master new technologies as they come up in the Data, AI & Cloud teams.

Key Responsibilities

● Design, develop, and maintain scalable data pipelines and ETL processes to support downstream analytics and AI applications.

● Collaborate with ML Engineers to integrate data solutions into machine learning models and workflows.

● Work closely with clients to understand their data requirements and deliver tailored data solutions.

● Ensure data quality, integrity, and security across all projects.

● Optimize and manage data storage solutions in cloud environments (AWS, Azure, GCP).

● Utilize Databricks for data processing and analytics tasks, leveraging its capabilities to enhance data workflows.

● Monitor the performance of data pipelines, identify bottlenecks or failures, and implement improvements to enhance efficiency and reliability.

● Implement best practices for data engineering, including documentation, testing, and version control.

● Troubleshoot and resolve data-related issues in a timely manner.

Qualifications

● Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.

● 3 to 5 years of experience as a Data Engineer or in a similar role.

● Strong proficiency in SQL, Python, and other relevant programming languages.

● Hands-on experience with Databricks and its ecosystem.

● Familiarity with major cloud environments (AWS, Azure, GCP) and their data services.

● Experience with data warehousing solutions like Snowflake, Redshift, or BigQuery.

● Comfortable working with a variety of SQL, NoSQL and graph databases like PostgreSQL and MongoDB;

● Knowledge of data integration tools.

● Understanding of data modeling, data architecture, and database design.

● Excellent problem-solving skills and attention to detail.

● Strong communication and collaboration skills.

Highly Desirable Skills

● Experience with real-time data processing frameworks (e.g., Apache Kafka, Spark Streaming).

● Knowledge of data visualization tools (e.g., Tableau, Power BI).

● Familiarity with machine learning concepts and frameworks.

● Experience working in a client-facing role.

Job Types: Full-time, Permanent

Work Location: In person