Job
Description
Design, build, and maintain scalable and efficient data architectures to support current and future business needs. Ensure data modelling practices align with best practices for maintainability, scalability, and performance. Solve business problems by applying advanced data engineering and complex data transformation techniques on large volumes of data. Stay abreast of the latest technologies and tools in data engineering (e.g., cloud platforms, databases, ETL tools) and lead the evaluation and integration of new solutions. Identify opportunities to implement emerging technologies (e.g., machine learning, AI-driven data management) to enhance data capabilities. Develop, optimize, and manage batch and real-time data ingestion pipelines. Work closely with software engineers and data scientists to ensure seamless integration of data pipelines into the larger system architecture. Automate data validation and monitoring processes to ensure data quality and integrity. Foster a collaborative environment focused on continuous learning, growth, and innovation. Manage the day-to-day operations of the team, setting priorities and ensuring timely delivery of data-related projects. The skills you bring: Strong knowledge of database and data warehousing technologies (e.g., SQL, NoSQL, etc) Strong experience in data engineering or a related field, with a strong focus on designing, developing, and managing large-scale data pipelines and architectures. Proficiency in building ETL/ELT pipelines using python / spark (pyspark). Advanced skills in programming languages such as Python, SQL. Expertise in designing and optimizing data models and complex SQL queries for high-volume systems. Strong problem-solving abilities, with a focus on diagnosing and resolving complex data engineering challenges. Relevant certifications in data engineering, cloud platforms (e.g., AWS, GCP, Azure), or big data technologies (e.g., Hadoop, Spark) are a plus. Experience in HANA, Snowflake and other database system.
