Job
Description
As a Senior Data Engineer at our client's AI & Automation team, your responsibilities will include: - Designing, building, and maintaining scalable, high-performance data pipelines and ETL/ELT processes across diverse database platforms. - Architecting and optimizing data storage solutions to ensure reliability, security, and scalability. - Leveraging generative AI tools and models to enhance data engineering workflows, drive automation, and improve insight generation. - Collaborating with cross-functional teams (Data Scientists, Analysts, and Engineers) to understand and deliver on data requirements. - Developing and enforcing data quality standards, governance policies, and monitoring systems to ensure data integrity. - Creating and maintaining comprehensive documentation for data systems, workflows, and models. - Implementing data modeling best practices and optimizing data retrieval processes for better performance. - Staying up-to-date with emerging technologies and bringing innovative solutions to the team. Qualifications required for this role: - Bachelor's or Masters degree in Computer Science, Information Technology, or a related field. - 5+ years of experience in data engineering, designing, and managing large-scale data systems. - Strong expertise in database technologies including SQL, NoSQL (MongoDB or Cassandra or CosmosDB), Snowflake or Redshift or BigQuery or Microsoft Fabrics, and Azure. - Hands-on experience implementing and working with generative AI tools and models in production workflows. - Proficiency in Python and SQL, with experience in data processing frameworks (e.g., Pandas, PySpark). - Experience with ETL tools (e.g., Apache Airflow, MS Fabric, Informatica, Talend) and data pipeline orchestration platforms. - Strong understanding of data architecture, data modeling, and data governance principles. - Experience with cloud platforms (preferably Azure) and associated data services. Skills required for this role: - Advanced knowledge of Database Management Systems and ETL/ELT processes. - Expertise in data modeling, data quality, and data governance. - Proficiency in Python programming, version control systems (Git), and data pipeline orchestration tools. - Familiarity with AI/ML technologies and their application in data engineering. - Strong problem-solving and analytical skills, with the ability to troubleshoot complex data issues. - Excellent communication skills, with the ability to explain technical concepts to non-technical stakeholders. - Ability to work independently, lead projects, and mentor junior team members. - Commitment to staying current with emerging technologies, trends, and best practices in the data engineering domain. As a Senior Data Engineer at our client's AI & Automation team, your responsibilities will include: - Designing, building, and maintaining scalable, high-performance data pipelines and ETL/ELT processes across diverse database platforms. - Architecting and optimizing data storage solutions to ensure reliability, security, and scalability. - Leveraging generative AI tools and models to enhance data engineering workflows, drive automation, and improve insight generation. - Collaborating with cross-functional teams (Data Scientists, Analysts, and Engineers) to understand and deliver on data requirements. - Developing and enforcing data quality standards, governance policies, and monitoring systems to ensure data integrity. - Creating and maintaining comprehensive documentation for data systems, workflows, and models. - Implementing data modeling best practices and optimizing data retrieval processes for better performance. - Staying up-to-date with emerging technologies and bringing innovative solutions to the team. Qualifications required for this role: - Bachelor's or Masters degree in Computer Science, Information Technology, or a related field. - 5+ years of experience in data engineering, designing, and managing large-scale data systems. - Strong expertise in database technologies including SQL, NoSQL (MongoDB or Cassandra or CosmosDB), Snowflake or Redshift or BigQuery or Microsoft Fabrics, and Azure. - Hands-on experience implementing and working with generative AI tools and models in production workflows. - Proficiency in Python and SQL, with experience in data processing frameworks (e.g., Pandas, PySpark). - Experience with ETL tools (e.g., Apache Airflow, MS Fabric, Informatica, Talend) and data pipeline orchestration platforms. - Strong understanding of data architecture, data modeling, and data governance principles. - Experience with cloud platforms (preferably Azure) and associated data services. Skills required for this role: - Advanced knowledge of Database Management Systems and ETL/ELT processes. - Expertise in data modeling, data quality, and data governance. - Proficiency in Python programming, version control systems (Git), and data pipeline orchestration tools. - Familiarity with AI/ML technologies and their applicatio
