Sr Data Engineer

Job Description

As part of the CE Data Platform team, your role will involve establishing a clear vision for data engineering practices, harmoniously aligning it with the data architecture. Collaboration with product managers is essential to comprehend business requirements and identify opportunities for data leverage. This position also entails the responsibility of creating, designing, and developing complex data processing pipelines. A cooperative relationship with the data intelligence team is necessary for designing scalable implementations and production of data models. The role involves writing clean, iterative code, utilizing various continuous delivery practices to deploy, support, and operate data pipelines. The selection of suitable data modeling techniques, optimization and design of physical data models, and understanding of the trade-offs between various data modeling techniques, form an integral part of this role. Job Qualifications : You are passionate about data, possessing the ability to build and operate data pipelines, and maintain data storage within distributed systems. This role requires a deep understanding of data modeling and experience with modern data engineering tools and platforms, along with cloud warehousing tools. It is perfect for individuals who can go deep into coding and leading junior members to implement a solution. Experience in defining and implementing data governance and security policies is crucial. Knowledge of DevOps and the ability to navigate all the phases of the data & release life cycle is also essential. Professional Skills: You are familiar with AWS and Azure Cloud. You have extensive knowledge of Snowflake , SnowPro Core certification is a must have. You have used DBT at least in one project to deploy models in production. You have configured and deployed Airflow and integrated various operator in airflow (especially DBT & Snowflake). You can design build, release pipelines and understand of Azure DevOps Ecosystem. You have excellent understanding of Python (especially PySpark) and able to write metadata driven programs. Familiar with Data Vault (Raw , Business) also concepts like Point In Time , Semantic Layer. You are resilient in ambiguous situations and can clearly articulate the problem in a business friendly way. You believe in documenting processes and managing the artifacts and evolving that over time. Good to have skills: You have experience with data visualization techniques and can communicate the insights as per the audience. Experience with Terraform and Hashicorp Vault highly desirable. Knowledge of docker and Streamlit is a big plus.