Principal Data Engineer

Job Description

The Principal Data Engineer will lead the data engineering team, help formulate the organizational data strategy, and work with the cloud analytics platform owners and business stakeholders to ensure alignment with business objectives. This role encompasses both data architect and data engineering responsibilities. The individual will be responsible for formulating the organizational data strategy, including standards of data quality, data flow within the organization, and data security. They will translate business requirements into technical specifications by understanding business needs, examining existing data structures, and creating a blueprint for an integrated framework of accessible, secure data aligned with business strategies. They will also define processes for testing and maintaining the database and lead the Data Engineering team in agile delivery, supporting the Product Owners in defining requirements and delivery timelines. The role requires knowledge of systems development, including solution architecture, data lifecycle management, data architecture and engineering, project management approaches, and design and testing techniques. Proficiency in data modelling and design is essential. Knowledge of programming languages such as Python and SQL is crucial. Experience with multi-cloud architecture, including GCP and Azure is important. Understanding and experience with AI/ML technology is required. The ability to implement common data management and reporting technologies, as well as familiarity with columnar and NoSQL databases, data visualization and unstructured data are expected. Description Team Leadership Guide and Mentor Team Members : Lead, coach, and mentor Data Engineers and ML Engineers to empower them to deliver innovative and effective data solutions and work closely with BI Engineers and Data Scientists. Foster Collaboration : Promote a collaborative and inclusive team environment, encouraging open communication and knowledge sharing among team members. Drive Technical Excellence : Set high standards for technical excellence and ensure the team adheres to best practices in data engineering, analytics, and operations. Support Professional Development : Identify and support the professional development needs of team members, providing opportunities for growth and skill enhancement. Strategic Vision : Develop and communicate a clear vision for the data engineering team, aligning team goals with the overall business strategy. 2. Data Architecture Define Future Data Architecture : Establish the future state of the data architecture and underlying infrastructure, ensuring it aligns with business goals and supports scalability and performance. Conceptual Data Modeling : Model business data conceptually and independently of applications and physical implementations, creating a blueprint for an integrated framework of accessible, secure data. Best Practices and Knowledge Sharing : Capture and share best-practice knowledge among the solutions architect community, fostering a culture of continuous improvement and innovation. Requirement Gathering and Delivery : Liaise with product owners and business teams to gather requirements and address issues, delivering data-driven projects that meet business needs. Project Guidance and Coordination : Advise and support projects regarding data matters, coordinating with multiple delivery teams and providing guidance on best practices in data engineering, analytics, operations, and support. Cloud Infrastructure Partnership : Partner with Cloud Solution Architects to ensure cloud infrastructure is built and configured to meet the evolving needs of the data analytics community. 3.Support to Product Owner Technical Vendor Management Ensure Compliance : Ensure that vendors understand and adhere to the company's data architecture standards and quality requirements. Evaluate Vendor Performance : Regularly assess vendor performance, providing feedback and addressing any issues to maintain high standards of service and delivery. Stay Informed on Vendor Offerings : Keep up to date with the latest offerings and advancements from vendors to leverage new technologies and solutions that can benefit the company. Business Requirements Support and serve as a backup for Product Owner for high level discussion with business on goals and priorities. Work with Product Owner to translate business goals and priorities into EDS priorities and a roadmap for analytics. Work with BA to translating business requirements to technical tasks for team as a backlog in ADO. Analytics Environment and Tooling Work with Product Owner on enhancing analytics environments and implementing new solutions. 4.Security and Governance Support Product Owner and Security Architect to define and implement business policies and procedures related to governing and securing data at McCormick. Ensure consistency between security model in SAP (or other source systems such as Salesforce) and analytics data lake