Senior Data Engineer

The Staff Data Engineer will play a crucial role within the E&DS Organization. In this position, they will collaborate with product management, customers, engineers, and data scientists to leverage their expertise in developing effective technical solutions that address business challenges. The Staff Data Engineer will focus on Data pipelining, Data stream, machine learning infrastructure, security, scalability, and automation, playing a key role in designing, developing, and deploying Data solutions and AI/ML models that provide business value. Furthermore, they will identify and assume responsibility for mid- to long-term deliverables while enhancing their domain and technical knowledge, mentoring and coaching junior engineers, and cultivating relationships with senior engineers across the company.

MAJOR TASKS & RESPONSIBILITIES:

• Lead and participate in design sessions with Enterprise and Hub Data Stewards, Engineering teams, Data Scientists, Product Managers, and business and IT stakeholders that result in documentation for data processing, storage, and delivery solutions.
• Understand business capability needs and processes as they relate to IT solutions by partnering with Product Managers and business and functional IT stakeholders, applying this knowledge to define business problems that need to be solved.
• Initiate and lead the evaluation of new technologies, such as Semantic layer, BigQuery,Domino, or Redshift, and new languages like Go or React, performing POCs and presenting results to others with the goal of providing technical recommendations.
• Help the team establish and improve processes and methodologies, such as SCRUM or Kanban, and/or lead the piloting of new ones.
• Implement data solutions according to design documentation using various tools and programming languages like Kafka, SQL, non-SQL databases, Scala, and Go, while following the team s established processes and methodologies.
• Facilitate and participate in code reviews, retrospectives, functional testing, integration testing, and other team activities focused on improving delivery quality.
• Provide reliable estimates for large-scale projects.
• Initiate collaboration with Product Owners, other engineers, and data stewards within the team and across data, technical platforms, and product teams to plan and align roadmaps, delivery dates, and integration efforts.
• Coach and mentor junior and aspiring Data Engineers on the team and across the data and engineering communities. Effectively discuss work or provide detail at the appropriate level for the audience: business partners, data scientists, engineering teams, etc. Create and maintain design and code documentation in GitHub, Haystack, SharePoint, or other repositories used by the team.
• Provide technical guidance and mentorship to junior team members.

WHO YOU ARE:

• Bachelor s degree in Computer Science, Software Engineering, or a related field. An additional 3 years of relevant experience is an acceptable substitute for the degree requirement.
• 8+ years of professional software engineering experience is required. An advanced degree in a relevant field counts as 2 years (Masters) or 4 years (PhD) of experience. Professional software engineering experience to include:
• 5+ years of experience engineering data-intensive software using streaming and resource-based design principles.
• 5+ years of experience in working with cloud technologies and services.
• Demonstrated experience with data architecture and modeling, including designing both logical and physical models for datasets. Proficiency in working with relational databases such as Postgres, MySQL, Oracle, etc.
• Proven experience modeling large datasets in distributed databases such as Apache Cassandra.
• Strong interpersonal skills and a desire to work in a highly collaborative environment. Familiarity with relevant industry trends.
• Desired experience with: Agriculture, life sciences, bioinformatics, biochemistry, genetics, biology, or related disciplines. Platform-as-a-Service software such as Cloud Foundry or Kubernetes. Stream processing, e.g. Kafka, Spark Streaming, Akka, etc. Knowledge of machine learning or other data science practices. Experience contributing to open source projects.
• Experience with big data technologies (e.g., Hadoop, Spark) and data engineering practices.
• Previous experience in a leadership or mentoring role is a plus.