Senior Data Engineer

Job Title: Senior Data Engineer

About the role:

The Senior Data Engineer role focuses on building and maintaining data pipelines, ensuring data quality, and collaborating with various teams within Springer Nature.

About us:

We re looking for a Senior Data Engineer to join S prin ger Nat ure Global Business Systems (GBS) within Springer Nature Technology . Springer Nature is a leading publisher of scientific books, journals and magazines with over 3000 journal titles and one of the world s largest corpora of peer-reviewed scientific text data. You would be joining a new programme of work to transform how Springer Nature uses its data: building up data capabilities, creating a data platform and engineering capability (technology, people and process) to create a foundation for the future, adding value to cross-organisation i nitiatives and kick-starting data-driven i nnovation.

Across the programme, our teams are cross-functional, diverse and made up of different experience levels. A ll team members collaborate to deliver the best solutions that satisfy our customers needs.

We are committed to growing and nurturing our people for the long-term. We attend co nference s and make time for people to explore new technology that interests them and is relevant to our work as well as host data engineering community-of-practice sessions to share knowledge.

We work in a supportive environment. We value face-to-face co-working and require two days per week in the office on average in a month.

The position is based in either Pune (India) or Lisbon ( Portugal ), with some travel required, typically 1-2 times a year. This role is on a small, autonomous team and you will be expected to impact what we do and how we work. We like to keep our processes light, and bureaucracy slim.

About y ou

Role r esponsibilities:

• Design, implement, test and maintain scalable and efficient ETL/ELT pipelines to ingest and transform large datasets from diverse sources.

• Architect, build, test and optimize data warehouses and data lakes, focusing on data consistency, performance, and best practices for analytics.

• Implement data quality checks, validation rules, and monitoring to ensure data accuracy, integrity, and security. Contribute to data governance initiatives.

• Continuously monitor and enhance database and query performance, identifying opportunities to streamline and improve data retrieval times.

• Define and design frameworks for monitoring data quality and data pipelines

• Monitor Google BigQuery consumption.

• Evaluate, select, and integrate new tools and technologies to enhance data capabilities. Automate manual processes to improve efficiency and accuracy.

• Mentor junior engineers, sharing best practices in data engineering and analysis, and helping to build a culture of technical excellence within the team.

• Proactively lead streams of data engineering work, collaborating with stakeholders and other non-engineering roles to come up with creative solut ions to the problems your team is presented with.

• Work with both data producers and consumers to optimise existing data products and the data within them to meet evolving business needs.

• Work collaboratively with other engineers, u sing t echniques like pair and ensemble programming, to foster collective ownership and help upskill, reskill and learn from other team members.

• Help guide technology choices, adopting company-standard tech stacks by default while responsibly investig ating alternative tools and services and looking for opportunities to innovate.

Skills experience:

Technologies you will be working with:

• SQL, Python, Google Cloud Platform (GCP), and Google BigQuery

• Data pipeline tools such as Apache Airflow and DBT

• Data modeling concepts, including dimensional and star schema design

• Data visualization tools like Looker

Essential:

• Several years of experience in D ata / Software eng ineering on a cloud platform.

• An understanding of data and distributed systems concepts.

• Several years of experience working with data from large ERP systems.

• Knowledge of data quality concepts and experienc e using too ls to identify data quality problems and come up w ith strategies to tackle them.

• Experience working with iterative software development principles:

• Continuous Integration Deployment (CI/CD)

• Automated testing at different levels

• Collaborative development techniques like pair and ensemble programming

• Experience owning software systems and/or data pipelines end-to-end, having full responsibility for operating them in production a nd responding to problems that arise.

• The ability to work with stakeholders and other non-technical roles to translate business requirements into technical work , and conversely to articulate necessary technical work to the same people for prioritisation.

• High competence in SQL and deep experience of at least one programming language e.g. Python, Java

Desirable:

• Experience with decentralised Data Mesh and Data Product architecture principles.

• Experience building and testing data pipelines with DBT

• Understanding and experience of User-Centric Design principles

What you will be doing:

1 month:

• Actively contributing to the codebase in collaboration with other engineers and deploying changes into production

• Familiarising yourself with our tech stack and processes

• Starting to understand the data landscape in and around your team

• Getting to know the various stakeholders and their general requirements

• Collaborating effectively with each discipline on the team

• Participating in technical discussions and sharing ideas

3 months:

• Gaining an understanding of the team s context within the wider organisation

• Leading a stream of work, working with both engineers and stakeholders

• Triaging support queries and diagnosing issues in live data pipelines

• Setting the technical direction of the work done by the team

• Holding discussions w ithin the engine ering team in order to improve product architecture and code quality

• Ensuring that data is stored securely and in compliance with GDPR

6 months:

• Take ownership of key projects and drive initiatives to enhance data capabilities .

• Switching context between multiple streams of work, providing guidance to both developers and stakeholders

• Fostering a culture of continuous feedback, giving and rece iving constructive feedback within your team , and proactively improving ways of working

• Mentoring others in the principles of data engineering and best practices and looking for opportunities to help other engineers on the team grow

• Arbitrating disagreements within the team and not avoiding difficult conversations

• Gauging the complexity and scope of a piece of work, breaking it into smaller pieces when appropriate with a focus on iteratively delivering value to end users