Data Engineer

Position Title - Data Engineer

Location - Kharadi, Pune

You will work with

The team is responsible for driving application simplification across the

organization, focusing on reducing operational complexity and technical debt. They work closely with the wider digital delivery and digital core teams to identify and pursue simplification opportunities. The team collaborates with business partners to align simplification efforts with broader transformation goals and drive measurable improvements in operational efficiency.

Let me tell you about the role

• A Data Engineer designs and builds scalable data management systems that support application simplification efforts.
• They develop and maintain databases and large-scale processing systems to enable efficient data collection, analysis, and integration.
• Key responsibilities include ensuring data accuracy for modeling and analytics, optimizing data pipelines for scalability, and collaborating with data scientists to drive data-driven decision-making.
• They play a crucial role in supporting the AppSim teams goals by enabling seamless access to reliable data for simplification initiatives.

What you will deliver

• As part of a cross-disciplinary team, you will collaborate with data engineers, software engineers, data scientists, data managers, and business partners to architect, design, implement, and maintain reliable and scalable data infrastructure for moving, processing, and serving data.
• You will write, deploy, and maintain software to build, integrate, manage, and assure the quality of data. Adhering to software engineering best practices, you will ensure technical design, unit testing, monitoring, code reviews, and documentation are followed.
• You will also ensure the deployment of secure, well tested software that meets privacy and compliance standards, and improve the CI/CD pipeline.
• Additionally, you will be responsible for service reliability, on-call rotations, SLAs, and maintaining infrastructure as code. By containerizing server deployments and mentoring others, you'll actively contribute to improving developer velocity.

What you will need to be successful (experience and qualifications)

Essential

• Pyspark, Python, SQL, Architecture Design, Project Management
• Deep, hands-on expertise in designing, building, and maintaining scalable data infrastructure and products in complex environments
• Development experience in object-oriented programming languages (e.g., Python, Scala, Java, C#)
• Advanced knowledge of databases and SQL
• Experience designing and implementing large-scale distributed data systems
• Strong understanding of technologies across all stages of the data lifecycle
• Ability to manage stakeholders effectively and lead initiatives through technical influence
• Continuous learning approach
• BS degree in computer science or a related field (or equivalent experience)

Desired

• No prior experience in the energy industry required
• Experience with data quality assessment and validation, basic data governance and security, Automation
• Nice to have: Experience with streaming data, basics of AI, LLMs

About bp

Our purpose is to deliver energy to the world, today and tomorrow. For over 100 years, bp has focused on discovering, developing, and producing oil and gas in the nations where we operate. We are one of the few companies globally that can provide governments and customers with an integrated energy offering. Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050

or sooner!

We will ensure that individuals with disabilities are provided reasonable

accommodation to participate in the job application or interview process, to perform crucial job functions, and to receive other benefits and privileges of employment.