Principal Data Engineer

Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things.
Job Title: Principal Data Engineer
Location: Manyata Tech Park, Bangalore
Business & Team: CTO -Engineering Platform

Impact & contribution: Here, you will create, manage and optimize the platforms used to power our entire business. You will have the opportunity to take responsibility for continuous improvement, ensuring our systems are simpler, faster and more secure. The cloud movement at CommBank is going strong and continues to grow. We are looking for out of the box thinkers who want to use technology to work on real-world problems that have the potential to change the lives of our 17 million+ customers.

We support our people with the flexibility to balance where work is done with at least half your time each month connecting in office. We also have many other flexible working options available including changing start and finish times, part-time arrangements and job share to name a few. Talk to us about how these arrangements might work for you As a Principal Engineer you will be responsible for various aspects such as Data Engineering mindset, DevSecOps mindset, and Engineering Quality and Productivity mindset. They also contribute to addressing high and medium-rated issues and planned work addressing underlying actions.

Roles & responsibilities:

· Lead the design and architecture of scalable, maintainable, and secure full-stack applications.

• Collaborate with product managers, designers, and other stakeholders to translate business requirements into technical solutions.

• Ensure adherence to best practices in software design, including modularity, reusability, and maintainability.

• Utilise AWS services such as S3, Lambda, Glue, Step Functions, and CloudWatch to build and manage data solutions.

• Implement best practices for AWS security, cost management, and performance optimisation.

• Stay updated with the latest AWS offerings and evaluate their potential impact on the data infrastructure.

• Lead the design and implementation of data management solutions, ensuring data integrity, quality, and accessibility.

• Develop and maintain data models, data pipelines, and ETL processes.

• Implement data governance and compliance measures to adhere to regulatory requirements.

Essential skills:

. Minimum 15+Years of experience

• Utilize AWS services such as S3, Lambda, Glue, Step Functions, and CloudWatch to build and manage data solutions.

• Implement best practices for AWS security, cost management, and performance optimization.

• Stay updated with the latest AWS offerings and evaluate their potential impact on our data infrastructure.

• Demonstrated expertise in dashboard and backend development, with a strong focus on SQL.

• Proficient in JavaScript, Python, SQL, and DBT.

• Extensive experience in building data lakes and data warehouses.

• Mastery of both SQL and NoSQL databases, data ingestion processes, ETL pipelines, and data integration.

• Architect and implement real-time data streaming solutions using technologies such as Apache Kafka, AWS Kinesis, or similar.

• Ensure high availability, scalability, and low latency of data streaming pipelines.

• Collaborate with data scientists and analysts to enable real-time data analytics and insights.

• Proficient in data profiling techniques and tools.

• Lead the design and implementation of data management solutions, ensuring data integrity, quality, and accessibility.

• Develop and maintain data models, data pipelines, and ETL processes.

• Implement data governance and compliance measures to adhere to regulatory requirements.

Education Qualification: Bachelor’s degree or master’s degree in engineering in Computer Science/Information Technology.