Data Engineering III-HO & Support - Corporate Centre of Excellence

About our team

DEX is a central data org for Kotak Bank which manages entire data experience of Kotak Bank. DEX stands for Kotaks Data Exchange. This org comprises of Data Platform, Data Engineering and Data Governance charter. The org sits closely with Analytics org. DEX is primarily working on greenfield project to revamp entire data platform which is on premise solutions to scalable AWS cloud-based platform. The team is being built ground up which provides great opportunities to technology fellows to build things from scratch and build one of the best-in-class data lake house solutions. The primary skills this team should encompass are Software development skills preferably Python for platform building on AWS; Data engineering Spark (pyspark, sparksql, scala) for ETL development, Advanced SQL and Data modelling for Analytics.

As a member of this team, you get opportunity to learn fintech space which is most sought-after domain in current world, be a early member in digital transformation journey of Kotak, learn and leverage technology to build complex data data platform solutions including, real time, micro batch, batch and analytics solutions in a programmatic way and also be futuristic to build systems which can be operated by machines using AI technologies.

The data platform org is divided into 3 key verticals

Data Platform

This Vertical is responsible for building data platform which includes optimized storage for entire bank and building centralized data lake, managed compute and orchestrations framework including concepts of serverless data solutions, managing central data warehouse for extremely high concurrency use cases, building connectors for different sources, building customer feature repository, build cost optimization solutions like EMR optimizers, perform automations and build observability capabilities for Kotaks data platform. The team will also be center for Data Engineering excellence driving trainings and knowledge sharing sessions with large data consumer base within Kotak.

Data Engineering

This team will own data pipelines for thousands of datasets, be skilled to source data from 100+ source systems and enable data consumptions for 30+ data analytics products. The team will learn and built data models in a config based and programmatic and think big to build one of the most leveraged data model for financial orgs. This team will also enable centralized reporting for Kotak Bank which cuts across multiple products and dimensions. Additionally, the data build by this team will be consumed by 20K + branch consumers, RMs, Branch Managers and all analytics usecases.

Data Governance

The team will be central data governance team for Kotak bank managing Metadata platforms, Data Privacy, Data Security, Data Stewardship and Data Quality platform.

If youve right data skills and are ready for building data lake solutions from scratch for high concurrency systems involving multiple systems then this is the team for you.

You day to day role will include

• Drive business decisions with technical input and lead the team.
• Design, implement, and support an data infrastructure from scratch.
• Manage AWS resources, including EC2, EMR, S3, Glue, Redshift, and MWAA.
• Extract, transform, and load data from various sources using SQL and AWS big data technologies.
• Explore and learn the latest AWS technologies to enhance capabilities and efficiency.
• Collaborate with data scientists and BI engineers to adopt best practices in reporting and analysis.
• Improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.
• Build data platforms, data pipelines, or data management and governance tools.

BASIC QUALIFICATIONS for Data Engineer

• Bachelor's degree in Computer Science, Engineering, or a related field
• 3-5 years of experience in data engineering
• Strong understanding of AWS technologies, including S3, Redshift, Glue, and EMR
• Experience with data pipeline tools such as Airflow and Spark
• Experience with data modeling and data quality best practices
• Excellent problem-solving and analytical skills
• Strong communication and teamwork skills
• Experience in at least one modern scripting or programming language, such as Python, Java, or Scala
• Strong advanced SQL skills

PREFERRED QUALIFICATIONS

• AWS cloud technologies: Redshift, S3, Glue, EMR, Kinesis, Firehose, Lambda, IAM, Airflow
• Prior experience in Indian Banking segment and/or Fintech is desired.
• Experience with Non-relational databases and data stores
• Building and operating highly available, distributed data processing systems for large datasets
• Professional software engineering and best practices for the full software development life cycle
• Designing, developing, and implementing different types of data warehousing layers
• Leading the design, implementation, and successful delivery of large-scale, critical, or complex data solutions
• Building scalable data infrastructure and understanding distributed systems concepts
• SQL, ETL, and data modelling
• Ensuring the accuracy and availability of data to customers
• Proficient in at least one scripting or programming language for handling large volume data processing
• Strong presentation and communications skills.
• For Managers,
  • Customer centricity, obsession for customer
  • Ability to manage stakeholders (product owners, business stakeholders, cross function teams) to coach agile ways of working.
  • Ability to structure, organize teams, and streamline communication.
  • Prior work experience to execute large scale Data Engineering projects