Gcp Data Engineer

Location : PAN India (Any Tech M office)

About the Role

Job Description GCP Data Engineer – India

In your role as an Engineer, you will work with change initiatives across the organization to help them design solutions that meet our architecture principles and drive the Bank towards its desired target state. You will work closely with the data modelers to implement various data ingestion and transformation patterns for the feeds coming in from core banking platforms to the warehousing system.

Responsibilities

• Work with change initiatives across the organization.
• Design solutions that meet architecture principles.
• Implement data ingestion and transformation patterns.
• Design job streams in Control-M or Apache airflow.
• Collaborate with cloud engineers for data distribution solutions.
• Understand interfacing application requirements and gain business signoffs.
• Control test management activities.
• Perform technology analyses and make recommendations.
• Conduct unit testing and support UAT testing.
• Handle periodic production release activities and documentation.

Qualifications

Experience of 5+ years in the following areas:

Required Skills

• Strong Programming skill in Python, SQL/PLSQL.
• Hands on experience with Pyspark for large scale distributed data processing.
• Solid understanding of Apache Airflow (DAG design, Scheduling, Orchestration).

Experience working with Google Cloud Platform especially:

• Big Query
• Postgres
• Cloud Storage
• Data Proc
• Cloud Composer
• GKE
• Knowledge of DevOps configuration management tools (TeamCity, Jenkins, uDeploy, Kubernetes, Maven etc).
• Building scalable data processing solution using Pyspark running on Dataproc/Dataflow.