Gcp Data Engineer

Location and Flexible locations - PAN India

Level of experience 5-7 years Core

• Experience in building multiple Data pipelines ETL and ELT process for Data ingestion, and transformation in GCP up to table level root warehouse layer i.e. BigQuery. Hands on exp in Python.
• Worked on creating various summary tables based on business logic and requirements in Big Query.
• In Big query, worked on framing queries based on business logic. Also worked on performing DML, SQL and DML operation in Big Query independently.
• Worked on creating a Dag in Cloud Composer.
• Scheduling & Monitoring jobs in Airflow
• Worked on manipulating root as per business requisition.
• Segregation, classification, and analysis of root as per business requisition.
• Scheduling the workflows as per business requirement
• Design and implement data pipelines using GCP services such as Dataflow, Dataproc, and Pub/Sub
• Develop and maintain data ingestion and transformation processes using tools like Apache Beam and Apache Spark
• Create and manage data storage solutions using GCP services such as BigQuery, Cloud Storage, and Cloud SQL
• Build and deploy machine learning models using GCP's AI Platform and TensorFlow
• Implement data security and access controls using GCP's Identity and Access Management (IAM) and Cloud Security Command Center
• Monitor and troubleshoot data pipelines and storage solutions using GCP's Stackdriver and Cloud Monitoring
• Collaborate with data scientists and analysts to understand their data requirements and provide solutions to meet their needs
• Automate data processing tasks using scripting languages such as Python and Bash
• Stay up-to-date with the latest GCP services and features and evaluate their potential use in the organization's data infrastructure
• Take part in team-building activities and provide suggestions

2.           Primary Skills in 5 liners that manager cannot negotiate on

•             Design and implement data pipelines using GCP services such as Dataflow, Dataproc, and Pub/Sub
•             Implementing and managing CI/CD pipelines using Jenkins, Cloud Build, and other automation tools
•             Create and manage data storage solutions using GCP services such as BigQuery, Cloud Storage, and Cloud SQL Good working understanding of Data Models & Meta Data Foundation
•             Python Scripting Hands on