Cloud Engineer - Abinitio

Senior Data Engineer - Abinitio, GCP

Hyderabad / Bangalore/ Chennai

-8 years

Full-Time

About Us

Key Responsibilities:

• ETL Development: Design and develop robust ETL pipelines using Ab Initio for efficient data ingestion, transformation, and processing.
• Big Data & Cloud Technologies: Implement Data Warehousing and Data Lake solutions on GCP to handle large-scale structured and unstructured datasets.
• Cloud Data Streaming: Design and develop real-time streaming solutions using GCP Pub/Sub, Dataflow, and BigQuery for data ingestion and analytics.
• Python Development: Write efficient, scalable, and reusable Python scripts for automation, data transformation, and integration with cloud services.
• Performance Optimization: Optimize queries, ETL workflows, and data processing pipelines to enhance efficiency and reduce latency.
• Collaboration & Best Practices: Work closely with data scientists, analysts, and cloud engineers to ensure seamless data flow and availability.
• Security & Compliance: Ensure data governance, security, and IAM policies are enforced across all data processing solutions.
  

Required Skills & Qualifications:

• 6-8 years of experience in data engineering and cloud-based data solutions.
• Expertise in Ab Initio for ETL pipeline design, implementation, and optimization.
• Proficiency in Python for data processing, scripting, and automation.
• Hands-on experience with Data Warehouses and Data Lakes (BigQuery, Snowflake, or similar).
• Strong experience in GCP Streaming technologies including Pub/Sub, Dataflow, and BigQuery for real-time data processing.
• Good understanding of cloud-based data architectures, including GCP components such as Cloud Storage, BigQuery, and Cloud Functions.
• Knowledge of batch and real-time data processing frameworks.
• Experience in SQL performance tuning and optimization for large-scale datasets.