Job
Description
Minimum 7+ years in data engineering with 5+ years of hands-on experience on GCP. Proven track record with tools and services like BigQuery, Cloud Composer (Apache Airflow), Cloud Functions, Pub/Sub, Cloud Storage, Dataflow, and IAM/VPC. Demonstrated expertise in Apache Spark (batch and streaming), PySpark, and building scalable API integrations. Advanced Airflow skills including custom operators, dynamic DAGs, and workflow performance tuning. Certifications Google Cloud Professional Data Engineer certification preferred. Key Skills Mandatory Technical Skills Advanced Python (PySpark, Pandas, pytest) for automation and data pipelines. Strong SQL with experience in window functions, CTEs, partitioning, and optimization. Proficiency in GCP services including BigQuery, Dataflow, Cloud Composer, Cloud Functions, and Cloud Storage. Hands-on with Apache Airflow, including dynamic DAGs, retries, and SLA enforcement. Expertise in API data ingestion, Postman collections, and REST/GraphQL integration workflows. Familiarity with CI/CD workflows using Git, Jenkins, or Bitbucket. Experience with infrastructure security and governance using IAM and VPC.
