Sr Data Engineer

As a seasoned Senior Data Engineer, your role will involve architecting and implementing end-to-end data pipelines on Google Cloud Platform (GCP) and AWS, focusing on leveraging BigQuery and Databricks. Your expertise in modern data engineering practices will be crucial in driving data quality and governance. Your responsibilities will include: - Architecting, developing, and maintaining ETL/ELT data pipelines primarily on GCP using BigQuery - Designing and implementing data ingestion frameworks with tools like Google Cloud Dataflow, Pub/Sub, Kafka, Fivetran, or Talend - Leading and optimizing data transformation processes using SQL, dbt, Apache Spark, or Dataflow to implement layered data architectures - Collaborating cross-functionally with data scientists, analysts, and business teams to translate requirements into scalable data assets - Building and automating workflows using orchestration tools like Google Cloud Composer or Apache Airflow - Managing data governance, lineage, and security practices to ensure compliance with internal policies and regulatory standards - Conducting root cause analysis and troubleshooting data issues to maintain high data integrity - Driving cloud cost optimization strategies related to data storage and compute resources Additionally, your experience with Databricks on AWS for Spark-based large-scale data processing and machine learning pipelines will be highly valued. You will also have the opportunity to mentor junior engineers and contribute to developing best practices and documentation for data engineering processes.,