Senior Data Engineer - BigQuery

As a senior data engineer, you will be responsible for designing and implementing data solutions while collaborating closely with cross-functional teams. Your role will involve ensuring data integrity, performance, and efficiency throughout the entire pipeline. Your key responsibilities will include designing, building, and maintaining scalable and efficient end-to-end data pipelines. This involves ensuring smooth data transformation, processing, and storage. You will also be required to implement and manage BigQuery transformations, utilizing SQL and stored procedures effectively. Collaboration with business stakeholders is crucial to understand data requirements, KPIs, and objectives. This will ensure that data pipelines are designed to meet specific business needs. Additionally, you will need to develop advanced SQL queries for data analysis, investigation, and troubleshooting purposes. Utilizing GCP services such as Cloud Composer, Cloud Scheduler, and Cloud Storage will be essential to streamline data workflows and processes. Designing data models with a focus on Star & Snowflake schemas, Fact & Dimension tables, and efficient data relationships is also a key aspect of the role. This is to ensure optimal query performance. Optimizing and tuning BigQuery queries to maximize performance and minimize costs will be part of your responsibilities. Implementing best practices for data governance, data quality, and security is also crucial to maintain data integrity. In terms of mandatory skills, you should have strong experience in end-to-end data pipeline development, advanced SQL expertise, extensive hands-on experience with BigQuery and data transformation, solid understanding of data modeling concepts, and proficiency in GCP services like Cloud Composer, Cloud Scheduler, and Cloud Storage. Nice to have skills include familiarity with CI/CD concepts and tools like Git and Terraform, experience with additional GCP services such as Dataflow, Cloud Build, Pub/Sub, Cloud Functions, Cloud Run, and Cloud Workstation, expertise in BigQuery performance tuning, experience with Python-based API development, and familiarity with Spark for large-scale data processing and transformations.,