8 Gcp Bigquery Jobs

Role Overview We are seeking a talented ETL Engineer to design, implement, and maintain end-to-end data ingestion and transformation pipelines in Google Cloud Platform (GCP). This role will collaborate closely with data architects, analysts, and BI developers to ensure high-quality, performant data delivery into BigQuery and downstream Power BI reporting layers. Key Responsibilities Data Ingestion & Landing Architect and implement landing zones in Cloud Storage for raw data. Manage buckets/objects and handle diverse file formats (Parquet, Avro, CSV, JSON, ORC). ETL Pipeline Development Build and orchestrate extraction, transformation, and loading workflows using Cloud Data Fusion. Leverage Data Fusion Wrangler for data cleansing, filtering, imputation, normalization, type conversion, splitting, joining, sorting, union, pivot/unpivot, and format adjustments. Data Modeling Design and maintain fact and dimension tables using Star and Snowflake schemas. Collaborate on semantic layer definitions to support downstream reporting. Load & Orchestration Load curated datasets into BigQuery across different zones (raw, staging, curated). Develop SQL-based orchestration and transformation within BigQuery (scheduled queries, scripting). Performance & Quality Optimize ETL jobs for throughput, cost, and reliability. Implement monitoring, error handling, and data quality checks. Collaboration & Documentation Work with data analysts and BI developers to understand requirements and ensure data readiness for Power BI. Maintain clear documentation of pipeline designs, data lineage, and operational runbooks. Required Skills & Experience Bachelors degree in Computer Science, Engineering, or related field. 3+ years of hands-on experience building ETL pipelines in GCP. Proficiency with Cloud Data Fusion , including Wrangler transformations. Strong command of SQL , including performance tuning in BigQuery. Experience managing Cloud Storage buckets and handling Parquet, Avro, CSV, JSON, and ORC formats. Solid understanding of dimensional modeling: fact vs. dimension tables, Star and Snowflake schemas. Familiarity with BigQuery data zones (raw, staging, curated) and dataset organization. Experience with scheduling and orchestration tools (Cloud Composer, Airflow, or BigQuery scheduled queries). Excellent problem-solving skills and attention to detail. Preferred (Good to Have) Exposure to Power BI data modeling and DAX. Experience with other GCP services (Dataflow, Dataproc). Familiarity with Git, CI/CD pipelines, and infrastructure as code (Terraform). Knowledge of Python for custom transformations or orchestration scripts. Understanding of data governance best practices and metadata management.

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Location: Gurgaon/Pune/Hyderabad/Bengaluru/Chennai Work Mode: Hybrid (2-3 days office in a week) Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Good understanding of Big Data querying tools, such as Hive, and Impala Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files Good understanding of SQL queries, joins, stored procedures, relational schemas Experience with NoSQL databases, such as HBase, Cassandra, MongoDB Knowledge of ETL techniques and frameworks Performance tuning of Spark Jobs Experience with native Cloud data services AWS/Azure/GCP Ability to lead a team efficiently Experience with designing and implementing Big data solutions Practitioner of AGILE methodology WE OFFER Opportunity to work on technical challenges that may impact across geographies Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications Opportunity to share your ideas on international platforms Sponsored Tech Talks & Hackathons Possibility to relocate to any EPAM office for short and long-term projects Focused individual development Benefit package: • Health benefits, Medical Benefits• Retirement benefits• Paid time off• Flexible benefits Forums to explore beyond work passion (CSR, photography, painting, sports, etc

EPAM has presence across 40+ countries globally with 55,000 + professionals & numerous delivery centers, Key locations are North America, Eastern Europe, Central Europe, Western Europe, APAC, Mid East & Development Centers in India (Hyderabad, Pune & Bangalore). Location: Gurgaon/Pune/Hyderabad/Bengaluru/Chennai Work Mode: Hybrid (2-3 days office in a week) Job Description: 5-14 Years of in Big Data & Data related technology experience Expert level understanding of distributed computing principles Expert level knowledge and experience in Apache Spark Hands on programming with Python Proficiency with Hadoop v2, Map Reduce, HDFS, Sqoop Experience with building stream-processing systems, using technologies such as Apache Storm or Spark-Streaming Good understanding of Big Data querying tools, such as Hive, and Impala Experience with integration of data from multiple data sources such as RDBMS (SQL Server, Oracle), ERP, Files Good understanding of SQL queries, joins, stored procedures, relational schemas Experience with NoSQL databases, such as HBase, Cassandra, MongoDB Knowledge of ETL techniques and frameworks Performance tuning of Spark Jobs Experience with native Cloud data services AWS/Azure/GCP Ability to lead a team efficiently Experience with designing and implementing Big data solutions Practitioner of AGILE methodology WE OFFER Opportunity to work on technical challenges that may impact across geographies Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications Opportunity to share your ideas on international platforms Sponsored Tech Talks & Hackathons Possibility to relocate to any EPAM office for short and long-term projects Focused individual development Benefit package: • Health benefits, Medical Benefits• Retirement benefits• Paid time off• Flexible benefits Forums to explore beyond work passion (CSR, photography, painting, sports, etc

Night Shift: 9:00PM to 6:00AM Hybrid Mode: 3 days WFO & 2 Days WFH Job Overview We are looking for a savvy Data Engineer to manage in-progress and upcoming data infrastructure projects. The candidate will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder using Python and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. Responsibilities for Data Engineer * Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements using Python and SQL / AWS / Snowflakes. * Identify, design, and implement internal process improvements through: automating manual processes using Python, optimizing data delivery, re-designing infrastructure for greater scalability, etc. * Build the infrastructure required for optimal extraction, transformation, and loading ofdata from a wide variety of data sources using SQL / AWS / Snowflakes technologies. * Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics. * Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. * Keep our data separated and secure across national boundaries through multiple data centers and AWS regions. * Work with data and analytics experts to strive for greater functionality in our data systems. Qualifications for Data Engineer * Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. * Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement. * Strong analytic skills related to working with unstructured datasets. * Build processes supporting data transformation, data structures, metadata, dependency and workload management. * A successful history of manipulating, processing and extracting value from large disconnected datasets. Desired Skillset:- * 2+ years of experience in a Python Scripting and Data specific role, with Bachelor degree. * Experience with data processing and cleaning libraries e.g. Pandas, numpy, etc., web scraping/ web crawling for automation of processes, APIs and how they work. * Debugging code if it fails and find the solution. Should have basic knowledge of SQL server job activity monitoring and of Snowflake. * Experience with relational SQL and NoSQL databases, including PostgreSQL and Cassandra. * Experience with most or all the following cloud services: AWS, Azure, Snowflake,Google Strong project management and organizational skills. * Experience supporting and working with cross-functional teams in a dynamic environment.

Skills : Data analytics projects, advanced analytics, data science, data- driven digital products, AI/ML/cognitive solutioning, assessment, road-mapping, transformation, data science implementation Required Candidate profile tools such as Tableau, Power BI, matplotlib, GCP BigQuery, SQL, Huggingface

Data Pipeline Development:- Design & guiding developers in develop and maintain data processing pipelines using PySpark. Collaborate with data engineers and analysts to integrate data from various sources. Implement data transformations, aggregations, and enrichment processes. Performance Optimization Guiding in optimize Spark jobs for performance and scalability. Monitor and troubleshoot performance issues in Spark applications. Implement best practices for efficient Spark code and resource utilization Data Model Design & Development Understanding of Life Insurance Industry leading to design & establish data models in GCP cloud and adhering to enterprise standards. Evaluate data related tools and technologies and recommend appropriate implementation patterns and standards methodologies to ensure latest and modern data architecture & data rules implementation. Create and maintain conceptual / logical data models to identify key business entities and visual relationship resulting in efficient data pipelines and optimized and non-redundant data. Reviewing data models with both technical and business audiences Data Quality and Integrity: ** Ensure data quality and integrity through rigorous testing and validation. Implement data validation and cleansing processes. Design & develop automated data quality checks and monitoring systems

Location : Chennai/Bangalore/Noida/Pune/Hyderabad Analyzing large volumes of data, clean and represent them in Dashboards, Reports Designing and optimizing data storage solutions, including data warehouses and data lakes. Ensuring data quality and integrity through data validation, cleansing, and error handling. Collaborating with data analysts, data architects, and software engineers to understand data requirements and deliver relevant data sets (e.g., for business intelligence). Keeping abreast of industry trends and emerging technologies in data analytics. Documenting data pipelines, processes, and best practices for knowledge sharing. Participating in data governance and compliance efforts to meet regulatory requirements. Providing technical support and mentoring to junior data engineers, if applicable. Continuously optimizing data architecture to support the company's evolving data needs. Collaborating with cross-functional teams to drive data-driven decision-making within the organization. Strong communication skills and ability to translate complex technical topics to senior stakeholders. Excellent knowledge of cloud computing technologies and current computing trends. Understanding of and experience with well-architected frameworks. Proven ability to collaborative with multi-disciplinary teams of Product Managers/Owners, Architects, Scrum masters and Subject Matter Experts Desired Azure,AWS and/or GCP Architect certifications and proven hands-on experience Bachelors degree in computer engineering, or equivalent with extensive practical experience. Specific Skills: Proficiency in Data Analytics tools such as GCP BigQuery, Looker / powerBI dashboards Proficiency in processing huge volumes of data, building statistical models Proficiency in data modeling and database management. Strong programming skills (e.g., Python, Java, or SQL). Experience with ETL (Extract, Transform, Load) processes. Familiarity with data warehousing and cloud platforms (e.g., AWS, Azure, or Google Cloud). Degrees in computer science or related field

Location : Chennai/Bangalore/Noida/Hyderabad Hive, Python, Java or SQL, Hadoop, Spark, ETL, GCP BigQuery, Cloud SQL, Dataflow, Dataproc, Cloud build, cloud run, cloud functions, pub-sub, cloud composer Data lake, Multi cloud (e.g.,Google Cloud, AWS, Azure)