Brillio logo

Lead Data Engineer - R01553055

5 - 7 years

0 Lacs

bengaluru karnataka india

Posted:2 weeks ago| Platform: Foundit logo

Apply

Skills Required

airflow dbt

Work Mode

Remote

Job Type

Full Time

Job Description

Lead Data Engineer

Primary Skills

  • GCP, Big Query, Python, Airflow, SQL, DBT
Job requirements
  • About the Role
  • We are seeking a Senior Data Engineer with deep expertise in Google Cloud Platform (GCP) and BigQuery to lead cloud modernization initiatives, develop scalable data pipelines, and enable real-time data processing for enterprise-level systems. This is a high-impact role focused on driving the transformation of legacy infrastructure into a robust, cloud-native data ecosystem.
  • Key Responsibilities
  • 1. Data Migration & Cloud Modernization
  • Analyze legacy on-premises and hybrid cloud data warehouse environments (e.g., SQL Server).
  • Lead the migration of large-scale datasets to Google BigQuery.
  • Design and implement data migration strategies ensuring data quality, integrity, and performance.
  • 2. Data Integration & Streaming
  • Integrate data from various structured and unstructured sources, including APIs, relational databases, and IoT devices.
  • Build real-time streaming pipelines for large-scale ingestion and processing of IoT and telemetry data.
  • 3. ETL / Data Pipeline Development
  • Modernize and refactor legacy SSIS packages into cloud-native ETL pipelines.
  • Develop scalable, reliable workflows using Apache Airflow, Python, Spark, and GCP-native tools.
  • Ensure high-performance data transformation and loading into BigQuery for analytical use cases.
  • 4. Programming & Query Optimization
  • Write and optimize complex SQL queries, stored procedures, and scheduled jobs within BigQuery.
  • Develop modular, reusable transformation scripts using Python, Java, Spark, and SQL.
  • Continuously monitor and optimize query performance and cost efficiency in the cloud data environment.
  • Required Skills & Experience
  • 5+ years in Data Engineering with a strong focus on cloud and big data technologies.
  • Minimum 2+ years of hands-on experience with GCP, specifically BigQuery.
  • Proven experience migrating on-premise data systems to the cloud.
  • Strong development experience with Apache Airflow, Python, and Apache Spark.
  • Expertise in streaming data ingestion, particularly in IoT or sensor data environments.
  • Strong SQL development skills; experience with BigQuery performance tuning.
  • Solid understanding of cloud architecture, data modeling, and data warehouse design.
  • Familiarity with Git and CI/CD practices for managing data pipelines.
  • Preferred Qualifications
  • GCP Professional Data Engineer certification.
  • Experience with modern data stack tools like dbt, Kafka, or Terraform.
  • Exposure to ML pipelines, analytics engineering, or DataOps/DevOps methodologies.
  • Why Join Us
  • Work with cutting-edge technologies in a fast-paced, collaborative environment.
  • Lead cloud transformation initiatives at scale.
  • Competitive compensation and benefits.
  • Remote flexibility and growth opportunities.

More Jobs at Brillio

React Js Frontend Developer

Pune, Bengaluru

4 - 8 yrs
|

INR 14 - 20 Lacs

Java Full Stack Developer

Pune, Bengaluru

3 - 8 yrs
|

INR 14 - 20 Lacs

Senior Software Development Lead - R01536938

Gurgaon, Haryana, India

Experience: Not specified

|

Salary: Not disclosed

Angular Lead

Hyderabad, Pune, Bengaluru

6 - 10 yrs
|

INR 14 - 20 Lacs

Network Perimeter Browsing - R01542500

Bengaluru, Karnataka, India

Experience: Not specified

|

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Google Play App Store
coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Brillio logo
Brillio

IT Services and IT Consulting

Dallas Texas

RecommendedJobs for You

Brillio logo

Lead Data Engineer - R01553055

Brillio

bengaluru, karnataka, india