Principal Data Engineer

6 years

0 Lacs

Posted:5 hours ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Contractual

Job Description

Job Title:

Contract Period:

Experience:

Working hours:


About the Role

We are looking for an accomplished Principal Data Engineer to lead the ideation, architecture,

design and development of our next-generation enterprise data platform. This role requires a

visionary technologist with deep hands-on expertise in cloud data engineering (GCP preferred),

data architecture, and software engineering.

You will work closely with cloud, security, and infrastructure architects to ensure seamless

alignment with the organization's broader technology and business strategies.


Key Responsibilities:

  • Lead the end-to-end architecture and development of a modern enterprise-scale data platform
  • from the ground up.
  • Collaborate with cloud and security architects to ensure platform scalability, performance, and
  • compliance.
  • Architect, design, and implement batch and real-time streaming data infrastructures and
  • workloads.
  • Build and maintain data lakehouse architectures within the GCP ecosystem.
  • Design and develop connector frameworks and reusable data ingestion pipelines to source data
  • from both on-premises and cloud systems.
  • Architect and implement metadata management, including data catalogs, lineage, quality, and
  • observability frameworks.
  • Design and develop data quality frameworks and governance processes to ensure reliability and
  • accuracy.
  • Develop microservices-based components using Kubernetes, Docker, and Cloud Run to
  • abstract platform and infrastructure complexities.
  • Design and optimize data storage, transformation, and querying performance for large-scale
  • datasets while ensuring cost efficiency.
  • Implement observability tooling (Grafana, Datadog) and DataOps best practices, including

CI/CD and test automation.

  • Collaborate with data scientists and analysts to define data models, schemas, and advanced
  • analytics capabilities.
  • Drive deployment, release management, and platform scalability initiatives.
  • Stay ahead of emerging data engineering trends, tools, and best practices to continuously
  • evolve the platform.


Required Skills & Experience

  • 6+ years of proven experience in modern cloud data engineering and enterprise data platform
  • architecture.
  • Demonstrated success in architecting and delivering large-scale greenfield data platform
  • projects.
  • Deep expertise in Google Cloud Platform (GCP) and its ecosystem — BigQuery, Cloud
  • Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, and Airflow.
  • Strong understanding of streaming technologies such as Kafka or Pub/Sub.
  • Hands-on experience with microservices architectures using Kubernetes, Docker, and Cloud
  • Run.
  • Proven ability to design semantic layers and metadata-driven architectures.
  • Expertise in data modeling, data architecture, and data governance principles.
  • Experience with observability and monitoring tools (Grafana, Datadog).
  • Strong understanding of DataOps principles, including automation, CI/CD, and testing for data
  • pipelines.
  • Experience architecting secure, scalable, and high-performance data solutions.

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now

RecommendedJobs for You