Principal Data Engineer

Role:

Experience:

Location:

About the Role:

Principal Data Engineer

Key Responsibilities:

• Architect and lead the development of scalable, fault-tolerant, and secure data infrastructure across streaming and batch pipelines.
• Design and implement enterprise-grade ETL/ELT frameworks to handle data ingestion, transformation, and enrichment from diverse internal and external sources.
• Build and scale data lakes, data warehouses, and lakehouse architectures to support high-volume, low-latency analytics and AI/ML use cases.
• Partner closely with product, analytics, engineering, and data science teams to deliver strategic data solutions aligned with business goals.
• Drive standards for data governance, lineage, access control, quality, and observability across systems.
• Lead modernization of data stack by evaluating and integrating emerging technologies (e.g., Iceberg, Delta Lake, Apache Flink, etc.).
• Optimize performance, reliability, and cost-efficiency of data workflows using best-in-class DevOps and MLOps practices.
• Own technical design reviews, architectural decisions, roadmap planning, and POCs for new data initiatives.
• Mentor and guide a team of data engineers, fostering a culture of excellence, ownership, and innovation.
• Ensure best practices in code quality, documentation, testing, and deployment across the team.

Required Skills & Qualifications:

• 10+ years of deep, hands-on experience in data engineering or backend systems involving distributed architecture.
• Proven expertise in Python, Scala, or Java with focus on scalable and modular design.
• Extensive experience with modern data processing frameworks such as Apache Spark, Apache Flink, or Beam.
• Strong background in AWS cloud services (S3, EMR, Redshift, Glue, Lambda, Athena, IAM, etc.); experience with infrastructure-as-code tools like Terraform is a plus.
• Experience in data orchestration using Apache Airflow or equivalent tools like Argo, Dagster, or Prefect.
• Proficient in designing real-time and batch data solutions using Kafka, Debezium, Kinesis, or similar technologies.
• Advanced SQL skills and hands-on knowledge of both relational (PostgreSQL, Redshift) and NoSQL databases (MongoDB, DynamoDB).
• Experience working with containerized environments using Kubernetes and CI/CD workflows.
• Strong understanding of data modeling, security, compliance, and access control at scale.
• Demonstrated ability to lead complex data projects and mentor cross-functional engineering teams.
• Excellent system design, architecture thinking, and communication skills with a strong sense of ownership.