5 years
0 Lacs
Posted:3 weeks ago|
Platform:
On-site
Full Time
Seeking a senior AWS Data Engineer with deep Snowflake expertise and 5 - 6+ years of experience to design, build, and optimize scalable cloud data platforms on AWS and Snowflake.
You will own end-to-end modern data pipelines, integrate diverse sources, and deliver secure, high-performance analytics solutions in collaboration with data science and
business teams.
Key Responsibilities
Design and implement scalable ETL/ELT pipelines using AWS Glue, Lambda, Step Functions, and EventBridge to ingest, transform, and load data into Snowflake and S3 data lakes.
Ingest heterogeneous sources: S3, SFTP (AWS Transfer Family), REST/SOAP/FAST APIs, relational databases, Kinesis real-time streams, and third-party SaaS platforms.
Build and maintain Snowflake-centric architecture: staging warehouse consumption layers, leveraging virtual warehouses, zero-copy cloning, and time travel.
Optimize Snowflake performance (clustering keys, materialized views, search optimization, query profiling) and costs (auto-suspend, resource monitors, serverless tasks).
Optimize Redshift query performance, storage, and compute costs manage S3 data lake zoning (raw refined curated).
Ensure data quality, consistency, lineage, and reliability via validation frameworks, idempotent processing, and automated testing.
Orchestrate complex event-driven workflows with Step Functions and EventBridge implement CI/CD using CodePipeline and GitHub Actions with Git version control.
Implement enterprise-grade security and compliance: VPC isolation, IAM least-privilege roles/policies, Secrets Manager, KMS encryption, PrivateLink, and CloudTrail auditing.
Set up comprehensive monitoring, alerting (CloudWatch, Snowsight), logging, and automated recovery/failover for production pipelines.
Maintain AWS Glue Data Catalog/Lake Formation for governance, metadata, and access control produce detailed technical documentation and runbooks.
Collaborate with data scientists, analysts, and business stakeholders to translate requirements into robust data models and delivery schedules.
Continuously identify and resolve performance bottlenecks, cost inefficiencies, and technical debt through optimization and refactoring initiatives.
Required Qualifications
Bachelors/Masters degree in Computer Science, Engineering, or related field.
5-6+ years of hands-on data engineering experience with at least 4 years focused on AWS cloud platforms.
Expert in Snowflake (architecture, Snowpipe, Streams/Tasks, Snowpark, cost & performance tuning).
Expert-level proficiency in Python and SQL for data transformation, scripting, and pipeline development.
Deep expertise with core AWS data services: Glue (Crawlers, Jobs, Spark), Lambda, Step Functions, EventBridge, S3, Kinesis, API Gateway, EC2, Athena, Lake Formation,
Redshift, Snowflake integration.
Strong experience with security services: IAM, VPC, KMS, Secrets Manager, Transfer Family, CloudTrail, GuardDuty.
Proven track record of building and supporting production ETL/ELT pipelines handling terabyte-scale data with strict SLAs.
Preferred Qualifications
AWS Certified Data Engineer Associate or Professional (or equivalent Speciality certifications).
SnowPro Associate or Professional (or equivalent Speciality certifications).
Experience with Snowpark, dbt, Airflow (MWAA), real-time analytics, and streaming (Kinesis + Lambda/MSK).
Familiarity with data governance tools, cost optimisation (Savings Plans, Reserved Instances, S3 Intelligent-Tiering), and big-data ecosystems (Spark, EMR).
Excellent communication skills and experience mentoring junior engineers.
Omni Recruit Private Limited
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Practice Python coding challenges to boost your skills
Start Practicing Python Nowpune, maharashtra, india
Salary: Not disclosed
pune, maharashtra, india
Salary: Not disclosed