Aws Data Engineer

Company Name:

Job Title:

Experience:

Location:

Work Mode:

Salary:

Employment Type:

Role Overview

The Senior Data Engineer designs, builds, and optimizes scalable, high-performance data platforms on AWS using Python. The role owns end-to-end data pipelines across batch and streaming, enabling analytics and AI workloads for cross-functional teams. It emphasizes modern data architecture, data quality, governance, and automation with strong focus on cost and performance. The engineer leads technical decisions, conducts code reviews, mentors junior staff, and sets best practices for reliability and observability. Success is measured by robust data products, efficient pipelines, and secure, compliant data operations.

Required Qualifications

• Expert-level Python for data engineering (pandas, PySpark, boto3, SQLAlchemy)
• Advanced AWS data services: Glue, Lambda, EMR, Step Functions, DynamoDB, Redshift, Athena, S3, Kinesis, QuickSight
• Strong SQL, data modeling, and performance tuning
• Hands-on design of data lakes and data warehouses on AWS
• ETL/ELT best practices including partitioning, error handling, and validation
• Data quality, observability, and lineage (AWS Data Catalog, Glue Data Quality or third-party)
• Data governance, security, and compliance (IAM, encryption/KMS, access control)
• CI/CD for data pipelines (CodePipeline, GitHub Actions, Cloud Build) and Git version control
• Infrastructure as Code (CloudFormation or Terraform)
• Containerization and DevOps concepts (Docker/Kubernetes)
• Integration of diverse data sources (RDBMS, APIs, Kafka/Kinesis, SaaS)
• Monitoring and troubleshooting with CloudWatch; capacity planning

Responsibilities

• Architect and implement scalable, fault-tolerant data pipelines with AWS Glue, Lambda, EMR, Step Functions, and Redshift
• Build and optimize data lakes and warehouses on S3, Redshift, and Athena, including partitioning and schema design
• Develop Python-based ETL/ELT frameworks and reusable transformation modules
• Integrate RDBMS, APIs, Kafka/Kinesis, and SaaS sources into unified data models
• Lead data modeling and design for performance and cost efficiency
• Drive data quality, observability, and lineage using AWS Data Catalog, Glue Data Quality, or third-party tools
• Define and enforce data governance, IAM policies, encryption, and access control
• Collaborate with Data Science, Analytics, Product, and DevOps to support analytical and ML workloads
• Implement CI/CD for data workflows using CodePipeline, GitHub Actions, or Cloud Build
• Provide technical leadership, code reviews, and mentoring to junior engineers
• Monitor and tune data infrastructure; troubleshoot incidents; lead capacity planning