AWS Data Engineer

Bengaluru east

Posted:1 week ago

Job Description

Responsibilities and JD in brief along with additional criteria to be considered:
  • Design and develop scalable AWS Data Engineering solutions and cloud-native data platforms.
  • Build batch and real-time data pipelines using AWS Glue, PySpark, Databricks, S3, Kinesis , Redshift and Kafka/MSK.
  • Develop and optimize Databricks, Delta Lake and Spark workloads for large-scale data processing with Pyspark , Databricks transformations.
  • Implement streaming solutions using Amazon Kinesis Data Streams, Firehose and Analytics for near real-time data ingestion and processing.
  • Handle high-volume (TB/PB scale) data pipelines with a focus on performance, scalability and cost optimization.
  • Develop complex SQL transformations, data models and analytics-ready datasets.
  • Implement CI/CD, Infrastructure as Code (Terraform/CloudFormation) and DevOps best practices.
  • Ensure data quality, governance, security, monitoring and production support.
  • Collaborate with architects, stakeholders and cross-functional teams in Agile delivery environments.
  • Troubleshoot and optimize end-to-end AWS data platforms, streaming pipelines and cloud infrastructure
  • Strong hands-on AWS Data Engineering experience across large-scale cloud data platforms.
  • Expertise in AWS S3, AWS Glue, AWS Lambda, AWS Redshift, AWS Lake Formation, AWS CloudWatch and AWS IAM.
  • Strong experience in real-time and streaming data processing using Amazon Kinesis Data Streams, Kinesis Firehose, Kinesis Analytics, Amazon Kafka or equivalent streaming technologies.
  • Hands-on coding proficiency in Python, PySpark and SQL with strong optimization skills.
  • Experience building high-volume, low-latency data ingestion and transformation pipelines.
  • Strong experience in Databricks, Delta Lake, Structured Streaming and Spark Performance Tuning.
  • Experience handling TB/PB-scale datasets, partitioning strategies, file optimization, schema evolution and data lifecycle management.
  • Strong understanding of Data Lake, Lakehouse and Modern Data Platform architectures.
  • Practical data modeling experience for analytical, operational and reporting workloads.
  • Experience implementing CI/CD pipelines using AWS CodePipeline, CodeBuild, CodeDeploy, GitHub Actions or Jenkins.
  • Strong troubleshooting skills for production issues, performance bottlenecks and cost optimization.
  • Excellent client communication, stakeholder management and problem-solving skills.

Skills Required

AWS Data Engineering Cloud Native AWS Glue Pyspark Kafka Spark Data Processing Analytics Data Ingestion Cost Optimization SQL

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Infosys logo
Infosys

IT Services and IT Consulting

Bangalore Karnataka

RecommendedJobs for You

D

AWS Data Engineer

Dreamwings Technologies

Chennai

Sunlife logo

India

Softview Infotech logo

AWS Data Engineer

Softview Infotech

Chennai

Objectways logo

Bengaluru

ADP Private Limited logo

AWS Data Engineer

ADP Private Limited

Hyderabad