Data Engineer

Techno Facts Solutions

8 - 13 years

16 - 27 Lacs

hyderabad chennai

Posted:-1 days ago| Platform:

Apply

Skills Required

data engineer aws glue pyspark lambda spark aws python

Work Mode

Hybrid

Job Type

Full Time

Job Description

Job Title:

Experience:

Location:

Mode of Work:

Senior Data Engineer

Key Responsibilities:

Design, develop, and maintain
ETL pipelines
using
AWS Glue
,
PySpark
, and
Lambda
functions.
Troubleshoot and resolve
Out of Memory (OOM)
and performance issues in Spark jobs.
Develop and optimize
real-time ETL processes
using AWS Lambda.
Implement and manage infrastructure using
Terraform
and
CloudFormation templates
.
Work with various
AWS services
such as
RDS
,
Redshift
,
IAM
, and
CloudWatch
for data processing and monitoring.
Write efficient
SQL
queries and manage complex data transformations.
Collaborate with cross-functional teams to design
data models
, ensure data quality, and support analytical and reporting needs.
Apply strong understanding of
database
and
data warehouse concepts
to design robust, scalable data architectures.

Required Skills:

Hands-on experience with
AWS Glue
and
Apache Spark
architecture.
Strong proficiency in
PySpark
and
Python
programming.
Proven experience handling
Spark OOM issues
and performance tuning.
Expertise in
AWS Lambda
,
Terraform
,
CloudFormation
,
RDS
,
Redshift
,
IAM
, and
CloudWatch
.
Strong
SQL
and
data modeling
skills.
Solid understanding of
data warehouse
and
relational database
concepts.
Ability to work in a
hybrid environment
and collaborate effectively with global teams.

Mock Interview

Practice Video Interview with JobPe AI

Start Data Engineer Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.