Data Engineer

SP Software

5 - 9 years

8 - 18 Lacs

bengaluru

Posted:2 months ago| Platform:

Apply

Skills Required

pyspark snowflake datawarehouse aws sql data engineering spark snowpipe

Work Mode

Hybrid

Job Type

Full Time

Job Description

Role & responsibilities

Job Summary:

We are seeking a highly skilled Data Engineer to design, build, and optimize scalable data pipelines using Snowflake, AWS services, and Apache Spark. You will be responsible for real-time and batch data ingestion, transformation, and orchestration across cloud platforms.

Key Responsibilities:

Develop and maintain data pipelines using
AWS Glue
,
Lambda
,
EMR
, and
Snowflake
.
Implement real-time ingestion using
Snowpipe
and
Streams
for CDC (Change Data Capture).
Write efficient
PySpark
or
Scala Spark
jobs for large-scale data processing.
Automate workflows and orchestrate jobs using
AWS Step Functions
,
Airflow
, or similar tools.
Optimize Snowflake queries and warehouse performance.
Collaborate with Data Scientists, Analysts, and DevOps teams to deliver reliable data solutions.
Monitor and troubleshoot data pipeline failures and latency issues.

Required Skills:

Strong experience with
Snowflake
architecture, SQL, and performance tuning.
Hands-on expertise in
AWS Glue
,
Lambda
,
S3
,
EMR
, and
CloudWatch
.
Proficiency in
Apache Spark
(PySpark or Scala).
Familiarity with
Snowpipe
,
Streams
, and
Tasks
in Snowflake.
Knowledge of CI/CD tools and infrastructure-as-code (Terraform, CloudFormation).
Experience with version control (Git) and agile development practices.

Preferred candidate profile

Snowflake, Spark with AWS Services, Pyspark,SQL

More Jobs at SP Software

Rpa Developer

Bengaluru

3 - 6 yrs

INR 7 - 17 Lacs

SSRS Developer - Contract-to-hire

Gurugram, Bengaluru

3 - 6 yrs

INR 7 - 9 Lacs

Technical Recruiter

Hyderabad

1 - 2 yrs

INR 2 - 3 Lacs

Hr Executive And Generalist

Hyderabad

Experience: Not specified

INR 1 - 3 Lacs

Adobe Analyst

Gurugram, Bengaluru

3.0 - 8.0 yrs

INR 9 - 16 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.