Data Engineer

Techno Facts Solutions

5 - 9 years

10 - 12 Lacs

bengaluru

Posted:Just now| Platform:

Apply

Skills Required

data bricks airflow kafka apache elt sql git aws lambda orchestration framework spark etl aws python

Work Mode

Remote

Job Type

Full Time

Job Description

Role & responsibilities

Build and maintain
scalable data pipelines
using
Databricks
and
Apache Spark
.
Develop and optimize
ETL/ELT processes
for structured and unstructured data.
Work with
Lakehouse architecture
for data storage, processing, and analytics.
Orchestrate pipelines using
Databricks Workflows
and
Jobs API
.
Integrate AWS services such as
S3, Lambda, CloudWatch
.
Perform
performance optimization
using pushdown techniques and indexing strategies.
Implement
data governance
with Unity Catalog, security policies, and access controls.
Monitor, troubleshoot, and optimize
Databricks jobs and clusters
.
Contribute to
end-to-end cloud migration projects
to AWS.
Work with
Airflow
for orchestration and
Git/GitHub
for code repositories.
Write complex
SQL queries
and manage cloud-based data migration workflows.

Must Have Skills

Databricks, Apache Spark
(hands-on development experience).
AWS Data Services
(S3, Lambda, CloudWatch).
Strong
Python programming
for cloud development.
Airflow
for orchestration.
Strong
SQL
and query optimization.
Experience in
end-to-end migration projects
to AWS.
Excellent analytical, problem-solving, and communication skills.

Good to Have Skills

Exposure to
Teradata, DataStage, SSIS
.
Knowledge of
Databricks Delta Live Tables
and
Delta Lake
.
Experience with
Kafka / Spark Streaming
.
CI/CD tools like
Jenkins
.
Terraform
for IaC (Databricks deployment automation).

Certification :

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.