Data Engineer

Techno Facts Solutions

7 - 10 years

9 - 15 Lacs

hyderabad

Posted:-1 days ago| Platform:

Apply

Skills Required

data engineering data bricks pyspark data quality ci/cd etl sql python

Work Mode

Work from Office

Job Type

Full Time

Job Description

Lead Data Engineer

Key Responsibilities

Design, build, and optimize large-scale
data pipelines
using
Databricks
and
PySpark
.
Develop efficient
ETL/ELT workflows
and automate data processes.
Work extensively with
Python
for data engineering tasks and modular coding.
Implement and optimize
Spark jobs
for scalability, cost efficiency, and performance.
Integrate data from various structured and unstructured sources.
Ensure high data quality through validation, testing, and monitoring.
Collaborate with data science and analytics teams for data availability.
Lead code reviews, provide technical guidance, and mentor junior engineers.
Work with
Azure / AWS / GCP
services integrated with Databricks.
Support CI/CD pipeline integration for data workflows.
Drive improvements in data governance, security, and best practices.

Required Skills

7.510 years of experience in
data engineering
roles.
Strong hands-on experience with
Databricks
(Azure, AWS, or GCP).
Expert knowledge of
PySpark
for distributed data processing.
Strong programming skills in
Python
.
Excellent understanding of
Spark SQL, Delta Lake
, and big data concepts.
Solid experience with
cloud platforms
(Azure preferred).
Strong SQL skills and experience with performance tuning.
Working knowledge of CI/CD tools, Git, and orchestration tools (ADF, Airflow, etc.).
Strong problem-solving skills and ability to lead technical discussions.

Good-to-Have Skills

Experience with MLflow, Lakehouse architecture, or data governance tools.
Exposure to Kubernetes, Docker, or container-based deployments.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.