Data Engineer - Data Lake

Wissen Infotech

7 - 10 years

10 - 14 Lacs

bengaluru

Posted:5 days ago| Platform:

Apply

Skills Required

pyspark delta lake transaction logs data engineer delta lake data bricks delta live tables directed acyclic graphs

Work Mode

Hybrid

Job Type

Full Time

Job Description

Data Engineer Data Lakes

Exp: 7+ Years

Location :Bangalore

Notice:Immediate

Key Experience & Expertise

Extensive hands-on experience in Partitioning Delta Lake Tables to achieve optimized query performance and storage efficiency.
Proficient in reading, interpreting, and reporting Delta Lake Transaction Logs to ensure data reliability and consistency.
Strong expertise in transforming streaming data and building Delta Live Tables (DLT) pipelines for real-time analytics.
Proven ability to design and implement Slowly Changing Dimensions (SCDs) using PySpark for scalable data warehousing solutions.
Skilled in configuring and leveraging Auto Optimize to enhance Delta Lake performance.
Practical knowledge of Directed Acyclic Graphs (DAGs) for workflow orchestration (e.g., Airflow, Databricks Workflows).
Proficient in developing and managing Materialized Views for accelerating queries and optimizing performance.
In-depth understanding of Medallion Architecture (Bronze, Silver, Gold layers) to structure data pipelines in a Lakehouse environment.
Strong focus on performance tuning and PySpark best practices, including:

Efficient implementation of Window Functions
Optimized Merge operations for handling large-scale datasets

More Jobs at Wissen Infotech

Business Analyst - Corporate Actions

Bengaluru, Mumbai (All Areas)

4 - 9 yrs

INR 18 - 20 Lacs

Informatica Developer

Bengaluru

6 - 11 yrs

INR 10 - 14 Lacs

AWS Cloud Developer with NodeJS

Bengaluru

4 - 9 yrs

INR 11 - 15 Lacs

It Procurement Executive

Bengaluru

4 - 8 yrs

INR 4 - 9 Lacs

Informatica Developer

Bengaluru

6 - 11 yrs

INR 10 - 16 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.