Data Engineer ( Azure Databricks, PySpark / Python )

IDESLABS PRIVATE LIMITED

6 - 11 years

5 - 9 Lacs

hyderabad pune

Posted:3 days ago| Platform:

Apply

Skills Required

sql data modeling python data warehousing concepts etl process hive scala pyspark data warehousing spark gcp hadoop etl big data msbi azure databricks data services ssas microsoft azure data engineering sql server data bricks sqoop aws ssis

Work Mode

Work from Office

Job Type

Full Time

Job Description

We are looking for a skilled Data Engineer with 6-12 years of experience to design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Live Tables. The ideal candidate will have a strong understanding of data warehousing concepts, ETL processes, and data modeling.

Roles and Responsibility

Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Live Tables.
Collaborate with cross-functional teams to understand data requirements and translate them into efficient data models and pipelines.
Implement best practices for data engineering, including data quality and security.
Optimize and troubleshoot complex data workflows to ensure high performance and reliability.
Develop and maintain documentation for data engineering processes and solutions.
Ensure data engineering processes align with industry standards and best practices.

Job Requirements

Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
Proven experience as a Data Engineer with a focus on Databricks, PySpark, Spark SQL, and Delta Live Tables.
Strong understanding of data warehousing concepts, ETL processes, and data modeling.
Proficiency in programming languages such as Python and SQL.
Experience with cloud platforms (e.g., AWS, Azure, GCP) and their data services.
Excellent problem-solving skills and the ability to work in a fast-paced environment.
Strong leadership and communication skills, with the ability to mentor and guide team members.

More Jobs at IDESLABS PRIVATE LIMITED

Bank Fast Platform Engineer

Bengaluru

3.0 - 7.0 yrs

INR 5 - 9 Lacs

Salesforce Commerce Cloud Professional

Hyderabad

3.0 - 5.0 yrs

INR 4 - 6 Lacs

Python Scripting with Terraform Professional

Hyderabad

3.0 - 5.0 yrs

INR 4 - 6 Lacs

Nodejs / AWS Developer

Hyderabad

3.0 - 5.0 yrs

INR 4 - 6 Lacs

Risk Business Analyst

Bengaluru

5.0 - 10.0 yrs

INR 8 - 13 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.