Lead I - Data Engineering - Databricks developer

5 - 9 years

0 Lacs

Posted:3 days ago| Platform: Shine logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

As a Databricks Developer with 5 to 9 years of experience, your role will involve designing, developing, and maintaining data pipelines using Databricks, Python/PySpark, and SQL. You will be responsible for implementing scalable ETL solutions, optimizing data workflows, ensuring data quality, and monitoring production data pipelines. Collaboration with data architects, analysts, and business stakeholders is essential to understand data needs and contribute to Agile environments. Key Responsibilities: - Design, develop, and maintain data pipelines using Databricks, Python/PySpark, and SQL. - Implement scalable ETL solutions to process structured and unstructured data. - Work closely with data architects, analysts, and business stakeholders to understand data needs. - Optimize data workflows for performance, scalability, and reliability. - Ensure data quality and integrity across systems. - Monitor production data pipelines and troubleshoot issues proactively. - Write clear, maintainable code and documentation. - Collaborate in Agile environments and contribute to sprint planning and reviews. Qualifications Required: - Strong hands-on experience with Databricks platform. - Expertise in Python and PySpark for data engineering tasks. - Advanced SQL skills for complex data querying and transformation. - Experience working with large-scale datasets in distributed environments. - Good understanding of data warehousing concepts and cloud data architecture. It is nice to have experience with Azure Data Lake, Delta Lake, or AWS S3, familiarity with CI/CD pipelines and version control (Git), and exposure to data governance, cataloging, and security best practices. As a Databricks Developer, you should hold a Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field. Your skills in Databricks, Python, PySpark, and SQL will be crucial for excelling in this role.,

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
UST logo
UST

IT Services and IT Consulting

Aliso Viejo CA

RecommendedJobs for You