Databricks Technical Lead

0 years

0 Lacs

Posted:1 week ago| Platform: Linkedin logo

Apply

Work Mode

Remote

Job Type

Contractual

Job Description

Senior DataBricks Data Engineer


Location: Remote

Duration: Long Term


We are seeking a highly skilled Data Engineer with extensive experience in Databricks and data streaming platforms to join our team. The ideal candidate will play a key role in managing streaming data platforms, ensuring seamless integration and performance across various data sources and sinks. You will be instrumental in designing and implementing cutting-edge data architectures and pipelines, focusing on leveraging Databricks for efficient data processing and analytics. Your expertise in Databricks and Apache Kafka ecosystems will drive our data strategy forward, enhancing our ability to deliver actionable insights and maintain robust data operations.


Key Responsibilities:

Databricks Platform:

  • Implement and optimize data pipelines within the Databricks Platform, applying medallion architecture principles for data organization.
  • Create and maintain data pipelines through batch and stream processing, with Stream Tables, Delta Live Tables, Change Data Capture 
  • Implement and manage DABs for efficient packaging, deployment, and version control of Databricks resources. Oversee job orchestration and scheduling.
  • Monitor data processes to ensure performance and reliability.


Data Stream Platform:

  • Design, develop and manage streaming data platforms compatible with Apache Kafka, utilizing Confluent or RedPanda.
  • Implement and maintain a Schema Registry to enforce data contracts and compatibility.
  • Implement and manage Kafka Connectors for integration with various data sources and sinks.


Programming Languages: 

  • Spark
  • SQL
  • Python for data processing and analytics.


Databases:

  • MySQL
  • DynamoDB
  • PostgreSQ



Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
Celebal Technologies logo
Celebal Technologies

Technology Consulting and Services

Ahmedabad

RecommendedJobs for You