Databricks Developer

Dear Candidate,

We at TATA Technologies looking for an experienced candidate for Databricks. Please check the below JD for more details

Job Title : Databricks Developer

Total Exp:3-6 Years

Location: Pune

Notice Period: Immediate-30 Days

Role & responsibilities

• Design and implement Databricks-based data architectures to meet business requirements.
• Develop and optimize data pipelines using PySpark, Scala, or SQL.
• Establish the Databricks Lakehouse architecture for batch and streaming data.
• Collaborate with cross-functional teams to integrate Databricks with cloud platforms (e.g., AWS, Azure, GCP).
• Ensure data security and compliance with best practices.
• Monitor and troubleshoot Databricks environments for performance and reliability.
• Stay updated on Databricks advancements and industry trends.

Desired Candidate Profile:

• Experience in data engineering using Databricks or Apache Spark-based platforms.
• Proven track record of building and optimizing ETL/ELT pipelines for batch and streaming data ingestion.
• Good understanding of AWS cloud services
• Proficiency in programming languages such as Python, Scala, or SQL for data processing and transformation.
• Expertise in Spark (PySpark,Spark SQL, or Scala) and Databricks notebooks for large-scale data processing.
• Expertise in Spark (PySpark, Spark SQL, or Scala) and Databricks notebooks for large-scale data processing. ions.
• Build and query delta lake storage solutions
• Process streaming data with Azure Databricks structured streaming
• Design Azure Databricks security and data protection solutions
• Flatten nested structures and explode arrays with spark
• Experience with orchestration tools like Databricks Jobs for scheduling and automation.
• Knowledge of Git for source control and CI/CD integration for Databricks workflows, cost optimization, performance tuning.
• Familiarity with Unity Catalog, RBAC, or enterprise-level Databricks setups.
• Ability to create reusable components, templates, and documentation to standardize data engineering workflows.
• Solutioning and presales - Architecting frameworks, defining roadmaps, and engaging with stakeholders.
• Experience in defining data strategy, evaluating new tools/technologies, and driving adoption across the organization.
• Must have experience of working with streaming data sources and Kafka.

Preferred Experience:

• Bachelors degree in computer science, Information Technology, or related field
• Proven experience as a Databricks Architect or similar role
• Complete knowledge in Azure Databricks platform architecture
• Databricks certification (e.g. Certified Data Engineer, Associate Developer)
• Expertise in Python/Scala/ SQL/R
• Experience with cloud platforms like AWS, Azure, or GCP

Strong understanding of data modelling and cloud integration

Note