Databricks Technical Lead - Big Data

Contract Duration : 4 Months (Extendable based on Performance)
Job Timings : India Evening Shift (till 11 : 30 PM IST)
Job Description :
We are looking for an experienced Databricks Tech Lead to join our team on a 4-month extendable contract. The ideal candidate will bring deep expertise in data engineering, big data platforms, and cloud-based data warehouse solutions, with the ability to work in a fast-paced remote environment.Key Responsibilities :
- Lead the design, optimization, and management of large-scale data pipelines using Databricks, Spark (PySpark), and AWS data services.- Productionize and deploy Big Data platforms and applications across multi-cloud environments (AWS, Azure, GCP).

- Build and manage data warehouse solutions, schema evolution, and data versioning.

- Implement and manage workflow orchestration using Airflow or similar tools.

- Work with complex business use cases and transform them into scalable data models and architectures.

- Collaborate with cross-functional teams to deliver high-quality data engineering and analytics solutions.

- Mentor team members and provide technical leadership on Databricks and modern data technologies.
Required Skills & Experience :
- 7+ years of experience in Data Warehouse, ETL, Data Modeling & Reporting.

- 5+ years of experience in productionizing & deploying Big Data platforms.

- 3+ years of hands-on Databricks experience.

- Strong expertise with :

a. SQL, Python, Spark, Airflow

b. AWS S3, Redshift, Hive Data Catalog, Delta Lake, Parquet, Avro

- Streaming platforms (Spark Streaming, Kafka, Hive)

- Proven experience in building Enterprise Data Warehouses and implementing frameworks such as Databricks, Apache Spark, Delta Lake, Tableau, Hive Metastore, Kafka, Kubernetes, Docker, and CI/CD pipelines.

- Hands-on experience with Machine Learning frameworks (TensorFlow, Keras, PyTorch) and MLOps practices.

- Strong leadership, problem-solving, and communication skills.