Databricks Lead

Databricks Lead

Description :

Role Overview:

We are looking for a seasoned and visionary Data Architect with over 14 years of experience to lead the design and implementation of enterprise-grade data architecture, governance frameworks, and scalable data platforms. This role requires deep expertise in modern data technologies, big data ecosystems, and cloud-native solutions. The ideal candidate will be a strategic leader who can bridge business needs with technical solutions, ensuring data integrity, quality, and accessibility across the organization.

Responsibilities:

• Lead the design and implementation of Data Architecture, Data Quality frameworks, and Data Governance initiatives.

• Define data modeling standards and establish foundational best practices across data platforms.

• Champion and promote data quality principles by developing and institutionalizing enterprise-wide standards and methodologies.

• Build and oversee data governance policies, procedures, and frameworks to ensure data integrity, security, and compliance.

• Drive enterprise-wide adoption of self-service data platforms and enablement of organizational data literacy and utilization.

• Define and maintain critical data standards and metadata to facilitate the consistent understanding and use of data as a shared enterprise asset.

• Create reusable templates and establish coding standards for sourcing, collecting, and transforming both streaming and batch data.

• Design scalable data schemas, object models, and flow diagrams to effectively structure, process, and integrate data across systems.

• Provide architectural strategy, assessments, and roadmaps to guide enterprise data management practices.

• Offer hands-on expertise in architecting and managing Big Data platforms and Data Lake technologies (e.g., AWS S3, Hive), along with exposure to ML and Data Science tools.

• Implement and manage industry-standard tools and frameworks such as Databricks, Delta Lake, Apache Spark, Airflow, Hive Metastore, Redshift, Kafka, Kubernetes, Docker, and CI/CD pipelines.

• Translate complex big data and analytics requirements into high-performance, scalable data models and guide analytics engineers on implementing these models.

• Define reusable templates and processes for data modeling, data flow design, and system integration.

• Lead, mentor, and upskill data analytics teams by promoting best practices, modern technologies, and effective processes in data platform engineering.

• Tech Skills:

• 6+ years of hands-on experience in Data Warehouse, ETL, Data Modeling & Reporting

• 5+ years of hands-on experience in productionizing and deploying Big Data platforms and applications.

• Minimum of 3 years of hands-on DataBricks experience required.

• Hands-on experience working with: Relational/SQL, distributed columnar data stores/NoSQL databases, time-series databases, Spark streaming, Kafka, Hive, Delta Parquet, Avro, and more

• Extensive experience in understanding a variety of complex business use cases and modeling the data in the data warehouse

• Highly skilled in SQL, Python, Spark, AWS S3, Hive Data Catalog, Parquet, Redshift, Airflow, and Tableau or similar tools

• Proven experience in building a Custom Enterprise Data Warehouse or implementing tools like Data Catalogs, Spark, Tableau, Kubernetes, and Docker

• Demonstrated industry leadership in the fields of Data Warehousing, Data Science, and Big Data related technologies

• Design and Implement Machine learning models with Tensorflow ,Keras and pytorch And MLops

~