Job Summary
As a Databricks Developer, you will be responsible for developing, optimizing, and maintaining data pipelines and analytic workflows on Databricks. You will collaborate with data engineers, analysts, data scientists, and business stakeholders to transform raw data into actionable insights. This role requires hands-on expertise with Databricks, Apache Spark, and cloud data platforms such as Azure, AWS, or GCP. You will play a key part in architecting scalable data solutions, ensuring data quality, and enabling advanced analytics and machine learning capabilities across the organization. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark to ingest, process, and transform large data sets from diverse sources. Implement ETL (Extract, Transform, Load) workflows to support analytics, reporting, and machine learning initiatives. Optimize data jobs for performance, scalability, and cost efficiency within the Databricks environment.
Collaborate with data engineers, data scientists, analysts, and business stakeholders to understand requirements and translate them into technical solutions.
Develop and maintain notebooks in Databricks using Python, Scala, SQL, or R.
Ensure data quality by implementing data validation, testing, and monitoring procedures. Integrate Databricks with cloud storage services (such as Azure Data Lake, AWS S3, or Google Cloud Storage) and other data sources. Automate data workflows and job schedules using Databricks Jobs, Workflows, or orchestration tools such as Apache Airflow or Azure Data Factory. Participate in code reviews, support best practices in software development, and contribute to process improvements. Monitor, troubleshoot, and resolve issues related to data pipelines and Databricks environment. Contribute to the design and implementation of data architectures, including data lakes, data warehouses, and streaming data solutions.
Document technical designs, data flows, and process workflows for knowledge sharing and compliance requirements.
Stay current with Databricks platform updates, industry trends, and best practices for big data, analytics, and cloud technologies.
