Databricks Developer SDS/BSV-ES

Responsible Area

AWS Databricks Developer

Key Responsibilities:

Overall, 5 to 8 years of experience in IT Industry. Min 6 years of experience working on Data Engineering.

Design and develop solution in combination of Cloudera, Palantir and Databricks.

Develop Python-based automation scripts to optimize workflows and processes.

Integrate Databricks with AWS ecosystem components such as S3, Glue, Athena, Redshift, and Lambda

Define architecture for data-driven solutions.

Provide solutioning for web applications and data pipelines.

Solutioning data analytics requirements with Palantir.

Stay updated with emerging technologies and quickly adapt to new tools and frameworks.

Work with CI/CD pipelines.

Ensure scalability, security, and performance of deployed solutions.

Collaborate with business teams, data engineers, and developers to align solutions with business goals.

Key activities

Mandatory Skills:

Databricks

o Expertise in design jobs and workloads.

o Proficiency in pyspark / scala for data manipulation, transformation and analysis

o Optimization of Databricks clusters and notebooks.

Python

Cloudera

Palantir:

DevOps Basics

Integrate Databricks with AWS ecosystem components such as S3, Glue, Athena, Redshift, and Lambda

Communication

Fast Learner

Good to have skills :

Familiarity with containerization technologies (Docker, Kubernetes).

Experience with real-time streaming technologies (e. g. , Kafka Streams, Spark Streaming).

Exposure to machine learning operationalization (MLOps) workflows.

Background in regulated industries with strict data compliance requirements.

Additional Information:

Certifications: Databricks Certified Associate

Educational qualification:

Bachelor s degree in Computer Science, IT, or related field.

Experience :

Overall, 5 to 8 years of experience in IT Industry. Min 6 years of experience working on Data Engineering

Mandatory/requires Skills :

Preferred Skills :