Databricks Engineer

Project cod e

STITTDBP

ECMS ID

531925

Skill

Databricks, Hadoop, Python, Spark, AWS, Airflow

Band in Infosys

JL5

Role

Technology Lead

Qualification

BE

Job Description

Databricks Engineer: 9+ years of total experience with 5 years relevant experience in the mandatory skills.

• Mandatory Skills: Databricks, Hadoop, Python, Spark, Spark SQL, PySpark, AirFlow And IBM StreamSet

Required Skills & Experience:

• Develop Data Engineering and ML pipelines in Databricks and different AWS services, including S3, EC2, API, RDS, Kinesis/Kafka and Lambda to build serverless applications


• Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).


• Knowledge on medallion architecture, DLT and unity catalog within Databricks.


• Experience in migrating data from on-prem Hadoop to Databricks/AWS


• Understanding of core AWS services, uses, and AWS architecture best practices


• Hands-on experience in different domains, like database architecture, business intelligence, machine learning, advanced analytics, big data, etc.


• Solid knowledge on Airflow


• Solid knowledge on CI/CD pipeline in AWS technologies


• Application migration of RDBMS, java/python applications, model code, elastic etc.


• Solid programming background on scala, python, SQL

Vendor Rate

9000 INR/Day

Work Location

Bangalore

If its before-onboarding, please confirm whether final BG report is required or interim report is fine to onboard candidate

BGC Before onboarding. There is a separate BGC from client side.

Client BGC and onboarding must need to be complete to start in the project.

BGC Vendor

Any

Drug Text required(Yes or No)

Novibhu