Databricks Data Engineer

Tata Consultancy Services

3 - 6 years

0 Lacs

chennai tamil nadu india

Posted:1 week ago| Platform:

Apply

Skills Required

databricks data reliability aws python design migrate pyspark talend architecture redshift development documentation training testing writing optimization

Work Mode

On-site

Job Type

Full Time

Job Description

About the Role

The Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.

Experience- 3 to 6 years

Location - Kolkata /Chennai/Trivandrum

Required Technical Skill Set ------ Databricks, AWS, Python & PySpark

Good to have- Experience in AWS platform (S3, EC2, EMR, MWAA, and Redshift).

Responsibilities

Collaborate with stakeholders to gather requirements and design solutions using the Databricks platform.
Migrate Python, Pyspark, MWAA DAG to Databricks Notebook and Workflow.
Work with BJs Computer operations team and repoint Talend/MWAA Autosys jobs to Databricks workflow.
Implement medallion architecture in Databricks Enterprise Data Platform with industry best practices and recommend the best framework for Redshift to Delta tables.
Provide technical expertise and guidance to the team members and stakeholders.
Contribute to the development of documentation, training materials, and user guides.
Participate in testing and quality assurance activities to ensure reliability and performance.
Optimize long-running Databricks workflows.
Development experience in Python, Pyspark.
Hands-on experience in writing complex SQLs and performance optimization of very complex SQL.

More Jobs at Tata Consultancy Services

Tcs Hiring For GRC , Pan India

Pune, Bengaluru, Mumbai (All Areas)

7 - 12 yrs

INR 9 - 19 Lacs

Java / Spring boot with API Developer

Pune

5 - 10 yrs

INR 9 - 19 Lacs

SAP PP and QM Consultant

Pune

4 - 9 yrs

INR 8 - 18 Lacs

SAP BTP Consultant

Pune

4 - 9 yrs

INR 9 - 19 Lacs

Customer Service Associate

Pune

1 - 6 yrs

INR 1 - 5 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.