Posted:2 weeks ago|
Platform:
On-site
Full Time
1. 8+ years with strong Databricks, Spark and Data Migration experience
2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.
3. Key Responsibilities:
a) Migration Strategy and Execution: analyze the existing DWH/DB architecture, Design & Implement scalable data pipelines in Databricks, SQL Script conversion, etc.
b) Development & Optimization: Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets
c) Collaboration & Governance: Work with Data Architect to define Target data models and migration approaches, Collaborate with DevOps teams to implement CI/CD pipelines, Data Governance teams on Security, lineage and cataloging; Mentor junior members in the team
d) Nice to have experience with Unity Catalog for governance and access control; Exposure to machine learning/AI pipelines, etc.
Tata Consultancy Services
Upload Resume
Drag or click to upload
Your data is secure with us, protected by advanced encryption.
Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.
We have sent an OTP to your contact. Please enter it below to verify.
Instantly access job listings, apply easily, and track applications.
kolkata, west bengal, india
Salary: Not disclosed
karnataka
Salary: Not disclosed
kolkata, west bengal, india
Salary: Not disclosed
karnataka
Salary: Not disclosed
