Job
Description
Key Responsibilities:Assess and understand existing data platforms e.g., Hadoop, Spark, onpremises systems, or other cloud data warehouses Design and implement end-to-end migration strategies to Databricks. Migrate data pipelines, notebooks, models, and jobs to Databricks. Optimize performance and cost-efficiency during and after the migration. Develop and maintain CI/CD pipelines for Databricks-based projects. Collaborate with data engineers, analysts, and business stakeholders to ensure smooth transition and minimal disruption. Document migration processes, architecture, and configurations. Required Skills & Qualifications: 5+ years of experience in data engineering or related roles. 2+ years of hands-on experience with Databricks. Experience migrating from systems like Hadoop, Spark, or legacy ETL tools to Databricks. Strong expertise in Apache Spark, PySpark, SQL, and Delta Lake. Familiarity with cloud platforms (Azure, AWS, or GCP) especially Azure Databricks or AWS Databricks. Experience with CI/CD, DevOps, and version control tools (Git, Azure DevOps, etc.). Strong problem-solving skills and ability to work independently. Excellent communication and documentation skills. Preferred Qualifications: Databricks certification (e.g., Databricks Certified Data Engineer Associate/Professional). Experience with MLflow, Unity Catalog, or data governance frameworks. Familiarity with Terraform or Infrastructure as Code (IaC) for Databricks setup. About Virtusa Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us. Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence. Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
