Senior Consultant - Databricks

Senior Data Engineer Databricks (Azure/AWS)

Role Overview:

We are looking for a hands-on

Senior Data Engineer

experienced in migrating and building large-scale data pipelines on

Databricks

using

Azure

or

AWS

platforms. The role will focus on implementing batch and streaming pipelines, applying the

bronze-silver-gold

data lakehouse model, and ensuring scalable and reliable data solutions.

Required Skills and Experience:

• 6+ years of hands-on data engineering experience, with 2+ years specifically working on Databricks in Azure or AWS.
• Proficiency in building and optimizing Spark pipelines (batch and streaming).
• Strong experience implementing bronze/silver/gold data models.
• Working knowledge of cloud storage systems (ADLS, S3) and compute services.
• Experience migrating data from RDBMS (Oracle, SQL Server) or Hadoop ecosystems.
• Familiarity with Airflow, Azure Data Factory, or AWS Glue for orchestration.
• Good scripting skills (Python, Scala, SQL) and version control (Git).
  

Preferred Qualifications:

• Databricks Certified Data Engineer Associate or Professional.
• Experience with Delta Live Tables (DLT) and Databricks SQL.
• Understanding of cloud security best practices (IAM roles, encryption, ACLs).
  

Responsibilities

Key Responsibilities:

• Design, develop, and operationalize scalable data pipelines on Databricks following medallion architecture principles.
• Migrate and transform large data volumes from traditional on-prem systems (Oracle, Hadoop, Exadata) into cloud data platforms.
• Develop efficient Spark (PySpark/Scala) jobs for ingestion, transformation, aggregation, and publishing of data.
• Implement data quality checks, error handling, retries, and data validation frameworks.
• Build automation scripts and CI/CD pipelines for Databricks workflows and deployment.
• Tune Spark jobs and optimize cost and performance in cloud environments.
• Collaborate with data architects, product owners, and analytics teams.
  

Attributes for Success:

• Strong analytical and problem-solving skills.
• Attention to scalability, resilience, and cost efficiency.
• Collaborative attitude and passion for clean, maintainable code.
  

Diversity and Inclusion:

An Oracle career can span industries, roles, Countries, and cultures, allowing you to flourish in new roles and innovate while blending work life in. Oracle has thrived through 40+ years of change by innovating and operating with integrity while delivering for the top companies in almost every industry. To nurture the talent that makes this happen, we are committed to an inclusive culture that celebrates and values diverse insights and perspectives, a workforce that inspires thought leadership and innovation. Oracle offers a highly competitive suite of Employee Benefits designed on the principles of parity, consistency, and affordability. The overall package includes certain core elements such as Medical, Life Insurance, access to Retirement Planning, and much more. We also encourage our employees to engage in the culture of giving back to the communities where we live and do business.At Oracle, we believe that innovation starts with diversity and inclusion and to create the future we need talent from various backgrounds, perspectives, and abilities. We ensure that individuals with disabilities are provided reasonable accommodation to successfully participate in the job application, and interview process, and in potential roles. To perform crucial job functions. Thats why were committed to creating a workforce where all individuals can do their best work. Its when everyones voice is heard and valued that were inspired to go beyond whats been done before.

About Us

As a world leader in cloud solutions, Oracle uses tomorrows technology to tackle todays challenges. Weve partnered with industry-leaders in almost every sectorand continue to thrive after 40+ years of change by operating with integrity.We know that true innovation starts when everyone is empowered to contribute. Thats why were committed to growing an inclusive workforce that promotes opportunities for all.Oracle careers open the door to global opportunities where work-life balance flourishes. We offer competitive benefits based on parity and consistency and support our people with flexible medical, life insurance, and retirement options. We also encourage employees to give back to their communities through our volunteer programs.Were committed to including people with disabilities at all stages of the employment process. If you require accessibility assistance or accommodation for a disability at any point, let us know by emailing [HIDDEN TEXT] or by calling +1 888 404 2494 in the United States.Oracle is an Equal Employment Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability and protected veterans status, or any other characteristic protected by law. Oracle will consider for employment qualified applicants with arrest and conviction records pursuant to applicable law.