205 Unity Catalog Jobs

About the Role The Databricks Data Architect will play a pivotal role in designing and scaling Lillyu2019s Data & AI platform on cloud environments such as AWS and Azure . This role focuses on leveraging Databricks Lakehouse to deliver secure, scalable, and high-performing data architectures that empower AI/ML, analytics, and data product development across global business functions. The ideal candidate will blend deep technical expertise , data platform design experience , and strategic vision to enable Lillyu2019s transformation into an AI-driven enterprise . Key Responsibilities 1. Platform & Architecture Leadership Design, implement, and optimize Lakehouse architectures using Databricks,...

Role : Data Engineer Experience : 10 to 20 Yrs Key Responsibilities: Serve as the point of contact and subject matter expert for all Databricks-related activities, including architecture, development, and operational best practices. Should work closely with Sales team, propose data roadmap to prospects intending to migrate to cloud, create proof of concepts to showcase our expertise. Design, develop, and manage ETL/ELT pipelines in Databricks using Python (PySpark), integrating various data sources to support business operations. Leverage Unity Catalog to ensure data lineage, security, and governance are properly managed across the Databricks environment. Implement and maintain CI/CD pipelin...

Strong hands-on experience with Databricks (Workflows, DLT Pipelines, Delta Lake, Unity Catalog, Jobs, Clusters) Unity Catalog, IAM Databricks Platform Unity Catalog background,

Role Overview: As a Senior Data Engineer, you will be responsible for designing, building, and optimizing scalable data pipelines for large-scale data processing. Your expertise in Python, Spark, Databricks, and Azure will be crucial in handling complex data engineering tasks. You will work with cloud platforms, specifically Azure, to process and analyze large datasets efficiently. Key Responsibilities: - Design, build, and optimize scalable data pipelines using Python and Apache Spark. - Utilize Azure Databricks and PySpark for big data processing and analysis. - Implement data integration and orchestration using Azure Data Factory. - Utilize Azure Data Lake Storage for big data analytics a...

Role Overview: You should have 5+ years of experience and must possess skills in DataBricks, Delta Lake, pyspark or scala spark, and unity catalog. It would be beneficial to have Azure/AWS Cloud skills. Your main responsibility will be to ingest and transform batch and streaming data on the Databricks Lakehouse Platform. Excellent communication skills are required for this role. Key Responsibilities: - Oversee and support the process by reviewing daily transactions on performance parameters - Review performance dashboard and scores for the team - Support the team in improving performance parameters by providing technical support and process guidance - Record, track, and document all queries ...

Job Description We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS . The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data. Responsibilities Key Responsibilities Define and design cloud-native data architectures on Databricks using Delta Lake, Unity Catalog, and related components. Develop and execute migration strategies for moving on-premises d...

Skills: Data Engineering, Azure Databricks, Data Modeling, architecture, pyspark, scala, SQL, About The Role We are seeking a Senior Databricks Architect to lead the design and implementation of scalable, secure, and cost-optimized data platforms on Azure using Databricks. You will own end-to-end architecture for data ingestion, transformation, governance, and analytics, and mentor engineering teams to deliver high-quality data solutions. Key Responsibilities Architect and implement enterprise-grade data platforms on Azure Databricks (DLT, Workflows, Unity Catalog) with strong security, governance, and reliability. Design end-to-end data pipelines: batch and streaming (Structured Streaming),...

Job Title: GCP Data Architect (Hybrid) Location: Noida, India Experience: 10+ Years About The Role We are looking for a highly experienced Data Architect to lead the design, architecture, and optimization of large-scale, enterprise-level data platforms. The ideal candidate will have a strong background in Databricks , PySpark , Google Cloud Platform (GCP) , Unity Catalog , ETL processes , and Terraform . You will play a key role in delivering scalable, secure, and high-performance data engineering solutions across the organization. Key Responsibilities Architect and design modern, scalable, and secure data engineering platforms on Google Cloud Platform (GCP), ensuring high availability and p...

As an experienced AI Platform Engineer, your role will involve designing, developing, and maintaining a hybrid AI Platform across multi-cloud and on-premises environments. You will be responsible for building an AI platform that enables real-time machine learning and GenAI at scale, while also incorporating governance and security frameworks. Key Responsibilities: - Design and deliver a hybrid AI Platform. - Develop and maintain scalable, efficient, and reliable data and AI pipelines. - Collaborate with data scientists, data engineers, and central data governance team to align with a standardized data strategy. - Design and implement MLOps and LLMOps framework. - Implement automated pipeline...

As a skilled and experienced Technology Stack Administrator, your role will involve managing and optimizing the on-premises infrastructure comprising RedHat OpenShift, Apache Spark, Apache Airflow, Delta Lake, Apache Kafka, and Debezium. Your responsibilities will include: - Platform Administration: - Installing, configuring, and maintaining RedHat OpenShift clusters in an on-premises environment. - Building and orchestrating data pipelines using Apache Airflow. - Implementing and managing Delta Lake for ACID-compliant data lake architecture. - Managing Apache Spark clusters for distributed data processing. - Developing real-time data streaming and CDC solutions using Apache Kafka and Debezi...

Role Overview: As an experienced Data Architect specializing in Databricks, you will be responsible for leading the architecture, design, and migration of enterprise data workloads from on-premises systems to Databricks on Azure or AWS. Your role will involve designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers) to support large-scale ingestion, transformation, and publishing of data. Key Responsibilities: - Define and design cloud-native data architecture on Databricks utilizing Delta Lake, Unity Catalog, and related services. - Develop migration strategies for transferring on-premises data workloads (Oracle, Hadoop,...

Job Title: Data Architect Job Summary: We are seeking a highly skilled Data Architect to design and implement scalable, secure, and high-performance data solutions on Microsoft Azure. This role requires deep technical expertise in Azure data services, data modeling, and modern data architectures (lakehouse, warehouse, etc.), with a strong focus on supporting analytics and business intelligence initiatives. Key Responsibilities: • Design and implement end-to-end data solutions on Microsoft Azure, including data lakes, data warehouses, and ETL/ELT pipelines. • Develop scalable and efficient data architectures to support large-scale data processing and analytics workloads. • Define and enforce ...

Job Title Data Engineer - Hadoop, Scala and Databricks Role Summary & Role Description As Data Engineer you will Collaborate across a variety of teams to enable our Data Platform needs and design, implement, and maintain a secure and scalable infrastructure platform spanning across AWS/Azure and our Data Center Own and ensure that internal and external SLA's meet and exceed expectations, System centric KPIs are continuously monitored Work on our data lake, data warehouse, and stream processing systems to create a unified query engine, multi-model databases, analytics extracts and reports, as well as dashboard and visualizations Design and build systems for high availability, high throughput,...

Job Title Data Engineer - Hadoop, Scala and Databricks Role Summary & Role Description As Data Engineer you will Collaborate across a variety of teams to enable our Data Platform needs and design, implement, and maintain a secure and scalable infrastructure platform spanning across AWS/Azure and our Data Center Own and ensure that internal and external SLA's meet and exceed expectations, System centric KPIs are continuously monitored Work on our data lake, data warehouse, and stream processing systems to create a unified query engine, multi-model databases, analytics extracts and reports, as well as dashboard and visualizations Design and build systems for high availability, high throughput,...

Description About InspireXT : InspireXT is on a mission to build the most trusted supply chains in the world. Our Connected Intelligence vision combines Databricks with deep Oracle ERP/SCM, Salesforce, and industry accelerators to deliver compliant, AI-driven outcomes in Pharma, Manufacturing and Retail. We are investing significantly in our Databricks alliance and building a certified practice from Pune to support global clients across UK, Europe, US and Asia. Role Description As a Databricks Solution Architect at InspireXT, you will partner with our customers to design scalable, governed data architectures and lead migrations into the Databricks Lakehouse. You will bring technical depth an...

Databricks -(SME) with 8+ Yrs exp, Skills: Databricks, Unity Catalog, new Databricks offerings (i.e., Genie), Azure, Data Management, Data Engineering, Responsibilities: Design and build scalable data platforms using Azure and Databricks.

Looking for 10+ Y / highly experienced and deeply hands-on Data Architect to lead the design, build, and optimization of our data platforms on AWS and Databricks. This role requires a strong blend of architectural vision and direct implementation expertise, ensuring scalable, secure, and performant data solutions from concept to production. Strong hand on exp in data engineering/architecture, hands-on architectural and implementation experience on AWS and Databricks, Schema modeling . AWS: Deep hands-on expertise with key AWS data services and infrastructure. Databricks: Expert-level hands-on development with Databricks (Spark SQL, PySpark), Delta Lake, and Unity Catalog. Coding: Exceptional...

Looking for 10+ Y / highly experienced and deeply hands-on Data Architect to lead the design, build, and optimization of our data platforms on AWS and Databricks. This role requires a strong blend of architectural vision and direct implementation expertise, ensuring scalable, secure, and performant data solutions from concept to production. Strong hand on exp in data engineering/architecture, hands-on architectural and implementation experience on AWS and Databricks, Schema modeling . AWS: Deep hands-on expertise with key AWS data services and infrastructure. Databricks: Expert-level hands-on development with Databricks (Spark SQL, PySpark), Delta Lake, and Unity Catalog. Coding: Exceptional...

As a Senior Data Engineer at Persistent Ltd., you will be responsible for designing and developing scalable data pipelines using Azure Databricks, PySpark, and Delta Lake. Your key responsibilities will include: - Implementing and optimizing data warehousing solutions using Snowflake. - Integrating AI capabilities into data workflows using tools like ChatGPT, Copilot, Databricks AI Functions, or Snowflake Copilot. - Collaborating with data scientists and analysts to enable AI-powered insights, including summarization, classification, and sentiment analysis. - Building and maintaining ETL/ELT workflows to ensure data quality, performance, and governance. - Working with stakeholders to underst...

Job Description: As a Databricks Consultant with 4-6 years of professional experience in data engineering, your role will primarily focus on Databricks. You will be responsible for programming expertise in python, SQL, and Spark. Your key responsibilities will include: - Handling data pipelines, medallion architecture, Lakehouse, delta tables, time travel, unity catalog, and processing large volumes in Databricks. - Demonstrating good knowledge of data modelling, data warehousing, and Slowly Changing Dimension (SCD) approaches. - Utilizing your communication skills effectively to collaborate with team members and stakeholders. Qualification Required: - 4-6 years of professional experience in...

Role Overview: You will serve as the Senior Analytics Engineer at Lennox, taking on the responsibility of designing, developing, and supporting the Azure Databricks platform and Power BI Dashboards & reports. Your key focus will be on independently gathering requirements, designing and building models and reports. You will also be required to interact with various levels of business audiences to clearly define the necessary data to support strategic and tactical decision making. In collaboration with the Data Engineering team, Business Users, and other stakeholders, you will design and deploy end-to-end Analytics solutions to meet business needs. Your involvement in the project will span acr...

You have the opportunity to join as a DevOps/SRE with a Minimum of 7 years of experience in the Software Industry. You will be responsible for delivering independently and should have hands-on experience with GitHub Actions. Your duties will include automating builds and deployments, as well as having advanced knowledge of SAST tools such as Coverity, Blackduck, and SONAR. Additionally, you should have a working knowledge of Docker for containerization and be proficient in orchestration platforms like Kubernetes. Your expertise in the Azure cloud platform, along with cloud-native concepts and tools such as Terraform, CloudFormation, Apache Kafka/Event hub, Service Bus, Application Gateway, a...

Experience: Minimum 8 - 12 Years Shift: 5PM - 2 AM (US Shift) Mode of work: REMOTE - Work from Home Location: Hyderabad Only Notice Period: Immediate Joiners Only Background Verification of the candidates will be done. Strictly NO moonlighting. KEY ACCOUNTABILITY We are seeking a hands-on MDM Architect preferably with strong healthcare or a regulated industry data expertise to design and build an in-house Master Data Management platform for a team that currently has a tech stack of Azure and Databricks. This individual will define the enterprise MDM strategy, build scalable architecture, and ensure compliance with healthcare data standards while enabling a single, trusted view of critical da...

As a Data Architect specializing in Databricks and Snowflake, your role involves designing and optimizing cloud-native data architectures to support scalable data engineering, advanced analytics, and AI/ML solutions in alignment with business requirements. Your key responsibilities include: - Designing and implementing Lakehouse architectures utilizing Databricks, Delta Lake, and Apache Spark. - Leading the development of data pipelines, ETL/ELT processes, and data integration strategies. - Collaborating with both business and technical teams to establish data architecture standards, governance, and security models. - Optimizing the performance and cost-efficiency of Databricks clusters and ...

Experience needed: 8-12 years Type: Full-Time Mode: WFO Shift: IST Location: Hyderabad, India Notice Period: Immediate to 15 days About the Role We are seeking an experienced Databricks Implementation Lead to spearhead end-to-end delivery of a large-scale data engineering and analytics project built on the Databricks Lakehouse platform . The ideal candidate will have deep expertise in data architecture , hands-on experience with Databricks (SQL, PySpark, Delta Lake, MLflow) , and the ability to lead teams through solution design, migration, optimization, and production rollout . Key Responsibilities 1. Project Leadership Own the complete Databricks implementation lifecycle from requirement g...