42 Databricks Workflows Jobs

Senior Databricks Administrator HyderabadEngineering R&D Engineering - Data Services /Full-Time /On-site Role Summary The Databricks Administrator will be responsible for the overall health, security, and performance of the Databricks platform. This includes managing user access, implementing and enforcing data governance policies, optimizing cluster resources, and ensuring data sensitivity policies are effectively applied across the data lakehouse. The administrator will also be crucial in identifying, reporting, and resolving discrepancies within the platform's operation and configuration. Key Responsibilities User Provisioning and Management: Onboard and offboard users, groups, and servic...

In your role as a Machine Learning Engineer, you will be responsible for developing and deploying end-to-end Machine Learning models using Python, Classical ML algorithms, and scalable Databricks workflows such as PySpark, MLflow, and Delta Lake. Your key responsibilities will include: - Leading GenAI initiatives, which involve tasks such as LLM fine-tuning, RAG pipelines, prompt engineering, and vector database integrations. - Building robust data pipelines, performing feature engineering, and optimizing models for performance and production readiness. - Collaborating with cross-functional teams to translate business problems into ML/GenAI solutions with measurable outcomes. - Conducting ad...

As an ML Platform Specialist, your role involves designing, implementing, and maintaining robust machine learning infrastructure and workflows using Databricks Lakehouse Platform. Your responsibilities include: - Designing and implementing scalable ML infrastructure on Databricks Lakehouse Platform - Developing and maintaining CI/CD pipelines for machine learning models using Databricks workflows - Creating automated testing and validation processes with Databricks MLflow - Implementing and managing model monitoring systems using Databricks Model Registry and monitoring tools - Collaborating with data scientists, software engineers, and product teams to optimize machine learning workflows - ...

Job Description We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS . The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data. Responsibilities Key Responsibilities Define and design cloud-native data architectures on Databricks using Delta Lake, Unity Catalog, and related components. Develop and execute migration strategies for moving on-premises d...

Job Title: Python Full Stack Developer (Generative AI Specialist) Experience Required: (4-7 Years) Location: [Bangalore] Employment Type: [Full-time] Skills Required: Proficiency in Databricks Workflows, Delta Live Tables (DLT), PySpark, and SQL for designing scalable ETL pipelines within the Medallion Architecture (BronzeSilverGold). Strong knowledge of Unity Catalog and Delta Lake. Hands-on experience with Databricks Genie, Vector Search, and MLflow for developing, fine-tuning, and deploying AI/LLM-based solutions. Expertise in Databricks SQL Endpoints, LangChain, and RAG (Retrieval-Augmented Generation) frameworks for conversational analytics and chatbot development. Familiarity with DevO...

Architect and implement scalable Azure Databricks solutions, including Data Lakehouse, Delta Lake, Structured Streaming, Unity Catalog, and Medallion Architecture. Design and manage advanced ETL/ELT pipelines utilizing PySpark, Databricks Workflows, Data Factory, and Azure Storage services. Lead ML engineering and MLOps implementation, leveraging MLflow for experiment tracking, model registry, versioning, deployment, and monitoring. Establish and enforce enterprise-grade data governance, security, lineage, and compliance using Unity Catalog, Azure AD, and cloud-native controls. Optimize Databricks clusters for performance, scalability, reliability, and cost-effectiveness in production enviro...

Job Description We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS . The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data. Responsibilities Key Responsibilities Define and design cloud-native data architectures on Databricks using Delta Lake, Unity Catalog, and related components. Develop and execute migration strategies for moving on-premises d...

As a Senior Developer with 8 to 12 years of experience, you will be responsible for developing and optimizing data solutions to drive business insights and efficiency. Your expertise in Databricks SQL, Databricks Delta Lake, Databricks Workflows, and PySpark will be crucial for this role. **Key Responsibilities:** - Develop and maintain scalable data pipelines using Databricks SQL and PySpark to support business analytics needs. - Optimize data workflows and processes within Databricks Delta Lake to ensure data integrity and performance. - Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions. - Implement best practices for data m...

As a Developer, you will play a key role in enhancing data solutions and optimizing workflows to positively impact both the company and society. You will leverage your expertise in Azure Data Lake Store, Azure DevOps, Python, Databricks SQL, Databricks Workflows, and PySpark to contribute to innovative projects that drive the company's success. **Key Responsibilities:** - Develop and implement data solutions using Azure Data Lake Store to enhance data storage and retrieval processes. - Collaborate with cross-functional teams to integrate Azure DevOps for streamlined development and deployment workflows. - Utilize Python to design and execute efficient data processing scripts ensuring high-qu...

As a Data Engineer at our company, you will be responsible for developing and maintaining scalable data pipelines using Databricks SQL and Databricks Workflows. Your role will involve collaborating with cross-functional teams to design and implement data solutions that align with business requirements. You will optimize and troubleshoot PySpark applications to ensure high performance and reliability. Your responsibilities will include providing technical guidance and mentorship to junior developers, conducting code reviews, and implementing data security measures to protect sensitive information. Working closely with stakeholders, you will gather and analyze requirements for new data project...

As a Senior Infrastructure Architect at our company, you will play a crucial role in our digital transformation journey. You will be part of a dynamic team that focuses on cybersecurity, architecture, and data protection for our global organization. Here is what you can expect in this role: - Participate in the domain technical and business discussions to shape the future architect direction. - Assist in analyzing, designing, and developing a roadmap based on current vs. future state in a cohesive architecture viewpoint. - Gather and analyze data to develop architectural requirements at the project level. - Support the infrastructure architecture governance model. - Design and deploy infrast...

As a Data Engineer, you will be responsible for the following key responsibilities: - Proficient in Python, including popular Python packages such as Pandas and NumPy, as well as SQL. - Strong background in distributed data processing and storage, such as Apache, Spark, and Hadoop. - Ability to work with large-scale data (TBs of data) and skills in data engineering to model data and create production-ready ETL pipelines. - Development experience with at least one cloud platform, with a high preference for Azure, and familiarity with AWS and GCP. - Knowledge of data lake and data lake house patterns, ETL performance tuning, and cost optimization. - Understanding of data structures, algorithms...

WPP is the creative transformation company. We use the power of creativity to build better futures for our people, planet, clients, and communities. Working at WPP means being part of a global network of more than 100,000 talented people dedicated to doing extraordinary work for our clients. We operate in over 100 countries, with corporate headquarters in New York, London and Singapore. WPP is a world leader in marketing services, with deep AI, data and technology capabilities, global presence and unrivalled creative talent. Our clients include many of the biggest companies and advertisers in the world, including approximately 300 of the Fortune Global 500. Our people are the key to our succ...

Job Title: Databricks SQL Engineer with Commercial Pharma Location: Remote Type: Contract About the Role: We are looking for a Databricks SQL Engineer to design, develop, and optimize data solutions using Databricks SQL. The ideal candidate will have expertise in SQL, performance tuning, data modeling, and cloud-based data platforms such as AWS, Azure, or GCP. Key Responsibilities: Develop and optimize SQL queries and notebooks in Databricks for data transformation, aggregation, and analysis. Optimize Databricks SQL workloads for performance, cost, and scalability. Monitor, debug, and troubleshoot Databricks SQL queries and jobs. Required Qualifications: 4+ years of experience with SQL devel...

Role: Data Engineer Experience: 3-5 yrs Notice Period: Immediate Work Mode: Hybrid Job Description Requirements: Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop) Large scale (TBs of data) data engineering skills - Model data, create production ready ETL pipelines Development experience with at least one cloud (Azure high preference, AWS, GCP) Knowledge of data lake and data lake house patterns Knowledge of ETL performance tuning and cost optimization Knowledge of data structures and algorithms and good software engineering practices Nice to Have: Experience with A...

Responsibilities ETL Development: Build and maintain ETL/ELT pipelines using Databricks & Apache Spark (PySpark/Scala). Data Warehousing: Design and implement Star Schema, Snowflake Schema, and Data Vault models. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Data Ingestion & Processing: Develop scalable ingestion pipelines using Structured Streaming, Auto Loader, and Databricks Workflows. Data Governance & Security: Implement Unity Catalog, role-based access controls (RBAC), and compliance standards. Performance Tuning...

As an Azure DB Architect with 8+ years of experience, your role will involve: - Designing Databricks workspaces and Unity Catalog for multi-tenant environments - Architecting Lakehouse solutions with Delta Lake - Demonstrating expertise in Spark core for real-time and batch pipeline designing - Implementing data modelling, security & governance using RBAC and ABAC - Ensuring data lineage and metadata management - Orchestrating workflows using Databricks Workflows - Applying best practices for artifact organization and naming conventions in Databricks Qualifications required for this role include: - Minimum of 10 years of experience in the field - Deep understanding of AWS services such as Re...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your key responsibilities will include: - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity, and consistency. - Optimizing data workflows for enhanced performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. - Implementing dat...

Role Overview: You will be leading the development of scalable, secure, and high-performance data solutions as a Lead Data Engineer. Your role involves utilizing your technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory to ensure data governance, metadata management, and regulatory compliance within insurance and financial services sectors. Key Responsibilities: - Design and architect end-to-end data engineering solutions across cloud platforms. - Create and manage robust data pipelines and ETL workflows using Python and Apache Spark. - Implement scalable Delta Lake solutions for structured and semi-structured data. ...

Req ID: 341055 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Databricks Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Databricks Data Platform - Architect Job Description Role Summary As a Databricks Architect , you will lead the design and implementation of scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business g...

As a highly skilled Sr. Developer with 7 to 10 years of experience, you will be an integral part of our dynamic team. Your expertise in Python, Databricks SQL, Databricks Workflows, and PySpark will be crucial in driving the success of our innovative projects. This role offers a hybrid work model with day shifts, providing you with the opportunity to contribute to scalable data processing systems and enhance data analytics capabilities. **Key Responsibilities:** - Develop and maintain scalable data processing systems using Python and PySpark to enhance data analytics capabilities. - Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data oper...

As a highly skilled Senior Developer with 7 to 10 years of experience, you will be responsible for developing and maintaining scalable data solutions using Databricks Unity Catalog and Azure Data Lake Store to enhance data accessibility and security. Your key responsibilities will include: - Collaborating with cross-functional teams to integrate Azure DevOps into the development lifecycle ensuring seamless deployment and continuous integration. - Utilizing Python and PySpark to design and implement efficient data processing pipelines optimizing performance and reliability. - Creating and managing Databricks SQL queries to extract, transform, and load data supporting business intelligence and...

Would being part of a digital transformation excite you Are you passionate about infrastructure security Join our digital transformation team operating at the heart of the digital transformation of our business. Our team is responsible for cybersecurity, architecture, and data protection for our global organization, advising on the design and validation of systems, infrastructure, technologies, and data protection. As a Senior Infrastructure Architect, you will participate in domain technical and business discussions, assist in analysis, design, and development of roadmap and implementation, gather and analyze data, develop architectural requirements, participate in the infrastructure archit...

The ML Platform Specialist role involves designing, implementing, and maintaining machine learning infrastructure and workflows on the Databricks Lakehouse Platform. Your primary responsibility will be to ensure the successful deployment, monitoring, and scaling of machine learning models across the organization. You will design and implement scalable ML infrastructure on the Databricks Lakehouse Platform, develop CI/CD pipelines for machine learning models, and create automated testing and validation processes using Databricks MLflow. Additionally, you will be responsible for managing model monitoring systems, collaborating with various teams to optimize machine learning workflows, and main...

At EVERSANA, we are proud to be certified as a Great Place to Work across the globe. Were fueled by our vision to create a healthier world. How Our global team of more than 7,000 employees is committed to creating and delivering next-generation commercialization services to the life sciences industry. We are grounded in our cultural beliefs and serve more than 650 clients ranging from innovative biotech start-ups to established pharmaceutical companies. Our products, services and solutions help bring innovative therapies to market and support the patients who depend on them. Our jobs, skills and talents are unique, but together we make an impact every day. Join us! Across our growing organiz...