29 Databricks Workflows Jobs

Job Title: Databricks SQL Engineer with Commercial Pharma Location: Remote Type: Contract About the Role: We are looking for a Databricks SQL Engineer to design, develop, and optimize data solutions using Databricks SQL. The ideal candidate will have expertise in SQL, performance tuning, data modeling, and cloud-based data platforms such as AWS, Azure, or GCP. Key Responsibilities: Develop and optimize SQL queries and notebooks in Databricks for data transformation, aggregation, and analysis. Optimize Databricks SQL workloads for performance, cost, and scalability. Monitor, debug, and troubleshoot Databricks SQL queries and jobs. Required Qualifications: 4+ years of experience with SQL devel...

Role: Data Engineer Experience: 3-5 yrs Notice Period: Immediate Work Mode: Hybrid Job Description Requirements: Proficient in Python (Including popular python packages e.g. Pandas, NumPy etc.) and SQL Strong background in distributed data processing and storage (e.g. Apache, Spark, Hadoop) Large scale (TBs of data) data engineering skills - Model data, create production ready ETL pipelines Development experience with at least one cloud (Azure high preference, AWS, GCP) Knowledge of data lake and data lake house patterns Knowledge of ETL performance tuning and cost optimization Knowledge of data structures and algorithms and good software engineering practices Nice to Have: Experience with A...

Responsibilities ETL Development: Build and maintain ETL/ELT pipelines using Databricks & Apache Spark (PySpark/Scala). Data Warehousing: Design and implement Star Schema, Snowflake Schema, and Data Vault models. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Databricks Optimization: Optimize Delta Lake storage, caching, and job execution to improve query performance. Data Ingestion & Processing: Develop scalable ingestion pipelines using Structured Streaming, Auto Loader, and Databricks Workflows. Data Governance & Security: Implement Unity Catalog, role-based access controls (RBAC), and compliance standards. Performance Tuning...

As an Azure DB Architect with 8+ years of experience, your role will involve: - Designing Databricks workspaces and Unity Catalog for multi-tenant environments - Architecting Lakehouse solutions with Delta Lake - Demonstrating expertise in Spark core for real-time and batch pipeline designing - Implementing data modelling, security & governance using RBAC and ABAC - Ensuring data lineage and metadata management - Orchestrating workflows using Databricks Workflows - Applying best practices for artifact organization and naming conventions in Databricks Qualifications required for this role include: - Minimum of 10 years of experience in the field - Deep understanding of AWS services such as Re...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your key responsibilities will include: - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity, and consistency. - Optimizing data workflows for enhanced performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. - Implementing dat...

Role Overview: You will be leading the development of scalable, secure, and high-performance data solutions as a Lead Data Engineer. Your role involves utilizing your technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory to ensure data governance, metadata management, and regulatory compliance within insurance and financial services sectors. Key Responsibilities: - Design and architect end-to-end data engineering solutions across cloud platforms. - Create and manage robust data pipelines and ETL workflows using Python and Apache Spark. - Implement scalable Delta Lake solutions for structured and semi-structured data. ...

Req ID: 341055 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Databricks Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Databricks Data Platform - Architect Job Description Role Summary As a Databricks Architect , you will lead the design and implementation of scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business g...

As a highly skilled Sr. Developer with 7 to 10 years of experience, you will be an integral part of our dynamic team. Your expertise in Python, Databricks SQL, Databricks Workflows, and PySpark will be crucial in driving the success of our innovative projects. This role offers a hybrid work model with day shifts, providing you with the opportunity to contribute to scalable data processing systems and enhance data analytics capabilities. **Key Responsibilities:** - Develop and maintain scalable data processing systems using Python and PySpark to enhance data analytics capabilities. - Collaborate with cross-functional teams to design and implement Databricks Workflows that streamline data oper...

As a highly skilled Senior Developer with 7 to 10 years of experience, you will be responsible for developing and maintaining scalable data solutions using Databricks Unity Catalog and Azure Data Lake Store to enhance data accessibility and security. Your key responsibilities will include: - Collaborating with cross-functional teams to integrate Azure DevOps into the development lifecycle ensuring seamless deployment and continuous integration. - Utilizing Python and PySpark to design and implement efficient data processing pipelines optimizing performance and reliability. - Creating and managing Databricks SQL queries to extract, transform, and load data supporting business intelligence and...

Would being part of a digital transformation excite you Are you passionate about infrastructure security Join our digital transformation team operating at the heart of the digital transformation of our business. Our team is responsible for cybersecurity, architecture, and data protection for our global organization, advising on the design and validation of systems, infrastructure, technologies, and data protection. As a Senior Infrastructure Architect, you will participate in domain technical and business discussions, assist in analysis, design, and development of roadmap and implementation, gather and analyze data, develop architectural requirements, participate in the infrastructure archit...

The ML Platform Specialist role involves designing, implementing, and maintaining machine learning infrastructure and workflows on the Databricks Lakehouse Platform. Your primary responsibility will be to ensure the successful deployment, monitoring, and scaling of machine learning models across the organization. You will design and implement scalable ML infrastructure on the Databricks Lakehouse Platform, develop CI/CD pipelines for machine learning models, and create automated testing and validation processes using Databricks MLflow. Additionally, you will be responsible for managing model monitoring systems, collaborating with various teams to optimize machine learning workflows, and main...

At EVERSANA, we are proud to be certified as a Great Place to Work across the globe. Were fueled by our vision to create a healthier world. How Our global team of more than 7,000 employees is committed to creating and delivering next-generation commercialization services to the life sciences industry. We are grounded in our cultural beliefs and serve more than 650 clients ranging from innovative biotech start-ups to established pharmaceutical companies. Our products, services and solutions help bring innovative therapies to market and support the patients who depend on them. Our jobs, skills and talents are unique, but together we make an impact every day. Join us! Across our growing organiz...

Data Architect - Databricks (Azure/AWS) Role Overview: We are seeking an experienced Data Architect specializing in Databricks to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS . The role involves designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers), supporting large-scale ingestion, transformation, and publishing of data. Required Skills and Experience: 8+ years of experience in data architecture or engineering roles, with at least 3+ years specializing in cloud-based big data solutions. Hands-on experti...

You should have over ten years of experience in software development, including recent years in a leadership position with increasing responsibilities. Your expertise should cover various areas such as C#, .Net, designing and developing cloud-native solutions on platforms like Azure, AWS, and Google Cloud Platform, using Infrastructure as Code tools like Terraform and Pulumi, employing configuration management tools like Ansible and Chef, working with containerization and orchestration technologies like Docker and Kubernetes, and integrating with Native and third-party Databricks solutions. Additionally, you should possess extensive experience specifically in Azure. It would be advantageous ...

Role overview Experience : 10- 18 10 plus years experience in software development with at least the last few years in a leadership role with progressively increasing responsibilities 2. Extensive experience in the following areas Proficient in C#, .NET framework, Python Knowledgeable in Angular / React / Node Designing and building cloud-native solutions (Azure, AWS, Google Cloud Platform) Well versed with LLM solutions, RAG, Vectorization Containerization and orchestration technologies (Docker, Kubernetes) Experience in native and third-party Databricks integrations (Delta Live Tables, Lakeflow, Databricks Workflows / Apache Airflow, Unity Catalog) Experience designing and implementing dat...

The Databricks Data Engineering Lead role requires a highly skilled individual who will architect and lead the implementation of scalable, high-performance data pipelines and platforms using the Databricks Lakehouse ecosystem. As a Data Engineering Lead, you will be responsible for managing a team of data engineers, establishing best practices, and collaborating with cross-functional stakeholders to unlock advanced analytics, AI/ML, and real-time decision-making capabilities. Your key responsibilities will include leading the design and development of modern data pipelines, data lakes, and lakehouse architectures using Databricks and Apache Spark. You will manage and mentor a team of data en...

We are looking for a seasoned Senior Data Architect with extensive knowledge in Databricks and Microsoft Fabric to join our team. In this role, you will be responsible for leading the design and implementation of scalable data solutions for BFSI and HLS clients. As a Senior Data Architect specializing in Databricks and Microsoft Fabric, you will play a crucial role in architecting and implementing secure, high-performance data solutions on the Databricks and Azure Fabric platforms. Your responsibilities will include leading discovery workshops, designing end-to-end data pipelines, optimizing workloads for performance and cost efficiency, and ensuring compliance with data governance, security...

As a Senior Infrastructure Architect at our organization, you will play a crucial role in our digital transformation journey. You will have the opportunity to be involved in cybersecurity, architecture, and data protection for our global organization. Collaborating with our team, you will provide expertise in designing and validating systems, infrastructure, technologies, and data protection. Your responsibilities will include participating in technical and business discussions to shape future architecture direction, analyzing data to develop architectural requirements, and contributing to infrastructure architecture governance. Additionally, you will be involved in designing and deploying i...

The ideal candidate ready to join immediately can share their details via email for quick processing at nitin.patil@ust.com. Act swiftly for immediate attention! With over 5 years of experience, the successful candidate will have the following roles and responsibilities: - Designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity...

We are seeking an experienced and forward-thinking Lead Data Engineer to spearhead the development of scalable, secure, and high-performance data solutions. You must possess in-depth technical knowledge in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory. Your expertise should also include a solid understanding of data governance, metadata management, and regulatory compliance within the insurance and financial services sectors. Proficiency in developing Python applications, Spark-based workflows, utilizing Delta Lake, and orchestrating jobs with Databricks Workflows or Azure Data Factory is essential. Furthermore, you should be able t...

As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark, specifically PySpark or Spark with Scala. Your role will involve building data ingestion and transformation frameworks for various structured and unstructured data sources. Collaboration with data analysts, data scientists, and business stakeholders is essential to understand requirements and deliver reliable data solutions. Working with large volumes of data, you will ensure quality, integrity, and consistency while optimizing data workflows for performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. Implementation of data quality c...

Candidates who are ready to join immediately can share their details via email for quick processing to nitin.patil@ust.com. Act fast for immediate attention! With over 5 years of experience, the ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines using Spark, either PySpark or Spark with Scala. They will also be tasked with building data ingestion and transformation frameworks for structured and unstructured data sources. Collaboration with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions is a key aspect of the role. The candidate will work with large volumes of data to ...

The ideal candidate ready to join immediately can share their details via email for quick processing at nitin.patil@ust.com. Act fast for immediate attention! With over 5 years of experience, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). You will also build data ingestion and transformation frameworks for structured and unstructured data sources. Collaboration with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions will be a key aspect of the role. Working with large volumes of data, ensuring quality, integrity, and consistency, and opt...

The ideal candidate for this position should have at least 5 years of experience and must be ready to join immediately. In this role, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark, specifically PySpark or Spark with Scala. You will also be tasked with building data ingestion and transformation frameworks for structured and unstructured data sources. Collaboration with data analysts, data scientists, and business stakeholders to understand requirements and deliver reliable data solutions is a key aspect of this role. Working with large volumes of data to ensure quality, integrity, and consistency is crucial. Additionally, optimizing dat...

As a Sr. Data Analytics Engineer at Ajmera Infotech Private Limited (AIPL) in Bengaluru/Bangalore, you will play a crucial role in building planet-scale software for NYSE-listed clients, enabling decisions that are critical and must not fail. With 5-9 years of experience, you will join a 120-engineer team specializing in highly regulated domains such as HIPAA, FDA, and SOC 2. The team delivers production-grade systems that transform data into a strategic advantage. You will have the opportunity to make end-to-end impact by building full-stack analytics solutions ranging from lake house pipelines to real-time dashboards. Fail-safe engineering practices such as TDD, CI/CD, DAX optimization, Un...