584 Delta Lake Jobs - Page 12

Role Overview: As an experienced Data Architect specializing in Databricks, your main responsibility will be to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS. You will be designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers) to support large-scale ingestion, transformation, and publishing of data. Key Responsibilities: - Define and design cloud-native data architecture on Databricks using Delta Lake, Unity Catalog, and related services. - Develop migration strategies for moving on-premises data workloads ...

Exp: 10+ years Job Role : AWS Architect with Databricks Job Location: Noida / Mumbai / Pune / Bangalore / Gurgaon / Kochi (Hybrid Work) Role & responsibilities Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark, Delta Lake) on AWS. Ingest and process structured, semi-structured, and unstructured data from multiple sources (RDBMS, APIs, streaming). Integrate AWS services such as S3, Glue, Lambda, Kinesis, Athena, Redshift with Databricks. Implement data quality checks, validation frameworks, and monitoring for production workloads. Optimize data storage, partitioning, and caching strategies for performance and cost efficiency. Collaborate with Data Architects and Analysts...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

At Capgemini Invent, we believe that difference drives change. As inventive transformation consultants, we blend our strategic, creative, and scientific capabilities to collaborate closely with clients in delivering cutting-edge solutions tailored to address the challenges of today and tomorrow. Our approach is informed and validated by science and data, superpowered by creativity and design, and all underpinned by purpose-driven technology. Your role will involve proficiency in various technologies such as MS Fabric, Azure Data Factory, Azure Synapse Analytics, Azure Databricks Lakehouses, OneLake, Data Pipelines, Real-Time Analytics, Power BI Integration, and Semantic Model. You will be re...

Skills Required: Familiarity with data processing engines such as Apache Spark, Flink, or other big data tools. Design, develop, and implement robust data lake architectures on cloud platforms (AWS/Azure). Implement streaming and batch data pipelines using Apache Hudi, Apache Hive, and cloud-native services like AWS Glue, Azure Data Lake, etc. Architect and optimize ingestion, compaction, partitioning, and indexing strategies in Apache Hudi. Develop scalable data transformation and ETL frameworks using Python, Spark, and Flink. Work closely with DataOps/DevOps to build CI/CD pipelines and monitoring tools for data lake platforms. Ensure data governance, schema evolution handling, lineage tra...

5+ years of experience in Data Engineering, with 3+ years specifically on Azure Databricks, Data factory. Strong experience with PySpark, Spark SQL, and Delta Lake. Proficiency in data transformations, ETL design, and performance tuning.

Role & responsibilities Hands on experience in Microsoft Fabric, Logic Apps, Azure OpenAI basics Experienced in Delta Lake, Power BI, or Azure DevOps. Knowledge of Spark, Scala, or other distributed processing frameworks. Exposure to BI tools like Power BI, Tableau, or Looker. Familiarity with data security and compliance in the cloud. Experience in leading a development team.

You are a highly skilled and experienced Data Modeler who will be joining our data engineering team. Your expertise lies in designing scalable and efficient data models for cloud platforms, with a focus on Oracle Data Warehousing and Databricks Lakehouse architecture. Your role will be crucial in our strategic transition from an on-prem Oracle data warehouse to a modern cloud-based Databricks platform. Your responsibilities will include designing and implementing conceptual, logical, and physical data models to meet business requirements and analytics needs. You will lead the migration of data models from Oracle Data Warehouse to Databricks on AWS or Azure cloud, reverse-engineer complex Ora...

You are looking for an Assistant Vice President (AVP) Databricks Squad Delivery Lead with at least 14 years of experience to join the team in Bangalore/Hyderabad/NCR/Kolkata/Mumbai/Pune. As the Databricks Squad Delivery Lead, you will be responsible for overseeing project delivery, team leadership, architecture reviews, and client engagement. Your role will involve optimizing Databricks implementations across cloud platforms such as AWS, Azure, and GCP, while leading cross-functional teams. Your key responsibilities will include leading and managing end-to-end delivery of Databricks-based solutions, serving as a subject matter expert (SME) for Databricks architecture, collaborating with arch...

The Firmwide Resiliency Office (FRO), part of the Office of the Chief Finance Officer, is responsible for designing the Firm's Resilience strategy. This includes Resiliency Planning, Testing, Exercising, Reporting, and Product and Concern Management. The team comprises technical product, data, and analytic roles that support business resiliency. FRO collaborates closely with senior leadership, Lines of Business, Functional Resiliency teams, and key functions such as Control Management, Risk Management & Compliance, and Audit to ensure the resiliency program aligns with the firm's risk-taking activities. Additionally, the team provides corporate governance, awareness, and training. As a Data ...

You are an experienced Data Architect who will be responsible for leading the transformation of enterprise data solutions, particularly focused on migrating Alteryx workflows into Azure Databricks. Your expertise in the Microsoft Azure ecosystem, including Azure Data Factory, Databricks, Synapse Analytics, Microsoft Fabric, and strong background in data architecture, governance, and distributed computing will be crucial for this role. Your strategic thinking and hands-on architectural leadership will ensure the development of scalable, secure, and high-performance data solutions. Your key responsibilities will include defining the migration strategy for transforming Alteryx workflows into sc...

As a Data Engineer, you will be responsible for building scalable data pipelines using PySpark. Your role will involve implementing complex business logic using Spark SQL, DataFrame, and RDD APIs. You should have strong programming skills in Python, with a solid understanding of data structures, algorithms, and software engineering principles. Your expertise in designing, developing, and maintaining batch and streaming data pipelines will be crucial. You should be familiar with ETL/ELT processes and best practices for data transformation, data quality, and performance optimization. Knowledge of the modern data engineering ecosystem, including distributed data processing, storage systems, and...

Role Summary: We are seeking a Data Engineer with strong expertise in Databricks and large-scale data engineering to design, build, and optimize our data lakehouse solutions. The ideal candidate will be hands-on with PySpark, SQL, Delta Lake, and cloud data platforms, and will play a key role in enabling data-driven decision-making across the organization. Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark. Implement and optimize data ingestion from multiple structured and unstructured sources (APIs, databases, streaming data, cloud storage). Build and manage Delta Lake architectures for data lakehouse solutions. Collaborate with data s...

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role: Data Engineer Location: Mumbai, 5th Floor, 3, North Avenue, Maker Maxity, Bandra Kurla Complex, Contract: 12+months, likely long term Work Mode: Hybrid Notice Period: Immediate - 15days Strong Experience in PySpark Hands-on expertise in building scalable data pipelines using PySpark. Proficiency in using Spark SQL, DataFrame, and RDD APIs to implement complex business logic. Proficient Programming Skills Solid coding skills in Python (preferred), with strong fundamentals in data structures, algorithms, and software engineering principles. Data Pipeline Development Proven experience designing, developing, and maintaining batch and streaming data pipelines. Understanding of ETL/ELT proce...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and the desire to create lasting value for clients. We serve leading enterprises, including the Fortune Global 500, with deep business knowledge, digital operations services, and expertise in data, technology, and AI. We are inviting applications for the position of Vice President, Databricks Squad Delivery Lead. The successful candidate will be responsible for overseeing the end-to-end delivery of Databricks-based solutions for clients, ensuring successful implementation, optimization, and scaling of big data and analytics solutions. This role involves dri...

As a Data Solutions Specialist at Quantiphi, you will be part of a global and diverse culture that values transparency, diversity, integrity, learning, and growth. We take pride in fostering an environment that encourages innovation and excellence, not only in your professional endeavors but also in your personal life. Key Responsibilities: - Utilize your 3+ years of hands-on experience to deliver data solutions that drive business outcomes. - Develop data pipelines using PySpark within Databricks implementations. - Work with Databricks Workspaces, Notebooks, Delta Lake, and APIs to streamline data processes. - Utilize your expertise in Python, Scala, and advanced SQL for effective data mani...

As a Databricks AWS/Azure/GCP Architect at Koantek based in Mumbai, you will play a crucial role in building secure and highly scalable big data solutions that drive tangible, data-driven outcomes while emphasizing simplicity and operational efficiency. Collaborating with teammates, product teams, and cross-functional project teams, you will lead the adoption and integration of the Databricks Lakehouse Platform into the enterprise ecosystem and AWS/Azure/GCP architecture. Your responsibilities will include implementing securely architected big data solutions that are operationally reliable, performant, and aligned with strategic initiatives. Your expertise should include an expert-level know...

Role: Assistant Manager - Data Engineering Experience: 4 to 8 years Location: Bengaluru, Karnataka , India (BLR) Job Description: Communication and leadership experience, with experience initiating and driving projects. Experience with data sets, Hadoop, and data modernisation tools. Experience in SQL or similar languages and in any one cloud platform AWS/Azure/GCP. Development experience in at least one object-oriented language (Python, Java, etc.). BA/BS in Computer Science, Math, Physics, or other technical fields. Job Responsibility: Data Engineering & Architecture Design and implement scalable and optimized data pipelines on Databricks using Delta Lake, PySpark, and SQL. Develop ETL/ELT...

Databricks Developer (5–7 yrs exp) skilled in Apache Spark, Design & Implementation, PySpark, Delta Lake, and cloud platforms (Azure/AWS). Must excel in data pipelines, ETL, governance, and performance tuning. Unity Catalog, ML, and CI/CD experience.

Key Responsibilities Design, build, and maintain a robust data platform on Databricks. Develop efficient ETL/ELT pipelines to ingest data from SQL Server, MongoDB, and InfluxDB into Databricks Delta Lake. Implement real-time and batch data ingestion strategies using tools such as Kafka, Azure Event Hubs, or similar. Optimize data storage and processing for performance, scalability, and cost-effectiveness. Build data models to support BI, advanced analytics, and machine learning use cases. Collaborate with stakeholders (Data Scientists, Analysts, Product Teams) to define data requirements. Ensure data quality, governance, and security across the platform. Monitor, troubleshoot, and enhance da...

As a Senior Databricks Developer at Newscape Consulting, you will play a crucial role in our data engineering team, focusing on building scalable and efficient data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure/AWS/GCP). Your responsibilities will include collaborating closely with data architects, data scientists, and business stakeholders to deliver high-performance, production-grade solutions that enhance user experience and productivity in the healthcare industry. Your key skills should include a strong hands-on experience with Databricks including Workspace, Jobs, DLT, Repos, and Unity Catalog. Proficiency in PySpark, Spark SQL, and optionally Sc...