615 Delta Lake Jobs - Page 13

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

Job Title: Senior Data Engineer Azure Databricks Experience: 7+ Years Location: Navi Mumbai (On-site) Job Overview: We are looking for a highly skilled Senior Data Engineer with expertise in Azure Databricks, big data technologies, and cloud-based data architectures. The ideal candidate will have hands-on experience building scalable data pipelines and platforms for high-volume batch and real-time processing, preferably in the manufacturing or energy sector. Key Responsibilities: • Design and implement robust data pipelines using Azure Databricks, Spark, and other Azure data services. • Collaborate with architects, data scientists, and business teams to understand requirements and translate ...

This is a contract position, requires working in US CST time zone as an extended member of the US team. Client is URUS/VAS. We are seeking a detail-oriented and results-driven Project Manager to lead projects within our Data Platform organization, with a strong focus on Databricks implementations . The ideal candidate will have experience managing end-to-end delivery of cloud-based data platforms and collaborating cross-functionally across data engineering, analytics, DevOps, and business stakeholders. Key Responsibilities: • Lead and manage full lifecycle projects related to data platform initiatives, especially Databricks-based solutions across AWS or Azure. • Develop and maintain project ...

You are a seasoned AI Solution Architect with over 7 years of experience, including a strong background in Generative AI and Large Language Models (LLMs) such as OpenAI, Claude, and Gemini. Your role will involve architecting end-to-end LLM solutions for chatbot applications, semantic search, summarization, and domain-specific assistants. You will design modular, scalable LLM workflows, leverage Databricks Unity Catalog for centralized governance, and collaborate with data engineers to optimize model performance and deployment. **Key Responsibilities:** - Architect end-to-end LLM solutions for various applications. - Design modular, scalable LLM workflows. - Leverage Databricks Unity Catalog...

You will be responsible for designing and building ETL pipelines in Azure Databricks (PySpark, Delta Lake) to load, clean, and deliver data across Bronze, Silver, and Gold layers. Your role will also involve implementing Data Lakehouse Architecture on Azure Data Lake Gen2 with partitioning, schema management, and performance optimization. Additionally, you will need to develop data models (dimensional/star schema) for reporting in Synapse and Power BI. Integration of Databricks with Azure services like ADF, Key Vault, Event Hub, Synapse Analytics, Purview, and Logic Apps will be a part of your tasks. Building and managing CI/CD pipelines in Azure DevOps (YAML, Git repos, pipelines) will also...

As the Vice President of Engineering at Teradata, you will lead the India-based software development organization for the AI Platform Group. Your primary responsibility will be to execute the product roadmap for key technologies such as Vector Store, Agent platform, Apps, user experience, and AI/ML-driven use-cases at scale. Success in this role will involve building a world-class engineering culture, attracting and retaining top technical talent, accelerating product delivery, and driving innovation to bring measurable value to customers. Key Responsibilities: - Lead a team of 150+ engineers to help customers achieve outcomes with Data and AI - Partner closely with Product Management, Produ...

Role Overview: As an experienced Data Architect specializing in Databricks, your main responsibility will be to lead the architecture, design, and migration of enterprise data workloads from on-premises systems (e.g., Oracle, Exadata, Hadoop) to Databricks on Azure or AWS. You will be designing scalable, secure, and high-performing data platforms based on the medallion architecture (bronze, silver, gold layers) to support large-scale ingestion, transformation, and publishing of data. Key Responsibilities: - Define and design cloud-native data architecture on Databricks using Delta Lake, Unity Catalog, and related services. - Develop migration strategies for moving on-premises data workloads ...

Exp: 10+ years Job Role : AWS Architect with Databricks Job Location: Noida / Mumbai / Pune / Bangalore / Gurgaon / Kochi (Hybrid Work) Role & responsibilities Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark, Delta Lake) on AWS. Ingest and process structured, semi-structured, and unstructured data from multiple sources (RDBMS, APIs, streaming). Integrate AWS services such as S3, Glue, Lambda, Kinesis, Athena, Redshift with Databricks. Implement data quality checks, validation frameworks, and monitoring for production workloads. Optimize data storage, partitioning, and caching strategies for performance and cost efficiency. Collaborate with Data Architects and Analysts...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

What this job involves: JLL, an international real estate management company, is seeking an Data Engineer to join our JLL Technologies Team. We are seeking candidates that are self-starters to work in a diverse and fast-paced environment that can join our Enterprise Data team. We are looking for a candidate that is responsible for designing and developing of data solutions that are strategic for the business using the latest technologies Azure Databricks, Python, PySpark, SparkSQL, Azure functions, Delta Lake, Azure DevOps CI/CD. Responsibilities Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyze large, disparate data sets to exceed b...

At Capgemini Invent, we believe that difference drives change. As inventive transformation consultants, we blend our strategic, creative, and scientific capabilities to collaborate closely with clients in delivering cutting-edge solutions tailored to address the challenges of today and tomorrow. Our approach is informed and validated by science and data, superpowered by creativity and design, and all underpinned by purpose-driven technology. Your role will involve proficiency in various technologies such as MS Fabric, Azure Data Factory, Azure Synapse Analytics, Azure Databricks Lakehouses, OneLake, Data Pipelines, Real-Time Analytics, Power BI Integration, and Semantic Model. You will be re...

Skills Required: Familiarity with data processing engines such as Apache Spark, Flink, or other big data tools. Design, develop, and implement robust data lake architectures on cloud platforms (AWS/Azure). Implement streaming and batch data pipelines using Apache Hudi, Apache Hive, and cloud-native services like AWS Glue, Azure Data Lake, etc. Architect and optimize ingestion, compaction, partitioning, and indexing strategies in Apache Hudi. Develop scalable data transformation and ETL frameworks using Python, Spark, and Flink. Work closely with DataOps/DevOps to build CI/CD pipelines and monitoring tools for data lake platforms. Ensure data governance, schema evolution handling, lineage tra...

5+ years of experience in Data Engineering, with 3+ years specifically on Azure Databricks, Data factory. Strong experience with PySpark, Spark SQL, and Delta Lake. Proficiency in data transformations, ETL design, and performance tuning.

Role & responsibilities Hands on experience in Microsoft Fabric, Logic Apps, Azure OpenAI basics Experienced in Delta Lake, Power BI, or Azure DevOps. Knowledge of Spark, Scala, or other distributed processing frameworks. Exposure to BI tools like Power BI, Tableau, or Looker. Familiarity with data security and compliance in the cloud. Experience in leading a development team.

You are a highly skilled and experienced Data Modeler who will be joining our data engineering team. Your expertise lies in designing scalable and efficient data models for cloud platforms, with a focus on Oracle Data Warehousing and Databricks Lakehouse architecture. Your role will be crucial in our strategic transition from an on-prem Oracle data warehouse to a modern cloud-based Databricks platform. Your responsibilities will include designing and implementing conceptual, logical, and physical data models to meet business requirements and analytics needs. You will lead the migration of data models from Oracle Data Warehouse to Databricks on AWS or Azure cloud, reverse-engineer complex Ora...

You are looking for an Assistant Vice President (AVP) Databricks Squad Delivery Lead with at least 14 years of experience to join the team in Bangalore/Hyderabad/NCR/Kolkata/Mumbai/Pune. As the Databricks Squad Delivery Lead, you will be responsible for overseeing project delivery, team leadership, architecture reviews, and client engagement. Your role will involve optimizing Databricks implementations across cloud platforms such as AWS, Azure, and GCP, while leading cross-functional teams. Your key responsibilities will include leading and managing end-to-end delivery of Databricks-based solutions, serving as a subject matter expert (SME) for Databricks architecture, collaborating with arch...

The Firmwide Resiliency Office (FRO), part of the Office of the Chief Finance Officer, is responsible for designing the Firm's Resilience strategy. This includes Resiliency Planning, Testing, Exercising, Reporting, and Product and Concern Management. The team comprises technical product, data, and analytic roles that support business resiliency. FRO collaborates closely with senior leadership, Lines of Business, Functional Resiliency teams, and key functions such as Control Management, Risk Management & Compliance, and Audit to ensure the resiliency program aligns with the firm's risk-taking activities. Additionally, the team provides corporate governance, awareness, and training. As a Data ...

You are an experienced Data Architect who will be responsible for leading the transformation of enterprise data solutions, particularly focused on migrating Alteryx workflows into Azure Databricks. Your expertise in the Microsoft Azure ecosystem, including Azure Data Factory, Databricks, Synapse Analytics, Microsoft Fabric, and strong background in data architecture, governance, and distributed computing will be crucial for this role. Your strategic thinking and hands-on architectural leadership will ensure the development of scalable, secure, and high-performance data solutions. Your key responsibilities will include defining the migration strategy for transforming Alteryx workflows into sc...

As a Data Engineer, you will be responsible for building scalable data pipelines using PySpark. Your role will involve implementing complex business logic using Spark SQL, DataFrame, and RDD APIs. You should have strong programming skills in Python, with a solid understanding of data structures, algorithms, and software engineering principles. Your expertise in designing, developing, and maintaining batch and streaming data pipelines will be crucial. You should be familiar with ETL/ELT processes and best practices for data transformation, data quality, and performance optimization. Knowledge of the modern data engineering ecosystem, including distributed data processing, storage systems, and...

Role Summary: We are seeking a Data Engineer with strong expertise in Databricks and large-scale data engineering to design, build, and optimize our data lakehouse solutions. The ideal candidate will be hands-on with PySpark, SQL, Delta Lake, and cloud data platforms, and will play a key role in enabling data-driven decision-making across the organization. Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark. Implement and optimize data ingestion from multiple structured and unstructured sources (APIs, databases, streaming data, cloud storage). Build and manage Delta Lake architectures for data lakehouse solutions. Collaborate with data s...

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role: Data Engineer Location: Mumbai, 5th Floor, 3, North Avenue, Maker Maxity, Bandra Kurla Complex, Contract: 12+months, likely long term Work Mode: Hybrid Notice Period: Immediate - 15days Strong Experience in PySpark Hands-on expertise in building scalable data pipelines using PySpark. Proficiency in using Spark SQL, DataFrame, and RDD APIs to implement complex business logic. Proficient Programming Skills Solid coding skills in Python (preferred), with strong fundamentals in data structures, algorithms, and software engineering principles. Data Pipeline Development Proven experience designing, developing, and maintaining batch and streaming data pipelines. Understanding of ETL/ELT proce...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and the desire to create lasting value for clients. We serve leading enterprises, including the Fortune Global 500, with deep business knowledge, digital operations services, and expertise in data, technology, and AI. We are inviting applications for the position of Vice President, Databricks Squad Delivery Lead. The successful candidate will be responsible for overseeing the end-to-end delivery of Databricks-based solutions for clients, ensuring successful implementation, optimization, and scaling of big data and analytics solutions. This role involves dri...