584 Delta Lake Jobs - Page 11

Role Overview: You will be leading the development of scalable, secure, and high-performance data solutions as a Lead Data Engineer. Your role involves utilizing your technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data Factory to ensure data governance, metadata management, and regulatory compliance within insurance and financial services sectors. Key Responsibilities: - Design and architect end-to-end data engineering solutions across cloud platforms. - Create and manage robust data pipelines and ETL workflows using Python and Apache Spark. - Implement scalable Delta Lake solutions for structured and semi-structured data. ...

As a Sr. Data Analytics Engineer at Ajmera Infotech, you play a pivotal role in powering mission-critical decisions with governed insights for NYSE-listed clients. You will be part of a 120-engineer team specializing in highly regulated domains such as HIPAA, FDA, and SOC 2, delivering production-grade systems that transform data into a strategic advantage. Your responsibilities will include: - Building scalable pipelines using SQL, PySpark, and Delta Live Tables on Databricks. - Orchestrating workflows with Databricks Workflows or Airflow, including implementing SLA-backed retries and alerting. - Designing dimensional models (star/snowflake) with Unity Catalog and Great Expectations validat...

Role Overview: We are looking for a Lead Data Engineer to own the end-to-end technical delivery of the Data Observability & Monitoring solution. This role involves designing the architecture, leading a small team of engineers, and working with stakeholders to ensure alignment between infrastructure metrics and business dashboards. Responsibilities: Own the architecture for the observability solution (ingestion, storage, serving layers). Lead a team of Data Engineers, reviewing code and ensuring best practices. Define strategy for integrating infrastructure metrics (cost, uptime, CPU, memory) into Delta Lake. Collaborate with Solution Architect (you) to design client-ready dashboards. Establi...

As a Senior Azure Data Engineer, your responsibilities will include: Building scalable data pipelines using Databricks and PySpark Transforming raw data into usable business insights Integrating Azure services like Blob Storage, Data Lake, and Synapse Analytics Deploying and maintaining machine learning models using MLlib or TensorFlow Executing large-scale Spark jobs with performance tuning on Spark Pools Leveraging Databricks Notebooks and managing workflows with MLflow Qualifications: Bachelors/Masters in Computer Science, Data Science, or equivalent 7+ years in Data Engineering, with 3+ years in Azure Databricks Strong hands-on in: PySpark, Spark SQL, RDDs, Pandas, NumPy, Delta Lake Azur...

Job Title: Data/ML Platform Engineer Location: Gurgaon, Pune, Bangalore, Chennai, Bhopal, Jaipur, Hyderabad (Work from office) Notice Period: ImmediateiSource Services is hiring for one of their client for the position of Data/ML Platform Engineer. As a Data Engineer you will be relied on to independently develop and deliver high-quality features for our new ML Platform, refactor and translate our data products and finish various tasks to a high standard. Youll be part of the Data Foundation Team, which focuses on creating and maintaining the Data Platform for Marktplaats. 5 years of hands-on experience in using Python, Spark,Sql. Experienced in AWS Cloud usage and management. Experience wit...

We are looking for an experienced Data Engineer/BI Developer with strong hands-on expertise in Microsoft Fabric technologies, including OneLake, Lakehouse, Data Lake, Warehouse, and Real-Time Analytics, along with proven skills in Power BI, Azure Synapse Analytics, and Azure Data Factory (ADF). The ideal candidate should also possess working knowledge of DevOps practices for data engineering and deployment automation. Key Responsibilities: Design and implement scalable data solutions using Microsoft Fabric components: OneLake, Data Lake, Lakehouse, Warehouse, and Real-Time Analytics Build and manage end-to-end data pipelines integrating structured and unstructured data from multiple sources....

Req ID: 341055 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Databricks Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Databricks Data Platform - Architect Job Description Role Summary As a Databricks Architect , you will lead the design and implementation of scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business g...

As a Data Engineer, you will be responsible for designing and building efficient data pipelines using Azure Databricks (PySpark). You will implement business logic for data transformation and enrichment at scale, as well as manage and optimize Delta Lake storage solutions. Additionally, you will develop REST APIs using FastAPI to expose processed data and deploy them on Azure Functions for scalable and serverless data access. Key Responsibilities: - Design and build efficient data pipelines using Azure Databricks (PySpark) - Implement business logic for data transformation and enrichment at scale - Manage and optimize Delta Lake storage solutions - Develop REST APIs using FastAPI to expose p...

Role Overview: As a Machine Learning Engineer at our growing team, you will be responsible for developing and implementing machine learning models to address real-world challenges and drive business value. You will collaborate closely with data scientists, software engineers, and product managers to create and deploy innovative solutions. Key Responsibilities: - Design, develop, and implement machine learning models for various applications such as predictions, customer churn analysis, and fraud detection. - Collect, clean, and preprocess large datasets for training and evaluation. - Conduct exploratory data analysis and feature engineering to enhance model performance. - Evaluate and select...

What is expected of you at this level: We are seeking an experienced Azure Data Engineer with 4+ years in data engineering to join our team. In this role, you will design, build, and optimize high-performance data pipelines on Azure, delivering scalable, efficient, and reliable data solutions. You will take ownership of developing and maintaining critical data workflows, ensuring they meet performance and quality standards. This position requires a strong focus on performance tuning, code optimization, deployment automation, and advanced debugging, as well as the ability to work independently and collaboratively to solve complex data challenges and drive continuous improvements across our da...

Role Overview: As an AVP - Databricks, your primary responsibility will be to lead and manage Databricks-based project delivery. You will ensure that solutions are designed, developed, and implemented according to client requirements and industry standards. Your role will involve acting as a subject matter expert on Databricks, providing guidance on architecture, implementation, and optimization to teams. Collaboration with architects and engineers to design optimal solutions for data processing, analytics, and machine learning workloads will also be a key aspect of your role. Additionally, you will serve as the primary point of contact for clients to ensure alignment between business requir...

As the AVP Databricks Squad Delivery Lead, you will play a crucial role in overseeing project delivery, team leadership, architecture reviews, and client engagement. Your primary responsibility will be to optimize Databricks implementations across cloud platforms such as AWS, Azure, and GCP, while leading cross-functional teams. - Lead and manage the end-to-end delivery of Databricks-based solutions. - Design scalable data pipelines and analytics platforms in collaboration with architects and engineers. - Oversee Databricks workspace setup, performance tuning, and cost optimization. - Ensure effective communication and alignment between business goals and technical solutions as the primary p...

Senior Data Engineer Databricks Location: Gurgaon (Hybrid 2 days in office) Job Type: Full Time About the Role We are looking for an experienced Senior Data Engineer with deep expertise in Databricks to design, build, and optimize large-scale data pipelines and cloud data platforms. This hybrid role requires 2 days a week in our Gurgaon office. Key Responsibilities Build & maintain scalable data pipelines using Databricks (PySpark, SparkSQL, Delta Lake). Design and optimize data lakehouse solutions on cloud (Azure/AWS/GCP). Improve pipeline performance, cost-efficiency, and reliability . Enable advanced analytics & ML use cases for data science teams. Ensure data quality, security, and gover...

About The Role Grade Level (for internal use): 11 Summary As a Data Engineer & AI Developer, you will play a key role in transforming complex data into clear, actionable insights. You will be responsible for building robust data pipelines, developing interactive dashboards, and integrating cutting-edge AI techniques to enhance our data processes and insights. This is a unique opportunity to blend expertise in data engineering and visualization with a forward-thinking approach to artificial intelligence. Key Responsibilities ETL/ELT Development: Design, build, and optimize scalable data pipelines using ETL/ELT processes to ingest, clean, and transform large datasets from various sources. AI-P...

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a AI Solution Architect to join our team in Bangalore, Karn?taka (IN-KA), India (IN). Role: - AI Solution Architect Experience: - 7+ Years Notice Period: - 30-60 Days Project Overview: We are seeking a seasoned AI Architect with strong experience in Generative AI and Large Language Models (LLMs)including OpenAI, Claude, and Geminito lead the design, orchestration, and deployment of intelligent solutions across complex use cases. You will architect conversational systems, ...

Data Scientist Profile Skill-Data Scientist, Demand Planning,Forecasting, SCM, Supply Chain, Data Brick,Pyspark,DeltaLake,ETL,Azure,AWS,Data Modeling Exp-8-12Yrs PKG Upto-32LPA Loc-Mumbai NP-Imm-30Days Ritika-8587970773 ritikab.imaginators@gmail.com Required Candidate profile Skill-Data Scientist, Demand Planning, Forecasting, SCM, Supply Chain, Data Brick, Pyspark, Delta Lake, ETL, Cluster Management, ML Model, Validation, Azure, AWS, Data Modeling NP-Imm-30Days

Total Experience: 1+ years Job Summary: We are looking for skilled Data Engineer(s) with strong expertise in Azure Data Factory to support ETL/ELT pipeline development, data integration, and transformation across cloud-based environments. Minimum Requirements: 1+ years of experience in data engineering roles. Strong hands-on experience with Azure Data Factory (ADF), including pipeline creation, triggers, and parameterized datasets. Proficiency in SQL for data extraction, transformation, and loading. Experience working with Azure Blob Storage, Azure SQL, and Synapse Analytics. Familiarity with source control tools such as Git and CI/CD pipelines for ADF. Strong debugging and troubleshooting s...

You're ready to gain the skills and experience needed to grow within your role and advance your career and there is a perfect software engineering opportunity for you at JPMorgan Chase within the Commercial & Investment Bank Team. As a Software Engineer II - Databricks Engineer + Python programming, you will serve as a seasoned member of an agile team to deliver trusted market-leading Data Engineering Solutions and Data products in a secure, stable, and scalable way. Your responsibilities will include implementing Databricks integration and interoperability with other cloud providers, collaborating with data scientists, analysts, and business stakeholders, developing and maintaining data pro...

Core Responsibilities Seasoned Databricks developer having 6+ years of experience is expected to design, build, and maintain scalable data pipelines and workflows using Azure Databricks. This includes: Developing ETL pipelines using Python, SQL, and Delta Live Tables Managing job execution through Databricks Jobs, including scheduling and parameterisation Implementing data transformation using built-in Databricks features Collaborating with data architects and analysts to ensure data models align with business needs Implementing security and governance using features like encryption, access control and data lineage tracking Data performance management using built-in Databricks features (i.e....

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

Job Title: Senior Data Engineer Azure Databricks Experience: 7+ Years Location: Navi Mumbai (On-site) Job Overview: We are looking for a highly skilled Senior Data Engineer with expertise in Azure Databricks, big data technologies, and cloud-based data architectures. The ideal candidate will have hands-on experience building scalable data pipelines and platforms for high-volume batch and real-time processing, preferably in the manufacturing or energy sector. Key Responsibilities: • Design and implement robust data pipelines using Azure Databricks, Spark, and other Azure data services. • Collaborate with architects, data scientists, and business teams to understand requirements and translate ...

This is a contract position, requires working in US CST time zone as an extended member of the US team. Client is URUS/VAS. We are seeking a detail-oriented and results-driven Project Manager to lead projects within our Data Platform organization, with a strong focus on Databricks implementations . The ideal candidate will have experience managing end-to-end delivery of cloud-based data platforms and collaborating cross-functionally across data engineering, analytics, DevOps, and business stakeholders. Key Responsibilities: • Lead and manage full lifecycle projects related to data platform initiatives, especially Databricks-based solutions across AWS or Azure. • Develop and maintain project ...

You are a seasoned AI Solution Architect with over 7 years of experience, including a strong background in Generative AI and Large Language Models (LLMs) such as OpenAI, Claude, and Gemini. Your role will involve architecting end-to-end LLM solutions for chatbot applications, semantic search, summarization, and domain-specific assistants. You will design modular, scalable LLM workflows, leverage Databricks Unity Catalog for centralized governance, and collaborate with data engineers to optimize model performance and deployment. **Key Responsibilities:** - Architect end-to-end LLM solutions for various applications. - Design modular, scalable LLM workflows. - Leverage Databricks Unity Catalog...

You will be responsible for designing and building ETL pipelines in Azure Databricks (PySpark, Delta Lake) to load, clean, and deliver data across Bronze, Silver, and Gold layers. Your role will also involve implementing Data Lakehouse Architecture on Azure Data Lake Gen2 with partitioning, schema management, and performance optimization. Additionally, you will need to develop data models (dimensional/star schema) for reporting in Synapse and Power BI. Integration of Databricks with Azure services like ADF, Key Vault, Event Hub, Synapse Analytics, Purview, and Logic Apps will be a part of your tasks. Building and managing CI/CD pipelines in Azure DevOps (YAML, Git repos, pipelines) will also...

As the Vice President of Engineering at Teradata, you will lead the India-based software development organization for the AI Platform Group. Your primary responsibility will be to execute the product roadmap for key technologies such as Vector Store, Agent platform, Apps, user experience, and AI/ML-driven use-cases at scale. Success in this role will involve building a world-class engineering culture, attracting and retaining top technical talent, accelerating product delivery, and driving innovation to bring measurable value to customers. Key Responsibilities: - Lead a team of 150+ engineers to help customers achieve outcomes with Data and AI - Partner closely with Product Management, Produ...