580 Delta Lake Jobs

As an Azure Data Engineer, your role involves architecting, configuring, and optimizing Databricks Pipelines for large-scale data processing within an Azure Data Lakehouse environment. Your key responsibilities include: - Architect, configure, and optimize Databricks Pipelines for large-scale data processing within an Azure Data Lakehouse environment - Set up and manage Azure infrastructure components including Databricks Workspaces, AKS/ACI, Storage Accounts, and Networking - Design and implement monitoring and observability frameworks using Azure Monitor, Log Analytics, Prometheus, and Grafana - Collaborate with platform and data engineering teams to enable microservices-based architecture...

As an ML Engineer at GlobalLogic, you will play a crucial role in developing and deploying machine learning models in a professional setting. Your responsibilities will include: - Strong understanding of machine learning algorithms and techniques, including supervised, unsupervised, and reinforcement learning. - Building and deploying models using Python with Azure Data Factory (ADF), Azure Databricks, PySpark, Delta Lake, ETL/ELT, data pipelines, and data lakehouse architecture. - Proficiency in Python and relevant machine learning libraries such as scikit-learn, TensorFlow, and PyTorch. - Experience with cloud computing platforms like AWS, Azure, and GCP, as well as containerization techno...

Job Description: You specialize in modeling ServiceNow data for analytics, transforming raw extracts into curated lakehouse formats. This includes partitioned layers, business domains, and semantic enrichment for consumption by GenAI and BI tools. You ensure schema enforcement, lineage capture, and data classification are integrated into the transformation logic. Your experience includes building Silver and Gold zones for structured ITSM and CMDB use cases. Key Responsibilities: - Specializing in modeling ServiceNow data for analytics - Transforming raw extracts into curated lakehouse formats, including partitioned layers and semantic enrichment - Ensuring schema enforcement, lineage capture...

Job Title: Senior Azure Data Engineer Experience: 912 Years Location: Noida Job Description We are looking for a Senior Azure Data Engineer with extensive experience in building and optimizing large-scale data pipelines and architectures using the Microsoft Azure ecosystem . The ideal candidate should have deep hands-on expertise in Azure Data Factory (ADF) , Azure Databricks , and Unity Catalog , along with a strong understanding of data governance, security, and performance optimization . The role involves designing scalable, reliable, and secure data platforms that enable data-driven decision-making across the enterprise. Key Responsibilities Design and implement end-to-end data pipelines...

Experience: 6-9 years Location: Chennai, Bengaluru Notice Period: Immediate to 30 Days Job Description Technical Skills: Strong hands-on experience with Apache Spark, PySpark, and Scala. Proficiency in SQL and data manipulation techniques. Experience with Hadoop ecosystem tools (HDFS, Hive, YARN, etc.). Knowledge of cloud platforms (AWS, Azure, or GCP) and managed Spark services (e.g., Databricks, EMR, Glue). Familiarity with data lakehouse concepts, Delta Lake, or Iceberg is a plus. Experience with CI/CD, Git, and workflow orchestration tools (Airflow, Oozie, etc.). Understanding of data modeling, partitioning, and schema evolution strategies. Exposure to containerization (Docker, Kubernete...

Required Skills & Experience 57 years of experience in data engineering, with at least 23 years of Databricks hands-on experience. Strong expertise in Apache Spark (PySpark/Scala/SQL) and distributed data processing. Solid experience with Delta Lake, Lakehouse architecture, and data modeling. Hands-on experience with at least one cloud platform: Azure Data Lake, AWS S3, or GCP BigQuery/Storage. Strong proficiency in SQL for data manipulation and performance tuning. Experience with ETL frameworks, workflow orchestration tools (Airflow, ADF, DBX Workflows). Good understanding of CI/CD, Git-based workflows, and DevOps practices. Exposure to MLOps and MLflow is a strong plus. Knowledge of data g...

Position Overview: The Power BI Architect is a semantic modeling expert responsible for designing and optimizing enterprise-grade tabular models and broader Fabric solutions. The role is focused on backend modeling and DAX, with the goal of ensuring high performance, scalability, and simplicity for end users within a self-service environment. This Architect will be a key advisor to other Power BI developers, helping to solve modeling challenges while upholding standards and ensuring alignment with best practices. The Architect must bring experience with CI/CD tools to operate effectively in this environment. Key Responsibilities: Semantic Modeling and Architecture: - Design performant, scala...

About Us At TymeX, we're empowering Tyme Group's digital banking ecosystem across Asia and Africa. Our mission: build a secure, scalable, intelligent data & AI platform that delivers real-time, personalized experiences to millions of users. We are seeking a Senior Solutions Architect - Data & AI Platforms someone who blends deep technical mastery with the ability to engage directly with stakeholders, understand their business needs, and design/implement high-impact solutions. Your Role As a Senior Solutions Architect - Data & AI Platforms , you will be the trusted technical advisor to our stakeholders, guiding them through their data & AI journeys. You will represent TymeX's capabilities acr...

Company Profile: Quantum Integrators is an international strategy and business consulting group whose mission is to help clients create and sustain competitive advantage. As a truly client-focused firm, our highly trained and experienced consultants consider it their mandate to help client organizations achieve a quantum state. Position : AI and Databricks Engineer Location: Pune/Nagpur, Maharashtra (Onsite) Duration: Full time Shift: US/UK Shift Experience: 5-7 years mandatory Job Description: Strong experience developing industry grade LLM and Gen-AI applications using Open AI, Anthropic (Claude), or other major providers. Hands-on experience with Databricks (Spark, Delta Lake, MLflow, Dat...

Our Company Changing the world through digital experiences is what Adobe's all about. We give everyone-from emerging artists to global brands-everything they need to design and deliver exceptional digital experiences! We're passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies interact with customers across every screen. We're on a mission to hire the very best and are committed to creating exceptional employee experiences where everyone is respected and has access to equal opportunity. We realize that new ideas can come from everywhere in the organization, and we know the next big idea could be yours! Sr Computer Scientist ...

Job Description Are You Ready to Make It Happen at Mondel?z International Join our Mission to Lead the Future of Snacking. Make It With Pride. Together with analytics team leaders youwill support our businesswith excellent data models to uncover trends that can drive long-term business results. How you will contribute You will: Work in close partnership with the business leadership team to execute the analytics agenda Identify and incubate best-in-class external partners to drive delivery on strategic projects Develop custom models/algorithms to uncover signals/patterns and trends to drivelong-term business performance Execute the business analytics program agenda using a methodical approach...

Job description Hiring for Azure Databrick Engineer Mandatory Skills: Databrick, Data lake, ETL Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to actively aid the consulting team in different phases of the project including problem definition, effort estimation, diagnosis, solution generation and design and deployment You will explore the alternatives to the recommended solutions based on research that includes literature surveys, information available in public domains, vendor evaluation information, etc. and build POCs You will create requirement specifications from the business needs, define the to-be-processes and det...

As a Data Engineer at Crowe, you will play a crucial role in providing integration infrastructure for analytical support and solution development for the broader Enterprise. Your expertise in API integration, pipelines or notebooks, programming languages (Python, Spark, T-SQL), dimensional modeling, and advanced data engineering techniques will be essential in creating and delivering robust solutions and data products. You will be responsible for designing, developing, and maintaining the Enterprise Analytics Platform to facilitate data-driven decision-making across the organization. Your success in this role will rely on your strong interest and passion in data analytics, ETL/ELT best pract...

As a Data Engineer, you will be responsible for managing and optimizing data pipelines for efficient data processing. Your role will involve working with a variety of technologies and platforms to ensure smooth data workflows. Key Responsibilities: - Utilize your 5+ years of experience in data engineering, including at least 2+ years of hands-on experience with Databricks. - Demonstrate strong expertise in Apache Spark using PySpark, Scala, and SQL for efficient distributed data processing. - Implement and maintain Delta Lake and Lakehouse architecture for effective data modeling. - Work with cloud platforms such as Azure Data Lake, AWS S3, or GCP BigQuery/Storage to manage data infrastructu...

As an AI and Databricks Engineer, you will be responsible for developing industry-grade LLM and Gen-AI applications using OpenAI, Anthropic (Claude), or other major providers. Your role will involve hands-on experience with Databricks, including Spark, Delta Lake, MLflow, Databricks Notebooks, and Unity Catalog. You will need expertise in Python for building Autonomous Gen-AI solutions and proficiency in LangChain, LlamaIndex for building agentic architectures. Additionally, experience with vector databases such as Pinecone, Weaviate, Chroma, FAISS, RAG, ARAG, BigQ, CAG, A/B testing, reinforcement learning, etc., is required. Integration of LLM systems with APIs and enterprise data sources w...

Expert-level Python programming, software design & data structures. Exp. deploying ML/LLM models into production. Hands-on with MLOps/GenAIOps stacks (MLflow, Kubeflow, SageMaker, LangChain, AutoGen, LangGraph, CrewAI) & their real-world application. Required Candidate profile Sr.. Python Developer lead 8–10 yrs of exp. who can not only build, deploy scalable Python and ML/GenAI solutions lead & mentor a technical team. end-to-end project delivery, ensure high engineering

Position: AVP Databricks Architect Champion Location: Hyderabad | Work Mode: 5 Days from Office | Shift: UK Timings | Experience: 14+ yrs We are looking for an experienced Databricks Architect Champion to lead data platform modernization and large-scale Lakehouse transformation initiatives. The ideal candidate will bring strong hands-on expertise in Databricks, Unity Catalog, Delta Lake, PySpark, and Databricks SQL , along with a deep understanding of data governance, security, and scalable architecture design . Key Responsibilities: Architect and implement enterprise-grade Databricks Lakehouse solutions across multi-cloud environments (Azure/AWS). Drive Unity Catalog adoption for secure dat...

Python Data Engineer: - Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights. Constructing infrastructure for efficient ETL processes from various sources and storage systems. Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements. Leading the implementation of algorithms and prototypes to transform raw data into useful information. Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations. Creating innovative data validation methods and data analysis tools. Ensuring compliance w...

At EY, we're all in to shape your future with confidence. We'll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go. Join EY and help to build a better working world. Data Engineer Life Sciences R&D (Senior Level) Experience: 8+ Years Role Overview As a senior Data Engineer, you will be responsible for building robust, scalable, and compliant data pipelines from raw scientific sources to integrated and consumption-ready data products in the R&D landscape. You will partner with architects, domain SMEs, and analysts to develop re-usable components supporting early discovery, development, and regulatory needs. Key Responsibilities...

Data Architect ?( 8+ years in data architecture) Job Summary: We're seeking a skilled Data Architect to lead the design and delivery of innovative, scalable data solutions that power our business. With deep expertise in Databricks and cloud platforms like Azure and AWS, you'll architect high-performance data systems and drive impactful analytics. If you thrive on solving complex data challenges, this is your chance to shine. Key Responsibilities: Design scalable data lakes, warehouses, and real-time streaming architectures. Build and optimize data pipelines and Delta Lake solutions using Databricks (Spark, Workflows, SQL Analytics). Develop cloud-native data platforms on Azure (Synapse, Data...

Work Location: ~CHENNAI, TN. Skill Required: Digital : Cloud Application Migration~Data Migration Experience Range in Required Skills: 5-7 Job Description: • Data Pipeline Design: Architect and implement scalable, reliable data pipelines using Databricks, Spark, and Delta Lake • Medallion Architecture Implementation : Ingest raw data into the Bronze layer, process and clean it in the Silver layer, and aggregate/transform it for analytics in the Gold layer • Data Modeling : Design and optimize data models for efficient storage, retrieval, and analytics • Data Quality & Security: Implement data validation, quality checks, and security controls throughout the pipeline • Automation & Monitoring:...

Develop and migrate data pipelines using Spark, Python/Scala. Move Hive and Control-M jobs to Airflow/DBT. Integrate with Kafka, Solace, S3, MinIO, and deploy on Kubernetes ensuring scalable, automated, production-ready solutions. Required Candidate profile Experienced Data Engineer (6–9 yrs) with strong Spark, Python/Scala, and SQL skills. Hands-on in Airflow, DBT, Kafka, Kubernetes, and data lakehouse tools.Capable of independent delivery and migration

Purpose of the Role: We are seeking a skilled Databricks with 4 to 6 years of experience in data engineering and cloud platforms. Designs and implements scalable data architectures and solutions. The jobholder has expertise in Databricks Lakehouse, data modelling, and cloud integration, ensuring high performance, security, and reliability. Job Responsibilities: Design and implement Databricks-based data architectures to meet business requirements. Develop and optimize data pipelines using PySpark, Scala, or SQL. Establish the Databricks Lakehouse architecture for batch and streaming data. Collaborate with cross-functional teams to integrate Databricks with cloud platforms (e.g., AWS, Azure, ...

Company Overview Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milest...

Strong hands-on experience with Databricks (Workflows, DLT Pipelines, Delta Lake, Unity Catalog, Jobs, Clusters) Unity Catalog, IAM Databricks Platform Unity Catalog background,