318 Delta Lake Jobs - Page 4

You are a skilled Data Engineer with expertise in the Azure Data Platform, specifically Azure Databricks, PySpark, Azure Data Factory, and Synapse Analytics. Your primary responsibility will be to design, develop, and optimize large-scale data pipelines and analytics solutions. Your key responsibilities will include developing and optimizing data pipelines using Azure Data Factory for ingestion into Azure Databricks, implementing PySpark transformations in Databricks with a focus on performance tuning, building and maintaining data layers, and ensuring efficient export strategies for reporting and analytics. You will also work with Azure Synapse for advanced querying, transformations, and re...

NTT DATA strives to hire exceptional, innovative, and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking an AI Solution Architect to join our team in Bangalore, Karnataka (IN-KA), India (IN). Role: - AI Solution Architect Experience: - 7+ Years Notice Period: - 30-60 Days Project Overview: We are seeking a seasoned AI Architect with strong experience in Generative AI and Large Language Models (LLMs) including OpenAI, Claude, and Gemini to lead the design, orchestration, and deployment of intelligent solutions across complex use cases. You will architect conversational syste...

As a Databricks Architect with 5-8 years of experience, you will be responsible for designing, implementing, and optimizing scalable data architectures leveraging Databricks on public cloud platforms. Your role will involve designing enterprise-grade solutions for batch and real-time data analytics, with hands-on expertise in Databricks, Spark, and cloud-based data services. Your key responsibilities will include designing and developing end-to-end data architectures and pipelines with Databricks, optimizing Databricks performance through efficient data processing, job tuning, and resource management, implementing data security, governance, and best practices, collaborating with stakeholders...

NTT DATA is looking for an Azure Databricks Developer to join their team in Bengaluru, Karnataka, India. As an Azure Databricks Developer, you will be responsible for advanced experience with Databricks, including Databricks SQL, DataFrames, and Spark using PySpark or Scala. You should have a deep understanding of Spark architecture and optimization strategies, such as tuning Spark SQL configurations, managing data partitions, handling data skew, and leveraging broadcast joins. Your role will also involve proficiency in building and optimizing large-scale data pipelines for ETL/ELT processes using Databricks and familiarity with Delta Lake and data lake architectures. Strong programming skil...

About CoPoint AI: CoPoint AI is a specialized consulting firm that focuses on transforming businesses through process improvement, data insights, and technology-driven innovation. Leveraging AI technologies, Microsoft cloud platforms, and modern web development frameworks, we deliver intelligent, scalable solutions that drive measurable impact for our clients. Our team partners across industries to design and deploy solutions that streamline operations, enhance customer experiences, and enable data-driven growth. Our Vision: We transform businesses through process improvement and data insights leveraging AI on the Microsoft stack. Our Values: - Be Purposeful: Think Straight, Communicate, Alw...

Ready to build the future with AI? At Genpact, we don't just keep up with technology we set the pace. AI and digital innovation are redefining industries, and were leading the charge. Genpacts AI Gigafactory , our industry-first accelerator, is an example of how were scaling advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. From large-scale models to agentic AI , our breakthrough solutions tackle companies most complex challenges. If you thrive in a fast-moving, innovation-driven environment, love building and deploying cutting-edge AI solutions, and want to push the boundaries of whats possible, this is your moment. Genpact (NYSE: G)...

We would be looking at some of the folks who have both expert skills in Talend and Pyspark. Engineers Skillset: 1. Talend Studio 2. PySpark 3. AWS Ecosystem (S3, Glue, CloudWatch, SSM, IAM etc) 4. Redshift 5. Aurora 6. Terradata Talend Skillset: 1. Good hands-on experience with Talend Studio and Talend Management Console (TMC) 2. In-depth understanding of Joblets, PreJobs, PostJobs, and SubJobs, with the ability to go through complex designs and understand Talend's data flow and control flow logic 3. Proficient in working with Talend components for S3, Redshift, tDBInput, tMap etc and various Java-based components Pyspark: 4. Solid knowledge of PySpark with the ability to compare, analyze an...

6-month extendable Hybrid role in Indore, Noida, Bangalore, Hyderabad, Gurgaon, or Pune. Requires to go 4 days a month in office 8+ yrs in ETL & pipeline expert skilled in Azure, Spark, SQL, Python/Scala, Synapse, Snowflake, and data architecture.

(A) should have hands-on experience in designing and implementing data solutions using Azure data services, including Azure SQL Database, Azure Data Lake Storage, and Azure Synapse Analytics. (B) Experience in setting up, managing Data Lake.

Role & responsibilities Develop and maintain scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake). Design and optimize data models in AWS Redshift for performance and scalability. Manage Redshift clusters and EC2-based deployments, ensuring reliability and cost efficiency. Integrate data from diverse sources (structured/unstructured) into centralized data platforms. Implement data quality checks, monitoring, and logging across pipelines. Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets. Required Skills & Experience: 36 years of experience in data engineering. Strong expertise in Databricks (Spark, Delta Lake, notebooks, job o...

Why Join Us Are you inspired to grow your career at one of Indias Top 25 Best Workplaces in IT industry Do you want to do the best work of your life at one of the fastest growing IT services companies Do you aspire to thrive in an award-winning work culture that values your talent and career aspirations Its happening right here at Iris Software. About Iris Software At Iris Software, our vision is to be our clients most trusted technology partner, and the first choice for the industrys top professionals to realize their full potential. With over 4,300 associates across India, U.S.A, and Canada, we help our enterprise clients thrive with technology-enabled transformation across financial servi...

We are looking for a Senior Data Engineer to be based out of our Chennai, Bangalore & Hyderabad offices. This role involves a combination of hands-on contribution, customer engagement, and technical team management. As a Senior Data Engineer, you will Design and build solutions for near real-time stream processing as well as batch processing on the Big Data platform. Role & responsibilities Minimum 6+ years of Professional experience with 4+ years of Big Data project experience Strong in Pyspark, Python, PLSQL, SQL Desirable to have ETL with batch and streaming (Kinesis). Build the solution for optimal extraction, transformation, and loading of data from a wide variety of data sources using ...

Job Objective: We are looking for Azure Data and Analytics Solutions Architect focusing on Data engineering to join our growing team of Data and Analytics specialized Consulting organization. This plays a key role in Digital journey of our clients spread globally with focus on Data & AI at the core of it. This role is in our setup to drive the exploratory sessions to find the opportunities, arrive at the Data and Analytics solutions that could involve technical, process and people. Once the solutions are found, this role will orchestrate, architect and technically drive the solutions and do the hands on when and where needed. This role also will mentor, train and enable the consultants at va...

Were looking for a Data Engineer to build and optimize PySpark-based data pipelines in our Python + Azure ecosystem. Youll work on: Designing and maintaining data pipelines (PySpark, Azure Databricks, ADF, Synapse, ADLS) Optimizing ETL workflows for scalability and performance Ensuring data quality, governance, and security Supporting AI/ML teams with reliable data flows You should have: ?? 3+ years of experience in Python & PySpark ?? Strong knowledge of Azure data services and SQL ?? Experience with big data and CI/CD Bonus: Kafka, Delta Lake, Lakehouse, ML/AI pipelines, Terraform Please fill the below form as it is mandatory (applications without form will be automatically rejected): http...

Greetings from NAM Info!!! Please go through the job description. If you are interested in this opportunity, please reply with the following information to praveen@nam-it.com : Full Name (as in Adhar): Current Location: Expected CTC: Present CTC: Least Notice Period(Last Working Day If any): PAN Number: Job Title: Azure Data Engineer Company: Thermax Limited Location: Pune Job Type: Full-Time Employee of Thermax Experience Level: Senior (7+ years) Interview Process: 2 - 3 Rounds virtual Job Summary: Thermax is seeking a Senior Azure Databricks Engineer to lead the design, development, and deployment of scalable data solutions on the Azure cloud platform. The ideal candidate will have deep ex...

As a Director / Associate Director in Cloud Engineering (Azure Stack), you will be responsible for leading multiple client engagements, driving end-to-end project delivery, and managing high-performing engineering teams specializing in Databricks. Your role will involve hands-on technical expertise, project and people management skills, and a proven track record of delivering large-scale data engineering solutions in cloud-native environments. With at least 14 years of experience in data engineering, including 3+ years in leadership or director-level roles, you will be expected to have a strong background in Databricks, Delta Lake, and cloud data architecture. Your responsibilities will incl...

As a Senior Data Architect, you will play a crucial role in designing and implementing scalable, secure, and high-performance Big Data architectures using Databricks, Apache Spark, and cloud-native services. Your expertise will be essential in leading the end-to-end data architecture lifecycle, from requirements gathering to deployment and optimization. Additionally, you will design repeatable and reusable data ingestion pipelines for various ERP source systems like SAP, Salesforce, HR, Factory, and Marketing systems. Collaboration with cross-functional teams to integrate SAP data sources into modern data platforms will be a key aspect of your role, along with driving cloud cost optimization...

You have 5+ years of experience in designing and building data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. You possess hands-on expertise with streaming and messaging systems like Apache Kafka, Confluent Cloud, RabbitMQ, or Azure Event Hub. Your experience includes creating producers, consumers, and topics, and integrating them into downstream processing. You have a deep understanding of relational databases and Change Data Capture (CDC). You are proficient in SQL Server, Oracle, or other RDBMSs, and have experience capturing change events using tools like Debezium or native CDC tools, transforming them for downstream consumption. Your proficiency extends to ...

You will be responsible for building and maintaining secure, scalable data pipelines using Databricks and Azure. This includes handling ingestion from various sources such as files, APIs, and streaming, performing data transformation, and ensuring quality validation. Collaboration with subsystem data science and product teams will be essential to prepare for machine learning readiness. Your technical expertise should cover working with Notebooks (SQL, Python), Delta Lake, Unity Catalog, ADLS/S3, job orchestration, APIs, structured logging, and Infrastructure as Code (IaC) tools like Terraform. Delivery practices like Trunk-based development, Test-Driven Development (TDD), Git, and Continuous...

You should have at least 5 years of experience working as a Data Engineer. Your expertise should include a strong background in Azure Cloud services and proficiency in tools such as Azure Databricks, PySpark, and Delta Lake. It is essential to have solid experience in Python and FastAPI for API development, as well as familiarity with Azure Functions for serverless API deployments. Experience in managing ETL pipelines using Apache Airflow is also required. Hands-on experience with databases like PostgreSQL and MongoDB is necessary. Strong SQL skills and the ability to work with large datasets are key for this role.,

About the Company At Tide, we are dedicated to creating a business management platform that aims to streamline operations for small businesses, enabling them to save valuable time and resources. Our services include offering business accounts, banking solutions, as well as a range of integrated administrative tools spanning from invoicing to accounting. Established in 2017, Tide has garnered a user base of over 1 million small businesses globally, catering to SMEs in the UK, India, and Germany. Headquartered in central London, we also have offices in Sofia, Hyderabad, Delhi, Berlin, and Belgrade, with a team of more than 2,000 employees. Tide is on a trajectory of rapid growth, continuously ...

The Infor Data Services team at Infor is looking for a Software Engineer to develop customer-focused data solutions. As a Software Engineer, you will primarily focus on building backend systems for Infor Datamesh, a platform that provides scalable and robust data solutions for enterprise customers. Your responsibilities will include designing, developing, and maintaining backend services and microservices using Python, Docker, and Kafka. You will also be integrating these services with various databases and external storage solutions to ensure high performance and scalability in alignment with Infor's data management strategy. Your essential duties will involve developing and maintaining dat...

You are an experienced Data Engineer proficient in Databricks, PySpark, and Python, seeking to join a healthcare data-focused team. Your responsibilities will include designing and maintaining scalable data pipelines and solutions for complex healthcare datasets, utilizing Azure Cloud and advanced distributed data processing frameworks. Strong client-facing communication skills are crucial as you will engage directly with US-based stakeholders. Your key responsibilities will involve designing, developing, and maintaining large-scale data processing systems using Databricks and PySpark. You will build and optimize robust, scalable data pipelines for data ingestion, cleaning, transformation, a...

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. As a Software Engineer, you will be responsible for executing software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches. Your primary focus will be on creating secure and high-quality production code, maintaining algorithms that run synchronously with appropriate systems, and producing architecture and design artifacts for complex applications. You will also be accountable for ensuring design constraints are met by software code development. In this role, you will gather, analyze, synthesize, and dev...

At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all. Our Technology team builds innovative digital solutions rapidly and at scale to deliver the next generation of Financial and Non-Financial services across the globe. The Position is a senior technical, hands-on delivery role, requiring the knowledge of data engineering, cloud infrastructure and platform engineering, platform operations, and p...