318 Delta Lake Jobs - Page 2

Role Summary & Role Description: Technical Manager with specific Oracle, PL/SQL and design, develop, and optimize data workflows on the Databricks platform. The ideal candidate will have deep expertise in Apache Spark, PySpark, Python, job orchestration, and CI/CD integration to support scalable data engineering and analytics solutions. Analyzes, designs, develops and maintains software applications to support business units. Expected to spend 80% of the time on hands-on development, design and architecture and remaining 20% on guiding the team on technology and removing other impediments Capital Markets Projects experience preferred Provides advanced technical expertise in analyzing, design...

Role: Data Engineer Location: Mumbai, 5th Floor, 3, North Avenue, Maker Maxity, Bandra Kurla Complex, Contract: 12+months, likely long term Work Mode: Hybrid Notice Period: Immediate - 15days Strong Experience in PySpark Hands-on expertise in building scalable data pipelines using PySpark. Proficiency in using Spark SQL, DataFrame, and RDD APIs to implement complex business logic. Proficient Programming Skills Solid coding skills in Python (preferred), with strong fundamentals in data structures, algorithms, and software engineering principles. Data Pipeline Development Proven experience designing, developing, and maintaining batch and streaming data pipelines. Understanding of ETL/ELT proce...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and the desire to create lasting value for clients. We serve leading enterprises, including the Fortune Global 500, with deep business knowledge, digital operations services, and expertise in data, technology, and AI. We are inviting applications for the position of Vice President, Databricks Squad Delivery Lead. The successful candidate will be responsible for overseeing the end-to-end delivery of Databricks-based solutions for clients, ensuring successful implementation, optimization, and scaling of big data and analytics solutions. This role involves dri...

As a Data Solutions Specialist at Quantiphi, you will be part of a global and diverse culture that values transparency, diversity, integrity, learning, and growth. We take pride in fostering an environment that encourages innovation and excellence, not only in your professional endeavors but also in your personal life. Key Responsibilities: - Utilize your 3+ years of hands-on experience to deliver data solutions that drive business outcomes. - Develop data pipelines using PySpark within Databricks implementations. - Work with Databricks Workspaces, Notebooks, Delta Lake, and APIs to streamline data processes. - Utilize your expertise in Python, Scala, and advanced SQL for effective data mani...

As a Databricks AWS/Azure/GCP Architect at Koantek based in Mumbai, you will play a crucial role in building secure and highly scalable big data solutions that drive tangible, data-driven outcomes while emphasizing simplicity and operational efficiency. Collaborating with teammates, product teams, and cross-functional project teams, you will lead the adoption and integration of the Databricks Lakehouse Platform into the enterprise ecosystem and AWS/Azure/GCP architecture. Your responsibilities will include implementing securely architected big data solutions that are operationally reliable, performant, and aligned with strategic initiatives. Your expertise should include an expert-level know...

Role: Assistant Manager - Data Engineering Experience: 4 to 8 years Location: Bengaluru, Karnataka , India (BLR) Job Description: Communication and leadership experience, with experience initiating and driving projects. Experience with data sets, Hadoop, and data modernisation tools. Experience in SQL or similar languages and in any one cloud platform AWS/Azure/GCP. Development experience in at least one object-oriented language (Python, Java, etc.). BA/BS in Computer Science, Math, Physics, or other technical fields. Job Responsibility: Data Engineering & Architecture Design and implement scalable and optimized data pipelines on Databricks using Delta Lake, PySpark, and SQL. Develop ETL/ELT...

Databricks Developer (5–7 yrs exp) skilled in Apache Spark, Design & Implementation, PySpark, Delta Lake, and cloud platforms (Azure/AWS). Must excel in data pipelines, ETL, governance, and performance tuning. Unity Catalog, ML, and CI/CD experience.

Key Responsibilities Design, build, and maintain a robust data platform on Databricks. Develop efficient ETL/ELT pipelines to ingest data from SQL Server, MongoDB, and InfluxDB into Databricks Delta Lake. Implement real-time and batch data ingestion strategies using tools such as Kafka, Azure Event Hubs, or similar. Optimize data storage and processing for performance, scalability, and cost-effectiveness. Build data models to support BI, advanced analytics, and machine learning use cases. Collaborate with stakeholders (Data Scientists, Analysts, Product Teams) to define data requirements. Ensure data quality, governance, and security across the platform. Monitor, troubleshoot, and enhance da...

As a Senior Databricks Developer at Newscape Consulting, you will play a crucial role in our data engineering team, focusing on building scalable and efficient data pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native services (Azure/AWS/GCP). Your responsibilities will include collaborating closely with data architects, data scientists, and business stakeholders to deliver high-performance, production-grade solutions that enhance user experience and productivity in the healthcare industry. Your key skills should include a strong hands-on experience with Databricks including Workspace, Jobs, DLT, Repos, and Unity Catalog. Proficiency in PySpark, Spark SQL, and optionally Sc...

As a Senior Data Engineer & Architect with 10-12 years of experience, you will be responsible for designing and implementing enterprise-grade Data Lake solutions using AWS technologies such as S3, Glue, and Lake Formation. Your expertise in building Data Lakes and proficiency with AWS tools like S3, EC2, Redshift, Athena, and Airflow will be essential in optimizing cloud infrastructure for performance, scalability, and cost-effectiveness. You will be required to define data architecture patterns, best practices, and frameworks for handling large-scale data ingestion, storage, compute, and processing. Developing and maintaining ETL pipelines using tools like AWS Glue, creating robust Data War...

As an Azure Databricks Lead, you will be responsible for designing, implementing, and optimizing data solutions using Azure Databricks. Your expertise will be crucial in creating robust data pipelines, ensuring data quality, and improving overall performance. Collaborating with cross-functional teams, you will deliver high-quality solutions that meet business requirements. You will design and develop scalable data processing pipelines using Azure Databricks and PySpark. Implementing ETL processes, you will ingest, transform, and load data from various sources. Collaboration with data engineers and architects will ensure efficient data movement and transformation. Establishing data quality ch...

Genpact is a global professional services and solutions firm with over 125,000 employees in 30+ countries, driven by curiosity, agility, and a commitment to creating value for clients. We serve leading enterprises worldwide, leveraging our expertise in digital operations, data, technology, and AI. We are seeking a Lead Consultant- Databricks Developer to solve cutting-edge problems and meet functional and non-functional requirements. As a Databricks Developer, you will work closely with architects and lead engineers to design solutions and stay abreast of industry trends and standards. Responsibilities: - Stay updated on new technologies for potential application in service offerings. - Coll...

Lead design & implementation of scalable data pipelines using Snowflake & Databricks. Drive data architecture, governance. Build ETL/ELT, optimize models, mentor team, ensure security, compliance.strong Snowflake, Databricks, SQL, Python Required Candidate profile Experienced Data Analytics Lead skilled in Snowflake, Databricks, SQL, Python. Proven leader in designing scalable pipelines, data governance, ETL/ELT, and team mentoring.

Location: Hyderabad, Telangana. Mode of work: Hybrid Kindly Send your Resume to 9361912009. Fullstack with Data exposure for Catalog ====== We are seeking an experienced Full Stack Developer with strong expertise in Databricks, SQL, and modern web technologies. The ideal candidate will be responsible for designing, developing, and maintaining scalable applications that integrate data engineering, analytics, and interactive user interfaces. This role requires a blend of front-end and back-end development skills, coupled with hands-on experience in Databricks and large-scale data management. Key Responsibilities: Design and develop full-stack applications using [React/Angular/Vue] for front-en...

Greetings from TCS!! TCS is Hiring for Senior Data Consultant Interview Mode: Virtual Required Experience: 10-16 years Work location: PAN INDIA Must-Have Databricks (including PySpark, Delta Lake, MLflow) Snowflake (data warehousing, performance tuning, data sharing) Data Lake architecture Cloud platforms: Azure, AWS, GCP CI/CD and DataOps practices Legacy system migration and modernization strategies Good-to-Have Exposure to GenAI/LLM integration Experience with streaming data (Kafka, Spark Streaming) Knowledge of data governance and cataloging tools (Unity Catalog, Purview) Familiarity with BI tools and dashboarding

Technical Skills progressive experience across both traditional and cloud architectures and ecosystems Experience engaging with clients in a consultative mode and demonstrated thought leader - guiding and advising clients on data and solution architecture Experience in architecting and implementing data solutions covering Data Lakes, Data Warehouses, BI Reporting and Analytic solutions across industries with a focus on Retail CPG and Manufacturing industries. Understanding of Data mesh, Lake house and other key solution constructs for architecting data solutions and platforms Expertise in designing and implementing data models using methodologies such as Data Vault 2.0, Dimensional modelling...

To transform data into a format that can be easily analyzed, you will be responsible for developing, maintaining, and testing infrastructures for data generation. Working closely with data scientists, you will play a key role in architecting solutions that enable data scientists to perform their tasks efficiently. Your main responsibilities will include: - Developing and maintaining scalable data pipelines and building new API integrations to support increasing data volume and complexity - Connecting offline and online data to enhance customer behavior understanding for personalization - Cleansing data, improving data quality, and preparing data for analysis - Processing data through cleansi...

Job Title: Senior Data Engineer Basic Function The Senior Data Engineer should be an expert familiar with all areas of data warehousing technical components (e.g., ETL, Reporting, Data Model), connected infrastructure, and their integrations. The ideal candidate will be responsible for developing the overall architecture and high-level design of the data schema environment. The candidate must have extensive experience with Star Schemas, Dimensional Models, and Data Marts. The individual is expected to build efficient, flexible, extensible, and scalable ETL design and mappings. Excellent written and verbal communication skills are required as the candidate will work very closely with diverse ...

Your journey at Crowe starts here with an opportunity to build a meaningful and rewarding career. At Crowe, you will enjoy real flexibility to balance work with life moments while being trusted to deliver results and make a significant impact. You will be embraced for who you are, your well-being will be cared for, and your career will be nurtured. Equitable access to opportunities for career growth and leadership is available to everyone. With a history spanning over 80 years, delivering excellent service through innovation is ingrained in our DNA across audit, tax, and consulting groups. We continuously invest in innovative ideas like AI-enabled insights and technology-powered solutions to...

You should have 5+ years of experience with skills in DataBricks, Delta Lake, pyspark or scala spark, unity catalog. Good to have skills in Azure/AWS Cloud. You will be responsible for ingesting and transforming batch and streaming data on the Databricks Lakehouse Platform. Excellent communication skills are required. Your responsibilities will include overseeing and supporting processes by reviewing daily transactions, reviewing performance dashboards, supporting the team in improving performance parameters, recording and tracking all queries received, ensuring standard processes are followed, resolving client queries within defined SLAs, developing understanding of processes/products for t...

You should have at least 5 years of hands-on experience in data engineering/ETL using Databricks on AWS/Azure cloud infrastructure and functions. Additionally, you must possess a minimum of 3 years of experience in Power BI and Data Warehousing, with the ability to perform root cause analysis on internal and external data and processes to address specific business questions and identify areas for improvement. It is preferred that you have experience with AWS services such as S3, Athena, Glue, and Lambda. A deep understanding of data warehousing concepts including Dimensional (star-schema), SCD2, Data Vault, Denormalized, OBT is essential for implementing highly performant data ingestion pipe...

The ideal candidate will be responsible for designing and implementing streaming data pipelines that integrate Kafka with Databricks using Structured Streaming. You will also be tasked with architecting and maintaining the Medallion Architecture, which consists of well-defined Bronze, Silver, and Gold layers. Additionally, you will need to implement efficient data ingestion processes using Databricks Autoloader for high-throughput data loads. You will work with large volumes of structured and unstructured data to ensure high availability and performance, applying performance tuning techniques like partitioning, caching, and cluster resource optimization. Collaboration with cross-functional t...

As a skilled Data Engineer with 3 to 5 years of experience, you will be responsible for designing, building, and maintaining robust ETL/ELT pipelines using Databricks (PySpark/SQL). You will develop and optimize data workflows and pipelines on Delta Lake and Databricks Lakehouse architecture. Your role will involve integrating data from multiple sources to ensure data quality, reliability, and performance. Collaboration with data scientists, analysts, and business stakeholders will be essential to translate requirements into scalable data solutions. Monitoring and troubleshooting production data pipelines for performance and cost optimization will also be part of your responsibilities. Worki...

You are a Spark & Delta Lake Specialist with 8+ years of experience and proficiency in Python, Spark (PySpark), SQL, and Delta Lake. Your role involves understanding Spark core concepts such as RDDs, DataFrames, DataSets, SparkSQL, and Spark Streaming. You have experience with Spark optimization techniques and deep knowledge of Delta Lake features like time travel, schema evolution, and data partitioning. Your responsibilities include designing and implementing data pipelines using Spark and Delta Lake as the data storage layer. You are expected to have proficiency in Python, Scala, Java for Spark development, and integration with ETL processes. Additionally, you have knowledge of data inges...

The ML Platform Specialist role involves designing, implementing, and maintaining machine learning infrastructure and workflows on the Databricks Lakehouse Platform. Your primary responsibility will be to ensure the successful deployment, monitoring, and scaling of machine learning models across the organization. You will design and implement scalable ML infrastructure on the Databricks Lakehouse Platform, develop CI/CD pipelines for machine learning models, and create automated testing and validation processes using Databricks MLflow. Additionally, you will be responsible for managing model monitoring systems, collaborating with various teams to optimize machine learning workflows, and main...