584 Delta Lake Jobs - Page 10

About The Team/Role As a Senior Staff SWE of Data Lake House Engineering, this technical leader will be responsible for the design and implementation of the Data Lake house platform (supporting both Data Engineering and Data Lake house applications), Data Engineering pipeline productionalization (E2E data pipelines, model development, deployment, monitoring, refresh, etc), Data Lake house, and related technology, systems, and tools. This technologist will drive the technology development and architecture in these areas to ensure the platforms, systems, tools, models, and services generated from this team are technically sound with strong security, quality, reliability, usability, scalability...

As an experienced professional with 3+ years of experience in building and operating production-grade applications and services across the stack, you will be responsible for the following key responsibilities: - Demonstrating strong programming skills in Python and/or Scala and SQL to write modular, testable, and well-documented code for batch and streaming workloads. - Hands-on experience with modern data engineering stacks including: - Distributed processing using Apache Spark (Databricks preferred), PySpark/Scala. - Orchestration with Azure Data Factory or Apache Airflow; implementing event-driven patterns with Azure Functions/Logic Apps. - Working with storage & formats such as Delta Lak...

As a Data Engineer, your role involves designing and implementing scalable data architectures to efficiently store, process, and manage structured and unstructured data. You will be responsible for developing robust data pipelines, integrating disparate data sources, and ensuring high data quality and security across multiple cloud platforms. Key Responsibilities: - Design and implement scalable, cloud-agnostic Data Lake, Data LakeHouse, Data Mesh, and Data Fabric architectures. - Develop and maintain robust data pipelines using services like AWS Glue, Azure Data Factory, GCP Dataflow, Apache Spark, or Apache Airflow. - Optimize Extract, Transform, Load (ETL) processes to integrate disparate...

Role Overview: You will be working as a Solutions Architect at Koantek, collaborating with customers to design scalable data architectures using Databricks technology and services. Your role will involve building secure, highly scalable big data solutions, focusing on achieving tangible, data-driven outcomes while ensuring simplicity and operational effectiveness. With your technical expertise and business acumen, you will engage in complex technology discussions, demonstrating the value of the Databricks platform throughout the sales process. Collaborating with Account Executives, you will interact with customers" technical leaders to become a trusted advisor who delivers concrete results. ...

As a Senior Engineer in the Enterprise Data Platform team at Magna, your role is crucial in designing, constructing, and managing scalable, secure, and high-performance data infrastructure across Azure, AWS, and Databricks. You will be expected to demonstrate expertise in cloud administration, CI/CD automation, and innovation in developing ingestion patterns and reference architectures for the platform. **Key Responsibilities:** - **Platform Engineering & Administration** - Administer and optimize Azure and AWS cloud environments, including resource provisioning, access control, monitoring, and cost management. - Manage and maintain Databricks workspaces, clusters, jobs, notebooks, and user ...

Role Overview: As a Java Developer with 3-5 years of experience, you will be responsible for designing, developing, and maintaining high-performance, scalable, and secure Java-based applications. Your strong expertise in Java/J2EE, Spring Boot, Hibernate, and JPA will be essential in ensuring the quality and efficiency of the applications you work on. Your role will involve working with SQL and NoSQL databases, implementing RESTful APIs, microservices, and event-driven architectures, and leveraging cloud platforms for deployment and scaling. Key Responsibilities: - Design, develop, and maintain high-performance, scalable, and secure Java-based applications using Spring Boot, JPA, and Hiberna...

You are a passionate team member of AiEdge Analytics, a group of certified experts with extensive experience in Microsoft Technologies for Data Engineering, Data Analytics, and Business Intelligence. By transforming raw data into valuable insights, you drive innovation and empower clients through expertise in Microsoft Data Analytics and AI-driven solutions. Your focus on data security and lasting partnerships ensures the delivery of high-quality, cost-optimized solutions that positively impact various industries and society. As an Azure Data Engineer on a contract basis with AiEdge Analytics, you will: - Utilize your 5-8 years of experience in Data Engineering, particularly with Extract Tra...

Eviden, part of the Atos Group, with an annual revenue of circa 5 billion is a global leader in data-driven, trusted and sustainable digital transformation. As a next generation digital business with worldwide leading positions in digital, cloud, data, advanced computing and security, it brings deep expertise for all industries in more than 47 countries. By uniting unique high-end technologies across the full digital continuum with 47,000 world-class talents, Eviden expands the possibilities of data and technology, now and for generations to come. Roles and Responsibility The Senior Tech Lead - Databricks leads the design, development, and implementation of advanced data solutions. Has To ha...

Job Title: Big Data Architect / Databricks Architect Company Overview: At Codvo, software and people transformations go hand-in-hand. We are a global empathy-led technology services company. Product innovation and mature software engineering are part of our core DNA. Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day. We continue to expand our digital strategy, design, architecture, and product management capabilities to offer expertise, outside-the-box thinking, and measurable results. The Role We are seeking an experienced Big Data Architect with a specialization in Databricks to design the sophisticated data solutions that drive bu...

We are currently considering only candidates based in Bangalore who can attend in-person interviews after CV shortlisting. The Modern Data Engineer is responsible for designing, implementing, and maintaining scalable data architectures using cloud technologies, primarily on AWS, to support the next evolutionary stage of the Investment Process. They build robust data pipelines, optimize data storage, and access patterns, and ensure data quality while collaborating across engineering teams to deliver high-value data products Key Responsibilities • Implement and maintain data pipelines for ingestion, transformation, and delivery • Ensure data quality through validation and monitoring processes ...

Senior Data Engineer Azure, Airflow & PySpark We're hiring a Senior Data Engineer (69 years of experience) to lead the modernization of our data ecosystem from legacy platforms (Azure Synapse, SQL Server) to a next-gen Microsoft Fabric lakehouse. If you bring deep Azure expertise, strong Airflow orchestration skills, and at least 4 years of hands-on PySpark experience, this role is for you. Key Responsibilities Drive the migration of data pipelines and stored procedures into Microsoft Fabric using PySpark, Delta Lake, and OneLake. Build and orchestrate scalable workflows with Apache Airflow and Azure Data Factory (ADF). Redesign and optimize legacy ADF pipelines for seamless Fabric integrati...

At ClearTrail, you will be part of a team dedicated to developing solutions that empower those focused on ensuring the safety of individuals, locations, and communities. For over 23 years, ClearTrail has been a trusted partner of law enforcement and federal agencies worldwide, committed to safeguarding nations and enhancing lives. We are leading the way in the future of intelligence gathering through the creation of innovative artificial intelligence and machine learning-based lawful interception and communication analytics solutions aimed at addressing the world's most complex challenges. **Key Responsibilities:** - Designing and developing high-performance, scalable applications using Java...

Job Description Associate Manager, Scientific Data Engineering The Opportunity Based in Hyderabad, join a global healthcare biopharma company and be part of a 130-year legacy of success backed by ethical integrity, forward momentum, and an inspiring mission to achieve new milestones in global healthcare. Be part of an organisation driven by digital technology and data-backed approaches that support a diversified portfolio of prescription medicines, vaccines, and animal health products. Drive innovation and execution excellence. Join a team that is passionate about using data, analytics, and insights to drive decision-making and create custom software, allowing us to tackle some of the world'...

As a Solution Architect at Ralph Lauren Corporation, you will play a pivotal role in designing and implementing enterprise-scale data solutions encompassing Data Engineering, Data Quality, and Solution Architecture. Your primary responsibility will be to develop robust, scalable, and secure data solutions that not only meet business objectives but also adhere to enterprise architecture principles. Collaborating closely with the Lead Architect, platform teams, and business stakeholders, you will be instrumental in creating integrated solutions throughout the data lifecycle. Role Overview: - Design end-to-end data solutions for ingestion, transformation, storage, and delivery across cloud and ...

As a Microsoft Fabric Professional at YASH Technologies, you will leverage your 12+ years of experience in Microsoft Azure Data Engineering to drive analytical projects. Your expertise will be crucial in designing, developing, and deploying high-volume ETL pipelines using Azure, Microsoft Fabric, and Databricks for complex models. - Design, develop, and deploy high-volume ETL pipelines using Azure, Microsoft Fabric, and Databricks - Utilize hands-on experience with Azure Data Factory, Databricks, Azure Functions, Synapse Analytics, Data Lake, Delta Lake, and Azure SQL Database for managing and processing large-scale data integrations - Optimize Databricks clusters and manage workflows to ens...

Role Overview: As a Data Engineer at our company, you will be responsible for building and maintaining secure, scalable data pipelines using Databricks and Azure. Your role will involve handling ingestion from diverse sources such as files, APIs, and streaming data, performing data transformation, and ensuring quality validation. Additionally, you will collaborate closely with subsystem data science and product teams to ensure ML readiness. Key Responsibilities: - Technical proficiency in Notebooks (SQL, Python), Delta Lake, Unity Catalog, ADLS/S3, job orchestration, APIs, structured logging, and IaC (Terraform). - Delivery expertise in trunk-based development, TDD, Git, and CI/CD for notebo...

Role Overview: As an AVP - Databricks, you will be responsible for leading and managing Databricks-based project delivery to ensure that all solutions meet client requirements, best practices, and industry standards. Your role will involve serving as a subject matter expert (SME) on Databricks, providing guidance to teams on architecture, implementation, and optimization. Collaboration with architects and engineers to design optimal solutions for data processing, analytics, and machine learning workloads will be a key aspect of your responsibilities. Additionally, you will act as the primary point of contact for clients, ensuring alignment between business requirements and technical delivery...

As a Cloud Platform Administrator in Bengaluru, you will be responsible for various key tasks related to user and access management, cluster and resource management, job management, monitoring and troubleshooting, platform maintenance, collaboration, and cost optimization. Your primary responsibilities will include: - Creating users, setting roles, and managing access controls for the cloud platform. - Provisioning clusters, monitoring resource usage, and optimizing performance for efficient operations. - Scheduling, managing, and optimizing jobs and workflows. Furthermore, you will play a vital role in: - Monitoring system performance, troubleshooting issues, managing updates, backups, inte...

Description We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. Job summary As a Software Engineer IIat JPMorgan Chase within the Corporate technology, you design and deliver trusted technology products in a secure, stable, and scalable way. You are responsible for implementing critical solutions across multiple technical areas to support the firm's business objectives. Job responsibilities Execute software solutions, design, development, and technical troubleshooting to solve complex problems. Create secure, high-quality production code and maintain algorithms that run synchronously with systems. Produce architecture and design a...

Inviting applications for the role of Senior Principal Consultant- Databricks Platform Architect! We are seeking a highly skilled and experienced Databricks Platform Architect to lead the design, implementation, and optimization of our Databricks platform. The ideal candidate will have a strong background in cloud-based data processing systems, data warehousing, and big data technologies. They will work closely with our data engineering team to ensure that our Databricks platform is optimized for performance, scalability, and reliability. Responsibilities Architect and design end-to-end Databricks-based solutions covering ingestion, transformation, governance, and consumption. Lead platform ...

Build and implement demand planning and forecasting models using statistical and machine learning techniques ARIMA, Prophet, LSTM Work extensively on Databricks (PySpark, notebooks, Delta Lake, ETL pipelines, cluster management) Required Candidate profile Develop validate ML models for business forecasting planning Perform data modeling, SQL querying, and ETL development Collaborate with cross-functional stakeholders and present insights effectively

**Job Description:** As a Python Azure Data Factory (ADF) professional, your role involves working with technologies such as Azure Databricks, PySpark, Delta Lake, ETL/ELT, data pipelines, and data lakehouse architecture. **Key Responsibilities:** - **Technical Delivery:** - Develop robust ETL/ELT workflows using Informatica to ingest, transform, and load data into our data warehouse, Azure Synapse Analytics. - Conduct exploratory data analysis (EDA) to validate pipeline outputs and ensure data accuracy. - Ensure data quality by implementing proactive checks and monitoring mechanisms. - Write and optimize complex SQL queries to support analytical and reporting team requirements. - Collaborat...

As a Databricks Architect at NTT DATA, your role involves leading the design and implementation of scalable, secure, and high-performance data solutions using the Databricks Lakehouse Platform. You will define architectural standards, guide engineering teams, and collaborate with stakeholders to align data strategies with business goals. Your main focus will be leveraging Databricks unified analytics capabilities to build enterprise-grade data platforms supporting advanced analytics, machine learning, and real-time data processing. Key Responsibilities: - Define and own the end-to-end architecture of data platforms on Databricks, covering ingestion, transformation, storage, and consumption l...

You will be working as a Databricks Developer with 3-6 years of experience, located in India. Joining the data engineering and AI innovation team, your main responsibilities will include: - Developing scalable data pipelines using Databricks and Apache Spark - Implementing AI/ML workflows with tools like MLflow and AutoML - Collaborating with data scientists to deploy models into production - Performing ETL development, data transformation, and model training pipelines - Managing Delta Lake architecture - Working closely with cross-functional teams to ensure data quality and governance Qualification Required: - Bachelor's degree in Computer Science or a related field - Proven experience work...

As a Data Engineer at our company, you will be responsible for designing, developing, and maintaining scalable data pipelines using Spark (PySpark or Spark with Scala). Your key responsibilities will include: - Constructing data ingestion and transformation frameworks for both structured and unstructured data sources. - Collaborating with data analysts, data scientists, and business stakeholders to comprehend requirements and deliver reliable data solutions. - Handling large volumes of data while ensuring quality, integrity, and consistency. - Optimizing data workflows for enhanced performance, scalability, and cost efficiency on cloud platforms such as AWS, Azure, or GCP. - Implementing dat...