3649 Azure Databricks Jobs - Page 21

Job description Job Summary: We are seeking a talented Data Engineer with strong expertise inAzure , Databricks, specifically in Unity Catalog, PySpark, and SQL, to join our data team. Youll play a key role in building secure, scalable data pipelines and implementing robust data governance strategies using Unity Catalog. Key Responsibilities: Design and implement ETL/ELT pipelines using Databricks and PySpark. Work with Unity Catalog to manage data governance, access controls, lineage, and auditing across data assets. Develop high-performance SQL queries and optimize Spark jobs. Collaborate with data scientists, analysts, and business stakeholders to understand data needs. Ensure data qualit...

As a Data Engineer, you will be responsible for the following key responsibilities: - Proficient in Python, including popular Python packages such as Pandas and NumPy, as well as SQL. - Strong background in distributed data processing and storage, such as Apache, Spark, and Hadoop. - Ability to work with large-scale data (TBs of data) and skills in data engineering to model data and create production-ready ETL pipelines. - Development experience with at least one cloud platform, with a high preference for Azure, and familiarity with AWS and GCP. - Knowledge of data lake and data lake house patterns, ETL performance tuning, and cost optimization. - Understanding of data structures, algorithms...

About Position: We are seeking Data Engineer with hands on experience in Python, Pyspark, Azure Databricks, ADF, etc., Role: Data Engineer Location: All Persistent Location Experience: 4 to 6 years Job Type: Full Time Employment What You'll Do: Technical Design and implement client requirements. Build data pipeline. Perform transformations using pyspark in data bricks. Create automated workflows with the help of triggers, Scheduled Jobs in Airflow. Design and development of Airflow dags to orchestrate data processing jobs. Design and development for ensuring timely notifications through email alerts in the event of job failures or critical system issues. Developing and implementing code to w...

We are looking for an experienced Data Engineer with strong hands-on expertise in PySpark , Azure Databricks , and Fabric development. The ideal candidate should have solid development experience (not support) in building and optimizing scalable data solutions within a fast-paced financial domain environment. Roles and Responsibilities Design, develop, and implement scalable data pipelines using PySpark , Azure Databricks , and Fabric . Work extensively on SQL for data transformations, optimization, and performance tuning. Collaborate with cross-functional teams to translate business requirements into efficient data engineering solutions. Ensure data quality, integrity, and compliance with o...

Technology Stack Data engineer with experience in working with Big data technologies e.g. spark/hadoop Design and implement scalable cloud infrastructure solutions using Google Cloud Platform services such Cloud Storage, BigQuery , Cloud Data flow, Cloud run etc Strong scripting and programming skills on python

Job Title Databricks Engineer- CHRTN Job Description Design, build, and maintain scalable and efficient ETL/ELT pipelines in Azure Databricks to process structured, semi-structured, and unstructured insurance data from multiple internal and external sources. • Collaborate with data architects, modellers, analysts, and business stakeholders to gather data requirements and deliver fit-for-purpose data assets that support analytics, regulatory, and operational needs. • Develop, test, and optimize data transformation routines, batch and streaming solutions (leveraging tools such as Azure Data Factory, Data Lake Storage Gen2, Azure Event Hubs, and Kafka) to ensure timely and accurate data deliver...

Role & responsibilities - Design and implement robust data pipelines using Azure Data Factory, Azure Databricks, and Azure SQL Database. - Work with large datasets to ingest, process, and analyze data from various sources. - Utilize Azure services for data storage and processing, ensuring optimal performance and scalability. - Collaborate with data scientists and business analysts to understand data needs and deliver solutions that meet requirements. - Ensure data quality, security, and compliance by implementing best practices in data governance. - Troubleshoot and resolve data-related issues, providing support for end-users and stakeholders. - Stay up-to-date with the latest Azure technolo...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications are...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Databricks Good to have skills : Microsoft Azure Analytics Services Minimum 2 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, an...

About The Role Skill required: Data Management - Microsoft Azure Data Factory Designation: Data Eng, Mgmt & Governance Assoc Mgr Qualifications: BE/BTech Years of Experience: 10 to 14 years What would you do? Data & AIDemonstrate skills in building data orchestration and pipeline using Azure Data Engineering What are we looking for? Microsoft Azure Data Engineering Alteryx Ability to perform under pressure Problem-solving skills Prioritization of workload Strong analytical skills Ability to manage multiple stakeholders Roles and Responsibilities: In this role you are required to do analysis and solving of moderately complex problems Typically creates new solutions, leveraging and, where need...

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way youd like, where youll be supported and inspired bya collaborative community of colleagues around the world, and where youll be able to reimagine whats possible. Join us and help the worlds leading organizationsunlock the value of technology and build a more sustainable, more inclusive world. Your Role Experience with Azure Data Bricks, Data Factory Experience with Azure Data components such as Azure SQL Database, Azure SQL Warehouse, SYNAPSE Analytics Experience in Python/Pyspark/Scala/Hive Programming. Experience with Azure Databricks/ADB Experience with building CI/CD pipelines in Data ...

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client"s challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role Advise CIOs & CDOs on the SAP Technology Transformation Strategy & Architecture & Roadmap. Leverage Clean Core Principles & SAP Best practices to propose right-fit technology architecture Develop solution architecture leveraging BTP ...

Role & responsibilities Data Engineering & Pipeline Management Design, manage, monitor, and maintain data pipelines and ETL processes using PySpark, Azure DataBricks, SQL Server Studio and Azure Data Factory. Ensure data flows are reliable, scalable, and optimized for performance. Validate data processes through automated testing and monitoring. Continuous Integration, Deployment & Reliability Perform deployments using CI/CD pipelines in Azure DevOps, aligned with team practices. Ensure continuous availability, reliability, and scalability of connected systems. Support performance, uptime, disaster recovery, and capacity growth. Operational Support & Collaboration Serve as Level 3 support fo...

Role Overview We are looking for a Data Product Owner to take ownership of high-impact data and AI products that generate measurable business results. This role bridges business and technology by ensuring that use cases built on our Common Data Platform (CDP) directly improve elevator service operations, sales, and customer outcomes. The ideal candidate combines strong business acumen with the ability to translate data opportunities into tangible value and adoption. Role & responsibilities Define and own the vision, roadmap, and value case for assigned data products Translate business goals into prioritized product features and ensure alignment with strategic objectives Collaborate with Data...

Azure Data Factory: - Develop Azure Data Factory Objects - ADF pipeline, configuration, parameters, variables, Integration services runtime - Hands-on knowledge of ADF activities(such as Copy, SP, lkp etc) and DataFlows - ADF data Ingestion and Integration with other services Azure Databricks: - Experience in Big Data components such as Kafka, Spark SQL, Dataframes, HIVE DB etc implemented using Azure Data Bricks would be preferred. - Azure Databricks integration with other services - Read and write data in Azure Databricks - Best practices in Azure Databricks Synapse Analytics: - Import data into Azure Synapse Analytics with and without using PolyBase - Implement a Data Warehouse with Azure...

Job Summary: We are looking for an experienced ETL Lead to manage and deliver enterprise-grade data integration solutions using Azure Data Factory (ADF), SSIS, SQL Querying, Azure SQL, Azure Data Lake, and preferably Azure Databricks. The role includes leading a team, building scalable ETL pipelines, and ensuring data quality and performance through efficient CI/CD practices. Key Responsibilities: Lead a team of engineers and manage ETL project lifecycles. Design, develop, and optimize ETL workflows using ADF and SSIS. Write complex SQL queries and perform performance tuning. Integrate data from varied sources into Azure SQL and Data Lake. Implement CI/CD pipelines for automated deployment a...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

Educational Requirements Bachelor of Engineering,BSc,MSc,MCA Service Line Data & Analytics Unit Responsibilities A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and facilitate deployment resulting in client delight. You will develop a proposal by owning parts of the proposal document and by giving inputs in solution design based on areas of expertise. You will plan the activities of configuration, configure the product as per the design, conduct conference room pilots and will assist in resolving any queries related to requirements and solution desi...

About The Role Project Role : Responsible AI Tech Lead Project Role Description : Ensure the ethical and responsible use of artificial intelligence (AI) technologies. Design and deploy Responsible AI solutions; align AI projects with ethical principles and regulatory requirements. Provide leadership, fosters cross-functional collaboration, and advocates for ethical AI adoption. Must have skills : Microsoft Azure Databricks, SQL Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As a Responsible AI Tech Lead, you will ensure the ethical and responsible use of artificial intelligence technologies. Your typical ...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Data Services, Python (Programming Language), Microsoft Azure Databricks, PySpark Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address c...

Role & responsibilities Design, build, and optimize end-to-end data pipelines and ETL/ELT workflows using Databricks, Spark, and DBT . Architect and implement data lakehouse and big data solutions across cloud platforms. Develop and deploy AI/ML models with strong focus on MLOps, automation, and CI/CD pipelines . Implement Infrastructure as Code (IaC) using Terraform, ARM Templates, or CloudFormation for data workloads. Work with Fabric, Palantir, Synapse, Snowflake, and Hadoop ecosystems for large-scale data integration. Collaborate with cross-functional teams to enable data governance, lineage, and data quality frameworks . Manage and optimize cloud-based data solutions on Azure, AWS, and ...

About The Role Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Microsoft Azure Data Services, Microsoft Azure Databricks Good to have skills : NA Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education Summary :Were seeking a visionary Senior Data Engineer to design and build on our Azure-based data platform. This role requires deep expertise in Databricks, Medallion Architecture, and Delta Lake, combined with strong orchestration skills using Azure Data Factory (ADF). Youll also leverage data virtualization technologi...

As an Associate Software Developer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big ...

About The Role Project Role : Application Lead Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact. Must have skills : Microsoft Azure Databricks Good to have skills : NA Minimum 5 year(s) of experience is required Educational Qualification : 15 years full time education Summary :As an Application Lead, you will lead the effort to design, build, and configure applications, acting as the primary point of contact. Your typical day will involve collaborating with various teams to ensure project milestones are met, facilitating discussions to address challenges, and guiding your team in implementing effective solutions. Y...

Title: Data Engineer Practice: Data Engineering & AI Location: Office (Pune / Mohali / Hyderabad) Job Description As a Data Engineer, you will be an integral part of our team, working on data pipelines, data warehousing, and data integration for various analytics and AI use cases. You will collaborate closely with Delivery Managers, ML Engineers and other stakeholders to ensure seamless data flow and accessibility. Your expertise will be crucial in enabling data-driven decision-making for our clients. To thrive in this role, you need to be a quick learner, get excited about innovation and be on the constant lookout to master new technologies as they come up in the Data, AI & Cloud reams. Key...